No CrossRef data available.
Article contents
A review of learning
Published online by Cambridge University Press: 07 July 2009
Abstract
Learning is one of the important research fields in artificial intelligence. This paper begins with an outline of the definitions of learning and intelligence, followed by a discussion of the aims of machine learning as an emerging science, and an historical outline of machine learning. The paper then examines the elements and various classifications of learning, and then introduces a new classification of learning based on the levels of representation and learning as knowledge-, symboland device-level learning. Similarity- and explanation-based generalization and conceptual clustering are described as knowledge level learning methods. Learning in classifiers, genetic algorithms and classifier systems are described as symbol level learning, and neural networks are described as device level systems. In accordance with this classification, methods of learning are described in terms of inputs, learning algorithms or devices, and outputs. Then there follows a discussion on the relationships between knowledge representation and learning, and a discussion on the limits of learning in knowledge systems. The paper concludes with a summary of the results drawn from this review.
Information
- Type
- Research Article
- Information
- Copyright
- Copyright © Cambridge University Press 1991
References
Barrow, HG, 1989. “AI, neural networks and early vision”. AISB Newsletter 89 6–25. This is the text of an invited talk at the AISB-89 conference. It aprovides a good summary of the history of the ideas and methods that led to the development of neural networks.Google Scholar
Carbonell, JG, 1982. “Experiential learning in analogical problem solving”. AAAI 168–173. This paper describes how analogical reasoning can be used to generate exemplary solutions to related problems, and more general plans can be induced from these solutions.Google Scholar
Carbonell, JG and Langley, P, 1987. “Machine learning”. In: Shapiro, SC and Eckroth, D (eds.), Encyclopedia of Artificial Intelligence. Wiley Interscience. The authors provide a survey of the history of machine learning, and describe the methods of machine learning task based categories.Google Scholar
Carbonell, JG, Michalski, RS and Mitchell, T, 1983. “An overview of machine learning”. In: Michalski, RS, Carbonell, JG and Mitchell, T (eds.), Machine Learning: An artificial intelligence approach. Morgan Kaufmann. The authors explain the objectives of machine learning, describe its evolution and various learning methods according to the underlying strategy.Google Scholar
Caudell, TP and Dolan, CP, 1989. “Parametric connectivity: training of constrained networks using genetic algorithms”. In: Proceedings of the Third International Conference on Genetic Algorithms, pp 370–374. San Mateo, CA:Morgan Kaufmann. The paper describes how genetic algorithms are used in training neural networks.Google Scholar
Cohen, PR and Feigenbaum, EA (eds.), 1982. The Handbook of Artificial Intelligence, Vol. III. Pitman. This is the first comprehensive handbook of artificial intelligence, covering a wide range of topics. However, I do not know if it has a new edition.Google Scholar
Darden, L, 1987. “Viewing the history of science as compiled hindsight”. The AI Magazine 8(2) 33–42. The author views the history of science as an important source for developing and testing the computational models of scientific discovery. She also provides informal definitions of logical and extralogical methods of inference such as deduction, induction, abstraction, abduction and analogy.Google Scholar
Davis, L, 1989. “Mapping neural networks into classifier systems”. In: Proceedings of the Third International Conference on Genetic Algorithms, pp 375–378. San Mateo, CA:Morgan Kaufmann. The paper describes the functional similarities between classifier systems and neural networks, and argues that any neural network can be transformed into a classifier system that is functionally isomorphic to the former.Google Scholar
De Jong, K, 1988. “Learning with genetic algorithsm: an overview”. Machine Learning 3 121–138. This paper describes how genetic algorithms can be applied to machine learning problems.CrossRefGoogle Scholar
Dietterich, T, 1986. “Learning at the knowledge level”. Machine Learning 1 287–316. The author introduces a theory of knowledge and symbol level learning, and analyzes the methods of some of the learning systems. The author examines the describability of the behaviour of these systems at different levels.CrossRefGoogle Scholar
Dietterich, T and Michalski, RS, 1983. “A comparative review of selected methods for learning from examples”. In: Michalski, RS, Carbonell, JG, and Mitchell, TM (eds.), Machine Learning: An artificial intelligence approach, Morgan Kaufmann. The authors examine various learning systems from the standpoint of the adequacy of their methods of representation, their rules of generalization, computational efficiency, and flexibility and extensibility.Google Scholar
Falkenheiner, B, 1987. “Scientific theory formation through analogical inference”. In: Proceedings of the Fourth International Workshop on Machine Learning, pp 218–229. Los Altos, CA: Morgan Kaufmann. This paper describes how analogy may be used to discover and refine qualitative models of the physical world.CrossRefGoogle Scholar
Fatmi, HA and Young, RW, 1970. Nature 228 (93). The authors define “intelligence” in computational terms, and discuss the implications of their definition.CrossRefGoogle Scholar
Feigenbaum, EA, 1963. “The simulation of verbal behaviour”. In: Feigenbaum, EA and Feldman, J (eds.), Computers and Thought, pp 297–309. McGraw-Hill.Google Scholar
Greiner, R, 1988. “Learning by understanding analogies”. Artificial Intelligence 35 81–125. This paper defines analogical inference, and describes how it can be used in problem solving.CrossRefGoogle Scholar
Harp, SA, Samad, T and Guha, A, 1989. “Towards the genetic synthesis of neural networks”. In: Proceedings of the Third International Conference on Genetic Algorithms, pp 360–369. San Mateo, CA:Morgan Kaufmann. This paper describes how genetic algorithms can be used in training neural networks. The authors evaluate the efficiency of their methods.Google Scholar
Hinton, GE, 1990. “Preface to the special issue on connectionist symbol processing”. Artificial Intelligence 46 1–4. The author draws attention to the representational gulf between neural networks and higher level learning systems, and introduces the reader to the latest attempts to bridge this gulf.CrossRefGoogle Scholar
Holland, JH, 1975. Adaptation in Natural and Artificial Systems. University of Michigan Press. In this important book the author develops his theory of genetic algorithms.Google Scholar
Holland, JH, 1986. “Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems”. In: Michalski, RS, Carbonell, JG and Mitchell, TM (eds.), Machine Learning: An artificial intelligence approach. Morgan Kaufmann. This paper describes the basic features of genetic algorithms, and explains how they can be combined with rule-based methods to provide general purpose inductive learning systems (or classifier systems) with an example.Google Scholar
Hopfield, JJ and Tank, DW, 1986. “Computing with neural circuits”. Science 233 625–633.CrossRefGoogle ScholarPubMed
Hunt, EB and Hovland, CI, 1963. “Programming a model of human concept formation”. In: Feigenbaum, EA and Feldman, J (eds.), Computers and Thought, pp 310–325. McGraw-Hill.Google Scholar
Hunt, EB, 1975. Artificial Intelligence. Academic Press. In this book the author gives a formal description of linear and nonlinear classifiers.CrossRefGoogle Scholar
McDermott, J, 1979. “Learning to use analogies”. In: Proceedings of the Sixth International Conference on Artificial Intelligence, pp 568–576. This paper is one of the early works on analogy. It examines the constituents of analogical reasoning and describes a rule-based system capable of analogical reasoning.Google Scholar
Michalski, RS, 1983. “A theory and methodology of inductive learning”. In: Michalski, RS, Carbonell, JG and Mitchell, TM (eds.), Machine Learning: An artificial intelligence approach. Morgan Kaufmann. In this work, the author views inductive learning as a heuristic search through a space of symbolic descriptions. The descriptions are generated by the application of inductive and deductive inference methods. The author classifies inductive learning into several different types and provides analytical descriptions of each type.CrossRefGoogle Scholar
Michalski, RS, 1986. “Understanding the nature of learning: issues and research directions”. In: Michalski, RS, Carbonell, JG and Mitchell, TM (eds.), Machine Learning. Morgan Kaufmann. In this paper, the author discusses various definitions of learning and formulates the goals and directions in machine learning research. He also examines the current research paradigms in machine learning, and reviews learning strategies.Google Scholar
Michalski, RS and Stepp, RE, 1983. “Learning from observation: conceptual clustering”. In: Michalski, RS, Carbonell, JG and Mitchell, TM (eds.), Machine Learning. Morgan Kaufmann. The authors introduce a detailed description of one of the major methods of learning from observation called “conceptual clustering”.CrossRefGoogle Scholar
Michalski, RS, Amarel, S, Lenat, DB, Michie, D and Winston, PH, 1986. “Machine learning: challenges of the Eighties”. In: Michalski, RS, Carbonell, JG and Mitchell, TM (eds.), Machine Learning. Morgan Kaufmann. The authors discuss the important tasks for machine learning research and the role of machine learning in artificial intelligence. They also discuss the strategies that should be followed in machine learning research in the near future.Google Scholar
Mitchell, T, 1983. “Learning and problem solving”. In: Proceedings of the Eighth International Joint Conference on Artificial Intelligence, pp 1139–1151. Karlsruhe, Germany: Morgan Kaufmann.Google Scholar
Mitchell, T, Keller, RM and Kedar-Cabelli, ST, 1986. “Explanation based generalization: A unifying view”. Machine Learning 1(1) 47–80. The authors describe one of the most productive methods of machine learning: explanation-based generalization, and explain how the method has been used in various learning tasks.CrossRefGoogle Scholar
Naft, J, 1989. “A modified Hopfield net approach to multi-objective design optimization for printed circuit board component placement”. Int. J. Neural Networks Research and Applications 1 78–85. This paper describes an application of neural networks to optimization problems.Google Scholar
Newell, A, 1982. “The knowledge level”. Artificial Intelligence 18(1) 87–127. This is one of the influential papers in artificial intelligence. The author investigates the representational issues of learning and problem solving at several different levels.CrossRefGoogle Scholar
Nilsson, NJ, 1965. Learning Machines: Foundations of trainable pattern-classifying systems. McGraw-Hill. In this book, the author provides a symbolic description of classifiers.Google Scholar
Odetayo, MO and McGregor, DR, 1989. “Genetic algorithms for inducing control rules for a dynamic system”. Proceedings of the Third International Conference on Genetic Algorithms, pp 177–182. San Mateo, CA: Morgan Kaufmann. This paper describes how genetic algorithms are used for automatically inducing control rules for a dynamic physical system.Google Scholar
Porter, B W and Kibler, DF, 1986. “Experimental goal regression: A method for learning problem-solving heuristics”. Machine Learning 1(3) 249–286. In this paper, the authors describe episodic learning, an enhanced version of explanation-based generalization. They explain with examples how the method has been implemented in a program.CrossRefGoogle Scholar
Quinlan, JR, 1983. “Learning efficient classification procedures and their application to chess end games”. In: Michalski, RS, Carbonell, JG and Mitchell, TM (eds.), Machine Learning: An artificial intelligence approach. Morgan Kaufmann.Google Scholar
Rosenblatt, F, 1958. “The perceptron: A probabilistic model for information storage and organization in the brain”. Psychological Review 65 386–407. This paper in one of the earliest works in connectionist systems.CrossRefGoogle ScholarPubMed
Rumelhart, DE and McClelland, JL, 1986. Parallel Distributed Processing: Explorations in the microstructures of cognition. The MIT Press. This book constitutes an important contribution to cognitive science.CrossRefGoogle Scholar
Simon, HA, 1983a. “Why should machines learn?” In: Michalski, RS, Carbonell, JG and Mitchell, TM (eds.), Machine Learning. Morgan Kaufmann. In this paper the author describes the objectives of machine learning.Google Scholar
Simon, HA, 1983b. “Search and reasoning in problem solving”. Artificial Intelligence 21 (1–2) 7–30. In this paper, the author discusses the role of reasoning in problem representation and problem solving.CrossRefGoogle Scholar
Simon, HA and Lea, G, 1974. “Problem solving and rule induction: A unified view”. In: Gregg, L (ed.), Knowledge and Cognition. Erlbaum. In this paper, the authors introduce a general model for learning, and describe its constituents.Google Scholar
Sowa, J, 1987. “Semantic networks”. In: Shapiro, SC (ed.), Encyclopedia of Artificial Intelligence. Wiley Interscience. The author provides a review of semantic networks, including a historical survey, early work, and discusses the relationships between logic, frame representation and semantic networks.Google Scholar
Stirling, L, 1984. “Logical levels of problem solving”. J. Logic Programming 2 151–163. This paper identifies three levels of knowledge necessary for intelligent problem solving: The levels of domain knowledge, method and strategies, and planning. The author relates these levels to the distinction between object and meta languages.CrossRefGoogle Scholar
Whitley, D, Starkweather, T and Fuquay, D, 1989. “Scheduling problems and traveling salesmen: The genetic edge recombination operator”. Proceedings of the Third International Conference on Genetic Algorithms, pp 133–140. San Mateo, CA: Morgan Kaufmann. This paper describes an application of genetic algorithms to one of the classic problems in artificial intelligence.Google Scholar
Winston, PH, (ed.), 1975. The Psychology of Computer Vision, pp 157–209. McGraw-Hill. Learning and discovery systemsGoogle Scholar
Akl, S, 1987. “Checkers-playing programs”. In: Shapiro, SC (ed.), Encyclopedia of Artificial Intelligence. Wiley Interscience. Various checkers playing programs and their learning methods are described.Google Scholar
Friedland, P, 1979. “Knowledge-based experiment design in molecular genetics”. In: Proceedings of the Sixth International Joint Conference on Artificial Intelligence,, pp 285–287. The paper describes MOLGEN, a rule-based system that designs experiments in molecular genetics.Google Scholar
Kocabas, S, 1990. “Conflict resolution as discovery in particle physics”. Machine Learning (in press). This paper describes an impasse resolution system that models certain discoveries in particle physics.Google Scholar
Kottai, RM and Bahill, T, 1989. “Expert systems made with neural networks”. Int. J. Neural Networks Research and Applications 1 211–226. This paper examines the similarities and differences between expert systems built on neural networks and conventional shells.Google Scholar
Kulkarni, D and Simon, HA, 1988. “The processes of scientific discovery: The strategy of experimentation”. Cognitive Science 12 139–175. This paper describes a theory driven system that models empirical discovery in biochemistry.CrossRefGoogle Scholar
Laird, JE, Newell, A and Rosenbloom, PS, 1986. “Chunking in SOAR”. Machine Learning 1 11–46. The paper describes a complex general system SOAR, which incorporates different learning methods.CrossRefGoogle Scholar
Laird, JE, Rosenbloon, PS and Newell, A, 1987. “SOAR: An architecture for general intelligence”. Artificial Intelligence 33 1–64. This paper describes SOAR's control structure and its capabilities.CrossRefGoogle Scholar
Langley, P, 1978. “BACONI: A general discovery system”. In: Proceedings of the Second National Conference of the Canadian Society for Computational Studies. The paper describes the earliest version of BACON, a well known quantitative discovery program.Google Scholar
Langley, P, 1981. “Data-driven discovery of physical laws”. Cognitive Science 5 31–54. In this paper, the author describes the general heuristics that are used in quantitative discovery.Google Scholar
Langley, P, Simon, HA, Bradshaw, GL and Zutkow, JM, 1987. Scientific Discovery: Computational explorations of the creative processes. The MIT Press. This book is an excellent introduction to the current research into the computational modelling of scientific discovery. It describes several discovery systems such as GLAUBER, STAHL and BACON.CrossRefGoogle Scholar
Lenat, DB, 1979. “On automated scientific theory formation: a case study using the AM program”. In: Hayes, J, Michie, D and Mikulich, LI (eds.), Machine Intelligence 9, pp 251–283. Halstead. The author describes one of the most important discovery programs, AM, which rediscovers many arithmetical concepts starting with the basic concepts of set theory. The paper explains how AM's heuristics work.Google Scholar
Lenat, DB, 1983. “EURISKO: A program that learns new heuristics and domain concepts”. Artificial Intelligence 21(1–2) 61–98. In this paper, the author describes EURISKO, a theory driven discovery system developed as a successor to the AM program. EURISKO has the additional capability of discovering new problem solving heuristics.CrossRefGoogle Scholar
Nordhausen, B and Langley, P, 1987. “Towards an integrated discovery system”. In: Proceedings of the Tenth International Joint Conference on Artificial Intelligence, pp 198–200. In this interesting paper, the authors describe IDS, a discovery system that integrates qualitative and quantitative methods. IDS is the first fully implemented discovery system with the capability of sensing and changing its environment.Google Scholar
O'Rorke, P, Morris, S and Schulenburg, D, 1990. “Theory formation by abduction”. In: Shrager, J and Langley, P (eds.), Computational Models of Scientific Discovery and Theory Formation. Morgan Kaufmann. In this paper, the authors view abduction as a general method for theory revision. They describe a discovery program, AbE, and its application to the paradigm shift in the eighteenth century chemistry.Google Scholar
Rajamoney, SA, 1990. “A computational approach to theory revision”. In: Shrager, J and Langley, P (eds.), Computational Models of Scientific Discovery and Theory Formation. Morgan Kaufmann. This paper describes a qualitative theory revision program with the capability of designing experiments to resolve contradictions in its explanations.Google Scholar
Rose, D and Langley, P, 1986. “Chemical discovery as belief revision”. Machine Learning 1 423–452. The paper describes one of the early examples of theory revision systems which has been applied to eighteenth century chemistry.CrossRefGoogle Scholar
Thagard, P, 1988. Computational Philosophy of Science. The MIT Press. This is an important book that relates machine learning and discovery to philosophy of science. It also describes a program that models theoretical discovery.CrossRefGoogle Scholar
Thagard, P and Holyoak, K, 1985. “Discovering the wave theory of sound: Inductive inference in the context of problem solving”. In: Proceedings of the Ninth International Joint Conference on Artificial Intelligence, pp 610–612. In this paper the authors describe a program that models the discovery of theory of sound.Google Scholar
Zytkow, JM and Simon, HA, 1986. “A theory of historical discovery: The construction of componential models”. Machine Learning 1 107–137. The authors describe a data driven system that simulates the discoveries of the componential models of substances in the seventeenth century chemistry.CrossRefGoogle Scholar
Brewka, G, 1987. “The logic of inheritance in frame systems”. In: Proceedings of the Tenth International Joint Conference on Artificial Intelligence, pp 438–488. This paper presents a formal description of inheritance in frame systems, together with a formalism for transforming knowledge represented in a frame into predicate logic statements.Google Scholar
Carnap, R, 1958. Introduction to Symbolic Logic and its Applications. Dover Publications. This book is interesting in its being one of the earliest and best examples of the linguistic philosophical approach to logic.Google Scholar
Charniak, E and McDermott, J, 1985. Introduction to Artificial Intelligence. Addison Wesley. This book is one of the well known textbooks in artificial intelligence.Google Scholar
Engelmore, R and Morgan, T (eds.), 1988. Blackboard Sytems. Addison Wesley. This book provides a detailed description of the methods used blackboard systems, and describes a number of applications.Google Scholar
Fikes, R and Kehler, T, 1985. “The role of frame-based representation in reasoning”. Commun. ACM 28(9) 904–920. The authors describe the elements and the methods of the frame-based representation with examples. The paper does not include the new techniques used in frame systems.CrossRefGoogle Scholar
Forbus, KD, 1984. “Qualitative process theory”. Artificial Intelligence 24 85–168. The author describes the principles of qualitative representation of physical processes.CrossRefGoogle Scholar
Frost, R, 1986. Introduction to Knowledge Base Systems. Collins Professional and Technical Books. This is one of the earliest and most detailed books on knowledge-based systems.Google Scholar
Hayes-Ruth, F, 1985. “Rule-based systems”. Commun ACM 26(9) 921–932. This paper describes rule-based representation and its methods with examples.CrossRefGoogle Scholar
Kocabas, S, 1989. Functional Categorization of Knowledge: Applications in modeling scientific research and discovery. PhD thesis, Department of Electronic and Electrical Engineering, King's College London. This work introduces a methodology for organizing knowledge into functional categories, so that different types of inference can be implemented more efficiently.Google Scholar
Kowalski, R, 1979. Logic for Problem Solving. North Holland. The book introduces first order predicate logic as a language for general problem solving.Google Scholar
Kowalski, R, 1984. “Logic programming in the fifth generation”. The Knowledge Engineering Review 1(1) 26–38. The paper describes how Prolog is used in knowledge representation and problem solving.CrossRefGoogle Scholar
Minsky, M, 1977. “Frame system theory, thinking: readings in cognitive science”. In: Johnson-Laird, PN and Wason, PC (eds.), Open University Set Book. Cambridge University Press, pp 355–376. This is one of the most influential papers in the development of knowledge-based systems.Google Scholar
Brugge, JA and Buchanan, BG, 1989. “Evolution of a knowledge based system for determining structural components of proteins”. Expert Systems 6(3) 144–156. The authors describe a knowledge-based system, ABC, which determines the structural components of proteins. The system is based on the blackboard control architecture.CrossRefGoogle Scholar
Buchanan, BG and Feigenbaum, EA, 1978. “Dendral and Meta-Dendral: Their application dimension”. Artificial Intelligence 11 5–24. The paper describes DENDRAL, which constitutes one of the earliest successful implementations of the rule-based methods in scientific research, and Meta-DENDRAL, a system that discovers the heuristics to be used by the former.CrossRefGoogle Scholar
Feigenbaum, EA, 1989. “An interview”. Expert Systems 6(2) 112–115. The author discussed the current developments in artificial intelligence, and makes projections into the future of the discipline is a leading researcher.CrossRefGoogle Scholar
Lenat, DB, Prakesh, M and Shepherd, M, 1986. “CYC: using common sense knowledge to overcome brittleness and knowledge acquisition bottlenecks”. The AI Magazine 7(4) 65–85. This paper describes some of the design ideas of one of the largest research projects in artificial intelligence. The project is called CYC, and aims at building a general knowledge base including a large amount of commonsense knowledge.Google Scholar
Leant, DB, and Feigenbaum, EA, 1987. “On the threshold of knowledge”. In: Proceedings of the Tenth International Joint Conference on Artificial Intelligence, pp 1173–1182. In this paper, the authors discuss the problems of building complex intelligent systems. Even though it contains some controversial views, the paper is an important contribution to artificial intelligence and should not be ignored.Google Scholar
Lenat, DB and Guha, RV, 1991. Building large Knowledge Based Systems: Representation and inference in the CYC Project. Addison Wesley. This is a new book in which the authors expand on Lenat et al.'s (1986) paper, and describe the knowledge representation and inference methods of CYC. Should not be missed by anyone who is interested in building large knowledge bases.Google Scholar
- 11
- Cited by