Neural Networks

12 Key excerpts on "Neural Networks"

• 

  eBook - PDF

  Neural Networks

  An Introductory Guide for Social Scientists

  • G David Garson(Author)
  • 1998(Publication Date)
  • SAGE Publications Ltd

    (Publisher)

  2 The Terminology of Neural Network Analysis This chapter lays the groundwork for the rest of the monograph by providing a terminological tour of Neural Networks. Synonyms for Neural Networks include neurocomputers, artificial neural systems (ANS), and natural or artificial intelligence. In fact, synonyms abound in such other terms as parallel distributed processing systems, connectionist systems, computational neuroscience, dynamical computation systems, adaptive systems, neural circuits and collective decision circuits. Neural network approaches are inspired by biology, with components loosely analogous to the axons, dendrites, and synapses of a living thing. As Figure 2.1 illustrates, in biological Neural Networks dendrites collect signals which they feed to the neuron, which processes it by sending a spike of electrical current along an axon, discharging it at a synapse connecting it to other neurons, which in turn are excited or inhibited as a result. In an artificial neural network, input signals are sent to a neural processing entity, also called a neuron, which after processing sends an output signal on to later neurons in the network. While artificial Neural Networks process events several orders of magni-tude faster than the best current computers, the human brain has some 10 billion neurons and perhaps 60 trillion synapses (Shepherd and Koch, 1990), giving it a complexity and what Faggin (1991, cited in Haykin, 1994: 1) terms an energetic efficiency about 10 orders of mag-nitude greater than current artificial Neural Networks. The biological analogy centres on the fact that Neural Networks do not operate on programmed instruction sets as do statistical packages. Rather they pass data through multiple processing entities which learn and adapt according to patterns of inputs presented to them. Data are not stored in these entities, nor is an 'answer ' stored at a particular address in the computer's memory.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Language and Complex Systems

  • William A. Kretzschmar, Jr(Authors)
  • 2015(Publication Date)
  • Cambridge University Press

    (Publisher)

  Computational neuroscience and connectionists employ neural network models in order to understand, say, the process of language acquisition among children. In the experiment presented here, evidence from survey research will be submitted to neural network algorithms in order to understand how a speaker might perceive and process such evidence over time. According to this model speech in the brain is not a single process but a massively interconnected one, a neural network. According to the prevailing connectionist model (e.g. Edelman 1987; Pulverm¨ uller 2003), neurons develop billions of connections in a massively parallel network, in which no action or perception could be considered to have a single or simple “impulse,” as earlier theory suggested. Bybee’s proposal that phonological information is stored on the basis of words (2001) must be understood in neuroscience not as the brain having some single physical location to store a word, not as a representation, but rather as the brain having a collection of interconnected neuronal pathways whose activation is related to a word. In hearing, for instance, Saussure’s (1916 (1986)) seemingly unitary “impulse” corresponds to thousands of neural fibers (specialized hair cells) in the inner ear that vibrate in response to specific frequencies, and in turn activate neurons that fire repeatedly and transmit signals Models in cognitive science 137 to the brain. Thus the brain receives patterns in response to a speech sound, changing every moment, that might be compared to the pixels on a visual monitor, some turned off and some turned on at any given moment in response to the stimulus of physical sound waves; just as what we consider to be a picture on a monitor actually consists of a configuration of pixels, we can recognize a “speech sound” as a particular configuration of neurons firing in response to input from the neural fibers.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  The Froehlich/Kent Encyclopedia of Telecommunications

  Volume 13 - Network-Management Technologies to NYNEX

  • Fritz E. Froehlich, Allen Kent(Authors)
  • 2021(Publication Date)
  • CRC Press

    (Publisher)

  Neural Networks and Their Application in Communications

  Introduction

  The human nervous system, as the most complex system of information processing, is the focus of intensive investigation by the life and systems scientists. Understanding of the functioning of this system will help life scientists to explain the causes and find therapeutic procedures for various neural disorders. On the other hand, study of the biological nervous system helps systems scientists and engineers to devise mechanisms that mimic specific features of the nervous system.

  Devices and units realizing such mechanisms are called artificial Neural Networks, neural systems, connectionist systems, or neurocomputers. These are equivalent terms used by various authors in the field of Neural Networks to refer to such systems and mechanisms. Artificial Neural Networks are advantageous in some regards compared to conventional systems of information processing and control. For instance, in terms of parallelism, Neural Networks are far superior to serial computers.

  The field of Neural Networks is based on mathematical modeling of the function of the mammalian nervous system in such a way that a specific feature of the nervous system is emulated by the model. Replication of the behavior of the nervous system with some degree of accuracy is therefore the ultimate goal of neural network scientists. This is pursued both at the single neural cell (neuronal) level, as well as at the systems level. It should be emphasized that successful utilization of artificial Neural Networks depends on the application needs versus the limitations in the capabilities of the utilized network and the technology used in the implementation of that network. In some applications, use of a specific neural network is justifiable; in others, their limitations outweigh their advantages.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Neural Networks for Applied Sciences and Engineering

  From Fundamentals to Complex Pattern Recognition

  • Sandhya Samarasinghe(Author)
  • 2016(Publication Date)
  • Auerbach Publications

    (Publisher)

  These efforts led to the development of artificial Neural Networks, which are widely used for solving a variety of problems in many fields remotely related to neurobiology, such as ecology, biology, engineering, agriculture, environ-mental and resource studies, and commerce and marketing. In this book, the term “Neural Networks” represents artificial Neural Networks, and “neurons” denotes artificial neurons. In this chapter, biological and artificial Neural Networks are intertwined, but the aim is to demonstrate the development of artificial Neural Networks that are useful for practical problem solving. The next section presents an incremental introduction to neural network concepts, neuron models, mechanisms of learning from data, and other fundamental issues of Neural Networks, so that the reader can better appreciate and understand the Neural Networks in the rest of the book. These discussions will also facilitate the exploration of deeper aspects of the nature of data modeling as relevant to many applied fields of study. 2.5 Neuron Models and Learning Strategies Neural computing has undergone several distinct stages. Early attempts occurred from the beginning of the 20th century to about 1969; 1969 to 1982 were quieter years, and 1982 marks the resurgence of activities that propelled a growth of Neural Networks that continues to this day [3] . This section highlights some of the important conceptual developments that are important for understanding and applying Neural Networks. During the early 20th century, William James, an eminent American psychologist, provided two important clues to neural modeling: (1) If two neurons are active together, or in immediate succession, then on reoccurrence they repeatedly excite each other and the intensity between them grows; (2) the amount of activity of a neuron is the sum of the signals it receives, with signals being proportional to the strength of the connection through which a signal is received [7] .

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Introducing Linguistics

  Theoretical and Applied Approaches

  • Joyce Bruhn de Garavito, John W. Schwieter(Authors)
  • 2021(Publication Date)
  • Cambridge University Press

    (Publisher)

  15.1 What Is Neurolinguistics? The area of linguistics which studies the biological and cognitive bases of language is neurolinguistics. In other words, neurolinguistics studies language and the brain. This branch of linguistics and neuroscience allows us to learn more about the physi- ological mechanisms which the brain uses for language. Neurolinguists investigate research questions such as: • How is language represented in the brain? • Is there a place in the brain where language is primarily located? • What are the effects on language of trauma to, or degeneration of, certain areas of the brain? What are the recovery patterns for lost language abilities? • What makes the human brain specialized and advanced enough for language that other species’ brains lack? • Does language use the same parts of the brain for other non-linguistic tasks such as playing a musical instrument or doing mathematics? • How is the bilingual brain different from the monolingual brain? 15 Neurolinguistics Language and the Brain John W. Schwieter 536 Neurolinguistics In addition to being highly related to psycholinguistics, neurolinguistics also informs the main branches of linguistics by exploring phenomena such as how the brain: • separates the speech we hear from background noise (phonetics); • represents the sound system (phonology); • stores and accesses morphemes (morphology); • combines words into phrases (syntax); and • uses structural and contextual information to comprehend language (semantics). PAUSE AND REFLECT 15.1 Have you ever heard that one side of the brain is more responsible for language? If so, which side is it? Later in the chapter, we will see whether or not this is true. 15.2 The Human Brain and Language Is language what makes us human? Although being human may have other special- ized traits, our ability for language does set us apart from other species. These superior language abilities exist because of our highly developed brain.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Computational Neural Networks for Geophysical Data Processing

  • M.M. Poulton(Author)
  • 2001(Publication Date)
  • Pergamon

    (Publisher)

  The field of computational Neural Networks tries to walk the fine line between preserving the richness and complexity of the biological associative memory model while using the language and logic of mathematics. Table 2.1 Impact of neurophysiological developments and advances in cognitive science on development of computational neura I networks. . . . . . Year Advance in biological / psychological understanding of brain Contribution to computational Neural Networks 1943 Mathematical description of a neuron McCulloch-Pitts Neuron 1949 Formulation of learning mechanism in the Hebbian learning brain Connectionist theories of sensory physiology Cortical physiology 1958 Perceptron 1973 1977 1981 1987 1991 Speech perception Use of non-linear threshold similar to neural activation function Early visual systems Visual perception Self Organizing Maps, Adaptive Resonance Theory Bi-directional associative memories Back-propagation Computer chip-based networks Hierarchical / modular networks REFERENCES 25 REFERENCES Cooper, L., 1973, A possible organization of animal memory and learning, in B. Lundquist, B., and Lundquist, S., Eds., Proceedings of the Nobel Symposium on Collective Properties of Physical Systems: Academic Press, 252-264. Fischbach, G., 1992, Mind and Brain: Scientific American, 267, 48-59. von Neumann, J., 1958, The Computer and the Brain: Yale University Press. This Page Intentionally Left Blank 27 Chapter 3 Multi-Layer Perceptrons and Back-Propagation Learning Mary M. Poulton 1. VOCABULARY The intent of this chapter is to provide the reader with the basic vocabulary used to describe Neural Networks, especially the multi-layer Perceptron architecture (MLP) using the back- propagation learning algorithm, and to provide a description of the variables the user can control during training of a neural network. A more detailed explanation of the mathematical underpinnings of many of the variables and their significance can be found in Bishop (1995) or Masters (1993).

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Philosophy of Psychology

  A Contemporary Introduction

  • Jose Luis Bermudez(Author)
  • 2004(Publication Date)
  • Routledge

    (Publisher)

  There are limits, of course, to what can be shown by a single example – and neural network models of language acquisition are deeply controversial. But even with these caveats it should be clear how the tools provided by artificial Neural Networks provide a way of implementing the co-evolutionary approach to thinking about the mind. Using artificial Neural Networks to model cognitive tasks offers a way of putting assumptions about how the mind works to the test – the assumption, for example, that the process of learning a language is a process of forming and evaluating hypotheses about linguistic rules. The test is, of course, in a sense rather contrived. As we saw earlier in the section, artificial Neural Networks are biologically plausible in only the most general sense. But, according to proponents of the artificial Neural Networks approach, to complain about this would be to misunderstand the point of the exercise. The aim of neural network modeling is not to provide a model that faithfully reflects every aspect of neural functioning, but rather to explore alternatives to dominant conceptions of how the mind works. If, for example, we can devise artificial Neural Networks that reproduce certain aspects of the typical trajectory of language learning without having encoded into them explicit representations of linguistic rules, then that at the very least suggests that we cannot automatically assume that language learning is a matter of forming and testing hypotheses about linguistic rules. We should look at artificial Neural Networks, not as attempts faithfully to reproduce the mechanics of cognition, but rather as tools for opening up novel ways of thinking about the mind and how it works.

  We will be exploring the details and plausibility of this new way of thinking about the mind in subsequent chapters, but it is worth sketching out some of the broad outlines now to round off the presentation of the neurocomputational mind. An initial clue is provided by the central feature of the models of past tense acquisition that we have been considering. One of the key tenets of the neurocomputational approach to the mind is to downplay the role in cognition of explicit representations. Traditional approaches to the mechanics of cognition view cognition as a process of rule-governed manipulation of symbols. This is particularly clear on the representational picture, according to which all cognition involves transforming symbolic formulae in the language of thought according to rules operating only on the formal features of those formulae. This way of thinking about cognition rests, of course, on it being possible to distinguish within the system between the representations on which the rules are exercised and the rules themselves. But this distinction comes under pressure in artificial Neural Networks. The only rules that can be identified in these networks are the rules governing the spread of activation values forwards through the network and the propagation of error backwards through the network. There is nothing in either of the two models of past tense acquisition corresponding to the linguistic rule that the past tense is formed by adding the suffix ‘– ed’ to the root of the verb. Nor are there any identifiable representations of the past tenses of irregular verbs. The network’s “knowledge” of the relevant linguistic rules lies in the distribution of weights across all the connections in the entire network. There is no sense in which its “knowledge” that the past tense of ‘go’ is ‘went’ is encoded separately from its knowledge that the past tense of ‘give’ is ‘gave’. There are no discrete representations within the system corresponding to the individual linguistic rules in terms of which we, as external observers, would characterize how the language works.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Grokking Deep Learning

  • Andrew W. Trask(Author)
  • 2019(Publication Date)
  • Manning

    (Publisher)

  Chapter 11. Neural Networks that understand language: king – man + woman == ?

  In this chapter

  • Natural language processing (NLP)
  • Supervised NLP
  • Capturing word correlation in input data
  • Intro to an embedding layer
  • Neural architecture
  • Comparing word embeddings
  • Filling in the blank
  • Meaning is derived from loss
  • Word analogies

  “Man is a slow, sloppy, and brilliant thinker; computers are fast, accurate, and stupid.”

  John Pfeiffer, in Fortune, 1961

  What does it mean to understand language?

  What kinds of predictions do people make about language?

  Up until now, we’ve been using Neural Networks to model image data. But Neural Networks can be used to understand a much wider variety of datasets. Exploring new datasets also teaches us a lot about Neural Networks in general, because different datasets often justify different styles of neural network training according to the challenges hidden in the data.

  We’ll begin this chapter by exploring a much older field that overlaps deep learning: natural language processing

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Exploring Neural Networks with C#

  • Ryszard Tadeusiewicz, Rituparna Chaki, Nabendu Chaki(Authors)
  • 2017(Publication Date)
  • CRC Press

    (Publisher)

  Chapter 1

  Introduction to Natural and Artificial Neural Networks

  1.1 Why Learn about Neural Networks?

  Here we will talk about the development of artificial Neural Networks that were derived from examinations of the human brain system. The examinations were carried out for years to allow researchers to learn the secrets of human intelligence. Their findings turned out to be useful in computer science. This chapter will explain how the ideas borrowed from biologists helped create artificial Neural Networks and continue to reveal the secrets of the human brain.

  This chapter discusses the biological bases of artificial Neural Networks and their development based on examinations of human brains. The examinations were intended to find the basis of human intelligence and continued secretly for many years for reasons noted in the next section. Subsequent chapters will explain how to build and use Neural Networks.

  As you already know, Neural Networks are easy to understand and use in computer software. However, their development was based on a surprisingly complex and interesting model of the nervous system in a biological model. We could say that Neural Networks are simplified models of some functions of the human brain (Figure 1.1 ).

  Figure 1.1

  Human brain: a source of inspiration for neural network researchers.

  1.2 From Brain Research to Artificial Neural Networks

  The intricacies of the brain have always fascinated scientists. Despite many years of intensive research, we were unable until recently to understand the mysteries of the brain. We are now seeing remarkable progress in this area and discuss it in Section 1.3. In the 1990s, when artificial Neural Networks were developed, much less information about brain functioning was available. The only known facts about the brain’s workings related to the locations of the structures responsible for vital motor, perception, and intellectual functions (Figure 1.2 ).

  Figure 1.2

  Localization of various functions within a brain. (Source: http://avm.ucsf.edu/patient_info/WhatIsAnAVM/images/image015.gif

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Neuromimetic Semantics

  Coordination, quantification, and collective predicates

  • Harry Howard(Author)
  • 2010(Publication Date)
  • Elsevier Science

    (Publisher)

  Chapter 10 Networks of real neurons This chapter takes a closer look at real Neural Networks, namely those for language and episodic memory. Given all of the speculation that we have indulged in about how logical coordinators, logical quantifiers, and collective predicates should be represented neurologically, it is about time that we looked at the areas of the brain that are held to be responsible for them. Unfortunately, the results of this investigation will be disheartening. Current techniques do not have the resolution to reveal how the human brain deals with such fine-grained aspects of language. On a more positive note, much more is known about episodic memory, and we will weave it into an analysis of how the two arguments of the logical operators are bound together by correlation. 10.1. NEUROLINGUISTIC NETWORKS 10.1.1. A brief introduction to the localization of language Dronkers, Pinker and Damasio, 2000, p. 1174, state the problem quite succinctly: The lack of a homologue to language in other species precludes the attempt to model language in animals, and our understanding of the neural basis of language must be pieced together from other sources. By far the most important source has been the study of language disorders known as aphasias, which are caused by focal brain lesions that result, most frequently, from stoke or head injury. The specific areas of the cerebral cortex responsible for linguistic functions were initially identified from post-mortem analyses of restricted lesions that correlated with a linguistic deficit during the patient's lifetime. 10.1.1.1. Broca's aphasia and Broca's region Post-mortem studies of patients with slow or non-fluent speech but unimpaired comprehension enabled Paul Broca to identify the posterior third of the left inferior frontal gyrus as the seat of speech production, see Broca (1861), and Schiller, 1992, pp. 186-7, and Ryalls and Lecours (1996) for historical perspective.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Modeling Language, Cognition And Action - Proceedings Of The Ninth Neural Computation And Psychology Workshop

  • Angelo Cangelosi, Guido Bugmann, Roman M Borisyuk(Authors)
  • 2005(Publication Date)
  • World Scientific

    (Publisher)

  Neural Computation, 8, 1135-1178. Christiansen, M. H., Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23(2), 157-205. Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179-211. Griining, A. (2004). Neural Networks and the Complexity of Languages. Ph.D. thesis, School of Mathematics and Computer Science, University of Leipzig. Hopcroft, J. E., Ullmann, J. D. (1979). Introduction to Automata Theory, Lan- guages, and Computation. Mass.: Addison-Wesley. Moore, C. (1998). Dynamical recognizers: Real-time language recognition by analog computers. Theoretical Computer Science, 201. Pollack, 3. B. (1991). The induction of dynamical recognizers. Machine Learning, Rodriguez, P. (2001). Simple recurrent networks learn context-free and context- Savitch, W . J., Bach, E., Marsh, W. (Eds.) (1987). The Formal Complexity of Siegelmann, H. T., Sontag, E. D. (1991). On the computational power of neural 7, 227-252. sensitive languages by counting. Neural Computation, 13, 2093-2118. Natural Language. Dordrecht: D. Reidel Publishing Company. nets. Journal of Computer and System Sciences. This page intentionally left blank This page intentionally left blank THE ACTIVE ROLE OF PROPER NAMES: EVIDENCE FROM NEURAL NETWORK EXPERIMENTS AND PHILOSOPHY OF LANGUAGE CONSIDERATIONS BARBARA GIOLITO University of Eastern Piedmont “A. Avogadro ”, via Duomo 6, 13100 Vercelli,Italy bgiolito@lett. unipmn. it The present work considers the possibility that names play an active role in language, as a point of attention around which properties can be clustered. The contribution of neural network experiments of linguistic asks is here discussed in support of such an hypothesis. 1. In philosophy of language, the cluster concept theory is one of the most influential descriptivist theories of proper names. This was first proposed by Wittgenstein (1953) and then developed by Searle (1958).

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Toward an Evolutionary Biology of Language

  • Philip Lieberman(Author)
  • 2006(Publication Date)
  • Belknap Press

    (Publisher)

  C H A P T E R 4 The Neural Bases of Language A t best, any present account of the neural bases of human lan-guage is tentative. However, I will attempt to provide an over-view of some current views on the nature of the neural bases of hu-man language. My focus is on the cortical-striatal-cortical circuits that yield the reiterative ability that, according to Hauser, Chomsky, and Fitch (2002), confers human recursive syntax. But the stud-ies that I review demonstrate that this creative faculty also plays a critical role in human speech and confers the flexibility of human thought processes. I present evidence that this faculty is linked and probably derives from elements of neural circuits that regulate mo-tor control. Many aspects of these circuits are still a mystery, so I will not attempt to provide a solution to how the mind or brain “works.” But reiteration, in itself, cannot be the “key” to language; without words it would be impossible to convey thoughts. There-fore, I also discuss studies that are exploring the neural bases of the brain’s dictionary, as well as some aspects of the neural control of speech production, because words must be conveyed. Some related issues such as neural plasticity and the lateralization of the brain are also noted. I start by reviewing well-established facts concerning the struc-130 ture of the brain and then attempt to explain distributed Neural Networks, which appear to be reasonable models for associative learning and other local operations that occur in particular neural structures. The chapter also briefly reviews procedures that are com-monly employed in neurophysiologic studies. I present evidence for cortical-striatal-cortical circuits that regulate motor control, syn-tax, and cognition, including studies of Broca’s syndrome, Parkin-son’s disease, and hypoxia. The findings of these studies contradict traditional theory by showing that Broca’s and Wernicke’s cortical areas are not the brain’s language organs.

  Sign up to read

  Learn more about book