KNOWLEDGE, SENSORY EXPERIENCE, AND
SENSOR TECHNOLOGY
DANIĂLE DUBOIS, MATT COLER, HEINRICH WĂRTCHE
INCAS3, Dr. Nassaulaan 9, 9401 HJ Assen, the Netherlands
The creation of artifacts is one of the factors that make us human. Artifacts contribute to our continual adaptation to the world by permitting better knowledge and control of it. The focus of this chapter is on the role of one specific kind of artifact: sensors. In contrast to our immediate perception of the world from our senses, sensors provide large amount of reliable measurements of the physical world that enhance human cognitive capacities in overcoming our perceptual limitations. However, ârawâ sensor data require interpretation that relies on different types of expertise and knowledge to provide relevant meaning for human (adaptive) purposes. We suggest that a cognitive approach to understanding the differences between the different types of knowledge provided by current sensors as artifacts and the human senses is of interest. This approach questions the conception of human cognition as an analytic system of processing information from the world rather than as one which interprets and gives meanings to the world. We posit that understanding the differences between human and artificial sensors can shape a new era of technological advancement that is uniquely collaborative insofar as it would rely on the partnership of scientists working in the Humanities and in the Natural Sciences. In this article we provide some data from cognitive research that outline the beginnings of a pluridisciplinary endeavor to conceive sensors which integrate performances of artifacts and the diversity and richness of human cognition, with the goal of transforming so-called âintelligentâ devices into cognitive sensors.
Keywords: senses, sensors, sensory cognition, technological development, experiential and scientific knowledge, information and meaning, information processing, interpretation
1. Introduction
Sensors are one among the many artifacts produced by humans in their continual process of adaptation to the world. Such devices are conceived to enhance human cognitive capacities in perceiving and interpreting sensory stimulation from their physical environment and to acquire knowledge of the physical world to allow a better mastery and control of it. Instead of directly facing the world through their senses (traditionally differentiated as olfaction, audition, vision, gustation, and touch), the use of sensors removes one from direct exposure to the world, preventing adaption through immediate sensory experience. Direct experience is, then, displaced and delegated to sensors. In this way, sensors can be construed as mediators that reliably facilitate the adaptation and adjustment of human behavior within the world.
This âperceptionâ of the world mediated by sensors still requires some knowledge about the sensor itself that enables the interpretation of the âimageâ of the world given as the output from sensors. However, the conception and use of sensors has often been viewed as somehow parallel or analogous to human cognitive abilities and to the development of both individual and shared knowledge of the world.
With that in mind, this contribution addresses differences in human and artificial perception alongside the knowledge it entails. This is achieved with a study of the implications of those differences on a fundamental ground (concerned with the elaboration of different types of knowledge of the world within the different disciplinary domains) as well as on the applied consequences of the development of advanced technologies. Accordingly, this contribution is divided into four sections. In the one which immediately follows, we provide some landmarks relating to the conceptions and uses of sensors, considering them as artifacts. The discussion concentrates on microphones (in 2.1) and thermometers (in 2.2). Thereafter, in 3 we explore the relationship between human senses and sensor technology. Using the development of temporal sensors (from sundials to watches) to frame the analysis, we look at how interactions with technology develop sensory experience and knowledge. From there, the focus is on sight and audition, respectively. The former, (3.1), is devoted to color vision and the technology of color. The former topic is treated with an analysis of the evolution of a conceptualization of colors as pigments to light and now as information. The latter treats the development of sensor technologies for colors, using the Munsell chart as an exemplification. The domain of audition is then tackled in 3.2, which outlines how auditory processing leads to a diversiy of sonic objects, emphasizing, here too, the role of technological devices and human sensory experience in the development of such a diversity . The comparison of those two sensory domains, conceptualized along the current development of digital technologies, elucidates the contribution of physical sciences involved in understanding and mastering the physical world as well as on exploring how human and social sciences explain the adaption of human cognitive abilities to the physical world, either directly through our senses or via the use of technological devices. In the penultimate section, 4, we generalize these examples in interpreting how sensory experience and know-how, as knowledges, including the construction of objectivity, may result from the coupling between sensor technology and cognitive processing (see 4.1). This diversity of knowledges can be accounted for with reference to the instrumental role of artifacts (in 4.1.1) and in close relation with the development of symbolic systems (in 4.1.2). These two sections are complemented with an exploration of how âsensors in useâ illustrate the two types of interactions between humans and machines: models of machines for humans and models of humans for machines (4.2.1 and 4.2.2, respectively). The next section, 4.3, is devoted to the current development of cognitive sensors, which we envision evolving into existence through a pluridisciplinary approach. This section gives an example of cognitive sensors as conceived at INCAS3 through presenting of an expert sensor system which can be thought of as a prototype. The final section, 5, offers some concluding remarks.
Without entering philosophical debates that underlie many of the concepts discussed above, this contribution intends to provide an evaluation of some of the evolutions of the technological development of sensors as artificial senses compared to human (ânaturalâ) ones and the consequences of these evolutions on knowledgesa. From an epistemological point of view, identifying the diversity of knowledges from senses and sensors is a stimulating arena in which to promote a close collaboration between sciences that aim to acquire knowledge of the world âas it isâ (i.e. the physical and natural sciences) and those identifying the world âfor its use, control and enjoyment by humansâ (i.e. human and social sciences), in close cooperation with engineering sciences. This necessitates a shift from the traditional positivist paradigm still present within the cognitivist approach in contemporary cognitive sciences. The productivity of an alternative âsituated cognitive approachâ integrating scientific knowledge from human sciences into technological development is currently one challenge for the Cognitive Systems Group at INCAS3 in its attempts to implement such a diversity of knowledge in developing âcognitive sensorsâ within different applied research questions.
2. Sensors as artifacts: Conceptions and uses
Sensors are such an ubiquitous part of the modern world, completely integrated into our cultures, that it is easy to take them for granted. Sensors are often understood as being superior and more trustworthy than humans insofar as they give the impression of avoiding human âsubjectivityâ in sensing. Furthermore, the information sensors provide (be they numbers on a scale, verbal labels, or something else altogether) is so easily readable for those trained in their use since childhood, that they lend the impression of furnishing a more direct, obvious and objective relation to the properties of the world than our human senses. That is, the obviousness of sensor functioning overlooks their status as artifacts, ignoring that they are historical constructions of human cultures and/or civilizations that rely on the evolution between knowledges and technology.
In the following subsections, we provide some landmarks in the history of sensors to clarify the relation between sensors and the world in contrast to human direct sensory experience. We will begin this discussion by providing some examples of sensors across human sense modalities to situate our work. Through two examples of sensors in common use, in the audio (2.1) and tactile (2.2) domains, we will introduce our rationale and try to make explicit the area of knowledge to which they belong. The first subsection, which deals with microphones will introduce the concept of signal, a polysemic term within the audio (in the technological domain) / acoustic (within the physical domain) / auditory (within the physiological and psychological domains) modalityb. It will further allow us to question how the microphone, contributes to a restrictive definition of acoustic phenomena with regards to human sensing. In the following subsection, we turn to thermometers. There, we will consider the implications of the shift from a direct sensory experience (that is, the sensation of cold or warmth) to the learned interpretation of numbers on a scale as representing temperature on an analog or digital thermometer. The overviews of the microphone and thermometer serve as vehicles to question how these different technologies enhance and overcome limitations of human senses through developing abstract and amodal (i.e. abstracted from any sensory modality) knowledge while the use and interpretations of their outcomes (that is, the data acquired) simultaneously imposes ordinary human experiential processing and learned knowledge. In due course, we develop this idea to show that further work on the relations between different types of knowledge is required and can be accessed mainly through identifying and connecting the different relevant academic scientific domains (natural vs human and social sciences) and integrating a meta-level not only for epistemological issues but also for applied purposes in conceiving and developing new technologies.
2.1. Microphones as filters: from signals to data
Beginning with sensors in the acoustic domain is useful not by virtue of the scientific merit of this topic, but because we will return to this issue in the description of cognitive sensor technology being developed at INCAS3 in section 4.3.2. As stated above, it is important to bear in mind that much of what one may consider âobviousâ about microphones (and for that matter, most sensors in common use) stems from a widespread general familiarity. Familiarity, however, obfuscates the large amount of previous technology and knowledge on which the sensors rely for their conception by engineers and their use by the public (who may share some of that same knowledge).
2.1.1. Filters and signals
To put it simply, when the membrane of a microphone moves, that movement is transformed into electrical signals. The microphone acts as a filter in that it reduces acoustic stimulations to a single analytic phenomenon thereby obstructing complete reconstruction of the original stimulations. The nature of the reduction depends on the qualities of the device (sensitivity, frequency response, linearity, etc.). At first glance, it may be tempting to suggest that these qualities are comparable with the similar properties of the human ear â but this is not so. In any case, subsequent electronic operations may transform this analog information into numerical (digital) code. Nowadays, the development of informatics and computers allows not only the coding and the transmission of the signal (as previously done through telephone technology) but also âinformation processingâ, which includes the processes of recording and storing. Additional processing allows the mimicking (and appearance of mimicking) of certain human auditory processes.
Parallel to the development of microphones as sensors, audio technology aims to create reproduction and restitution systems in order to produce an output signal that is as identical as possible to the input (minimizing or eliminating information loss). However, such efforts in physically exact reproduction from this input is unique from the rich sensory human experience of ânaturallyâ produced sounds.
Shifting from of reproduction to production of sounds intended to restitute a comparable sensory experience in digital musical instruments to that of the traditional musical instruments. It is for this reason that high technology audio system companies invest so much in development of more qualitative aspects of their audio systems. Thus, the electronic music industry attempts to give âexpressivenessâ to digital musical instruments, which lack the fundamental property of traditional mechanical instruments and so often fall short of the expectations of musicians and audiophiles. Similarly, recording companies develop remastering techniques that do not so much âcleanâ the old wax or vinyl analog recordings (even if it is technically possible to do so) as they reproduce some of the qualitative aspects of these âimperfectâ recordings which contribute to the experts' and amateurs' enjoyment.
2.1.2. From signals to data
Microphones can be made more sensitive to wider frequency and intensity ranges than can be perceived with the human ear. They can extend human perception in a way analogous to this same ability in other sensors (cf. the development of thermometers discussed in 2.2 which permit the measurement of extreme temperatures otherwise inaccessible to humans) thereby providing a potentially massive amount of data which require significant processing to be interpreted. One issue in recording and restitution audio systems is the consideration of their role as information providing devices, depending on the tasks they are supposed to fulfill.
The human mind actually reduces the amount of input from the infinite variety of sensory stimuli to give stability to the world. Humans must rely on constructed psychological invariants in their memory to behave coherently. This is mainly achieved through the elaboration of semantic categories1 which permit the infinite variations of stimulations to be understood as identical (or at least similar enough) to belong to the same category.2 Though the data provided by these sensors largely overcomes the human capacities, the difference between human senses and artificial sensors in data processing becomes a major issue for the usability of these sophisticated systems. For example, when hearing some acoustic stimulation, a person immediately identifies âsomethingâ; be it, the noise of a car, a male voice, a song reminiscent of one's childhood, an unintelligible language, etc. Human perception includes some memorized categorical knowledge that allows comparison of the similarities and differences with the incoming stimulation to...