Cognitive granularity: A new perspective over autistic and non-autistic styles of development


  • This work was supported by JSPS KAKENHI Grant Number 21330154.

Correspondence concerning this article should be sent to: Hideki Kozima, Department of Design Informatics, School of Project Design, Miyagi University, Gakuen, Taiwa-cho, Kurokawa-gun, Miyagi 981-3298, Japan. (E-mail:


Individuals with autism generally show better performance on operating physical objects than in communicating with people. However, we lack a plausible model of autism that explains why their physical and social capabilities develop in separate and unbalanced ways. This paper investigates this question from the viewpoint of “cognitive granularity,” which refers to the size of the basic elements operable in one's cognitive system. While it is constrained by one's perceptual and motor resolution, cognitive granularity determines the level of abstraction at which one can efficiently predict and control the physical and social world. Recent findings in autism research, including preference for causal predictability and abnormalities in neuroanatomical density, suggest that individuals with autism have finer cognitive granularity; they live in a different “Umwelt” from that which non-autistic people experience. The difference in cognitive granularity explains not only autistic individuals' unbalanced development as well as their difficulty in understanding others' minds, but also the spectrum of developmental styles in the entire population. Finally, from this unified perspective, we also discuss possible therapeutic interventions for autism.

Frith (2003) described autism as an “enigma,” but it is also enigmatic that non-autistic people can naturally communicate with each other in verbal and nonverbal ways. Research on autism has been trying hard to elucidate not only the nature of the developmental disorder, but also how and why typically developing people effortlessly learn to share language and communicate with each other in various ways. This paper proposes a new viewpoint to understand what causes the differences between autistic and non-autistic populations, and what both have in common.

Autism is a behavioral, developmental, and neurophysiological disorder characterized by a triad of impairments in social interaction, social communication, and social imagination (American Psychological Association, 1994; Wing, 1981). The impairment in social interaction includes difficulty in exchanging and sharing of interests and activities with others, which is often marked by absent or unusual eye contact and gestures. The impairment in social communication encompasses delay or failure in speech development, particularly that of pragmatic conversation, and use of stereotyped or repetitive speech. Also, the impairment in social imagination refers to, for example, inflexibility in maintaining diversity of interests and behaviors, stereotyped and sometimes meaningless actions and routines, and difficulty in coping with novel situations.

Previous studies of autism have described autism mainly as a developmental disorder of understanding the emotions and intentions of others. Being triggered by the findings that individuals with autism failed simple false-belief tasks, or “theory of mind” tests (Baron-Cohen, Leslie, & Frith, 1985), researchers described autism in terms of the social deficits that might stem from impairments in social perception, particularly of joint attention and reading mental states from facial and bodily gestures (Baron-Cohen, 1995).

In contrast, individuals with autism show relatively good performance on operating physical objects, such as various machines (Dautenhahn, Werry, Rae, Dickerson, Stribling, & Ogden, 2002). Physical objects provide accessible and relatively fixed cues for manipulation (Norman, 1988). Although their interests and activities are often restricted to specific aspects, individuals with autism are generally good at systematically understanding things and events in terms of causal and deterministic mechanisms (Baron-Cohen, 2002). Previous studies of autism, however, have not given plausible explanations for this relatively good capability in the physical realm or for the restricted range of interest and activity.

This paper investigates the underlying mechanism of how autistic individuals' capabilities for social and physical interaction develop in such separate and unbalanced ways, which has not been explained in a concordant way with the notion of other major symptoms. For this purpose, we take the viewpoint of “cognitive granularity,” which refers to the size of the basic units in one's cognitive processes, including understanding, memory, and mental operations, about the physical and social world. Cognitive granularity is constrained partly by perceptual resolution stemming from biological commonality and diversity, and partly by the sociolinguistic resolution to articulate our experience. Meanwhile, cognitive granularity determines the level of abstraction at which we efficiently predict and control the physical and social world.

From the viewpoint of cognitive granularity, we reformulate the observed collection of impairments and their underlying mechanisms in autism, including those related to the separate and unbalanced development in physical and social realms. Recent neuroanatomical research has reported that individuals with autism have a higher density of cortical processing elements (Buxhoeveden & Casanova, 2002), which suggests that they have finer cognitive granularity. This implies that autistic people live in a different “Umwelt” (ecological world) from that which non-autistic people experience in everyday life. Diversity in cognitive granularity explains not only the major impairments in autism, including the difficulties in social communication, but also the spectrum of cognitive styles in the entire population. From this unified perspective, we also discuss possible therapeutic interventions, particularly those using robots, whose behavior can be tuned to the cognitive granularity of each individual.

Granularity in prediction and control

Self, objects, and others

Young infants start partitioning the experiential world into “self,” physical realm of “objects,” and social realm of “others” in terms of predictability and controllability (Takahashi & Miyazaki, 2011). A portion of the world is called “predictable” when we observe no significant error from what we have expected in its behavior. A portion of the world is called “controllable” when we can selectively derive a desirable response from the predicted stimulus-response patterns; one's controllable world is a subset of the predictable world.

Based on this idea, Kozima (2011) proposed an elaborated model of the emergence of “self,” “objects,” and “others.” As illustrated in Figure 1, an infant first divides the world into a directly controllable world, that is, one's own self, and the rest, that is the external world. The self can be extended outward by tools that have high controllability. Next, from the external world, the physical and social realms emerge, both of which have various degrees of predictability and controllability. The physical realm is relatively predictable and/or controllable in terms of the cause-effect relationships among physically observable elements, while the social realm is relatively predictable and/or controllable in terms of the means-end relationships in human and animal actions.

Figure 1.

Partitioning the world into “self,” “objects,” and “others” in terms of predictability and controllability.

Granularity in physical and social realms

We believe granularity in prediction and control play an important role in the differentiation between the physical and the social realms. The physical realm emerges as a realm with microscopic predictability and/or controllability, which seems to be governed, or explained, by spatiotemporally fine-grained, causal, and deterministic mechanisms. In contrast, the social realm emerges as a realm with macroscopic predictability and/or controllability, which can be learned from spatiotemporally coarse-grained, teleological invariants in physically observable actions of others.

Consider a set of actions ai (i = 1, … , n), as illustrated in Figure 2, each of which begins from the same state x and ends at the same state y, but each moves along a different trajectory, {x, si,1, … , si,ni, y}, through a number of states. From the microscopic perspective, one observes individual samples of the microscopic transition, such as si,k → si,k+1 (k = 1, … , ni − 1), from which one would not find any significant invariants. However, from the macroscopic perspective, one observes repetitions of the invariant leaps, described as x → y, which have qualitatively different information from that which one would see in the microscopic trajectories.

Figure 2.

A set of actions ai seen as different trajectories at the microscopic level and as an invariant intention (the gray arrow) at the macroscopic level.

The above example demonstrates that, with different granularity, one would experience the world at a different level of abstraction. Being at the microscopic level means that one takes the physical or design stance (Dennett, 1989), whereby we understand and predict things and events in terms of causal and deterministic mechanisms with various complexity. In contrast, being at the macroscopic level means that one takes an intentional stance (Dennett, 1989) or a teleological stance (Gergely & Csibra, 2003), whereby we interpret the macroscopic invariant, x → y, as an intentional use of the means x to achieve the goal y under various conditions.

Granularity in the self

In addition to the differentiation of the external world, granularity also draws an important, but rather vague partition in the self, as shown in Figure 1.

At a moderate level of spatiotemporal granularity, one can intentionally control one's own actions to achieve a goal in a given context; also one can plan and perform a sequence of actions to achieve a higher goal. This level equates to that which we described above as the macroscopic level, at which understanding others' intentional actions and performing one's own intentional actions are closely connected to each other.

At a finer level of granularity, described formerly as the microscopic level, one's own body semiautomatically responds to the environmental stimuli, performing a large repertoire of standard microactions, each of which corresponds to different affordances in the environment (Norman, 1988). In contrast, at a coarser level of granularity, one's own future behavior becomes less predictable and/or less controllable, because the external world and the self are too complex for individuals to follow all the causal and deterministic transitions. The unpredictability and uncontrollability of the future self would relate to our free will, or how and why we feel that we have free will.

Granularity in brain and cognition

Individuals with autism generally show a preference for predictable and/or controllable objects, events, and situations (Takahashi & Miyazaki, 2011), particularly those with spatiotemporally microscopic causality. This preference would prevent them from learning invariant patterns in human intentional, situation-dependent behaviors, which non-autistic individuals naturally learn from phenomena at the level of macroscopic granularity. What causes this difference in the granularity of our perception and cognition?

Neuroanatomical abnormalities in autism

Recent neuroanatomical findings in autism research suggest where the autistic individuals' bias for microscopic granularity would come from. One of the suggested findings, reported by Casanova, van Kooten, Switala, van Engeland, Heinsen, Steinbusch, Hof, Trippe, Stone, and Schmitz (2006), is quantitative abnormalities in minicolumns in the brains of individuals with autism.

A minicolumn is the smallest identifiable organization that spans vertically through the cortical layers, consisting of approximately 100 neurons with internal and external connections; being bundled up densely and connected with each other horizontally, minicolumns uniformly cover the entire cortex, suggesting that they are the smallest functional units of the brain (Buxhoeveden & Casanova, 2002).

Casanova et al. (2006) found that the size of the minicolumns in the brains of individuals with autism tend to be smaller in size. Considering that autistic children tend to have larger brains due to an abnormal growth spurt (Herbert, 2005), the reduced size of the minicolumns suggests a larger number of minicolumns in total. Casanova et al. (2006) also reported that horizontal connections between the minicolumns tend to be shorter because of the smaller (thus weaker) output cells in the smaller minicolumns.

These minicolumnar abnormalities in autism have a broad range of implications. We discuss here (a) cognitive granularity and (b) information processing styles in autism.

Cognitive granularity in autism

The abnormality in the size and number of minicolumns means that individuals with autism would use more minicolumns to represent things and events in the world, suggesting that the cognitive categories they acquire would also be different in size and/or number from those of normal individuals.

To explain this, let us consider a sparse coding model of cortical information representation (Rinkus, 2010), as illustrated in Figure 3. In this model, a macrocolumn, known as a bundle of approximately 70 adjacent minicolumns, represents symbolic states of the world. Each minicolumn can detect some fragmentary features in the sensory input, and may activate only one of its approximately 20 output cells, which form a winner-takes-all circuitry based on mutual inhibition between the cells. Being bundled up within a macrocolumn, some 70 winners from each of the minicolumns encode (and decode as well) a particular combination of the features. Such a combination thus encoded by a macrocolumn would represent a partial state of the brain (Minsky, 1980, 1986), or a partial snapshot of the cortical activity, which would correspond to a symbolic representation of a concept or a cognitive category.

Figure 3.

Cortical sparse distributed coding model (Rinkus, 2010), where each activated pattern of 70 minicolums in a macrocolumn encodes a symbolic category. (Note that only seven of 20 output cells in a minicolumn are shown here for the sake of simplicity.)

Although the total numbers of macrocolumns in autistic and normal brains have not been compared yet clearly, the abnormality in the total number of minicolumns in the autistic brain suggests that they are likely to represent the world in finer and/or more specific manners. If the average number of minicolumns in a macrocolumn is constant over autistic and normal brains, an autistic brain has a larger number of macrocolumns to partition the world into a larger number of cognitive categories, each of which should have a smaller semantic extent. In contrast, if the total number of macrocolumns is constant (where a macrocolumn contains a larger number of minicolumns), an autistic brain would acquire cognitive categories, each of which is more specifically composed by a larger set of features as necessary conditions.

Information processing styles in autism

The abnormality in the size of the output cells in minicolumns suggests that individuals with autism have an atypical style of information processing, because smaller output cells have a metabolic bias towards shorter connections against longer ones (Casanova et al., 2006).

This would explain why individuals with autism generally have difficulties in integrating different aspects of information. For example, individuals with autism generally have weak central coherence (Frith, 2003), which is a limited ability to understand the “gestalt” of objects and events in a macroscopic context. This could be explained by the preference for local information processing due to the shortage of longer connections between the minicolumns in different brain regions. Another example is “stimulus overselectivity,” which is a tendency in individuals with autism to respond to only a limited number of cues among all the available information, particularly those across different modalities (Lovaas, Koegel, & Schreibman, 1979). Such correlations among stimuli have to be represented and processed by using longer, corticocortical, and sometimes interhemispheric connections across different brain regions.

Granularity and cognitive styles

As we have seen above, each individual with different cognitive granularity represents the world in a different resolution, whereby each of the units for information processing change not only in size, but also in their semantic relations with other units. This diversity in cognitive granularity may positively count for the personalities and, in some cases, the savant talents of individuals with autism. But this may also count for the various handicaps that individuals with autism may encounter in their everyday life with the neurotypical population.

Granularity matching

From a microscopic viewpoint, as can be seen in Figure 2, various human actions that share the same underlying intention would look like arbitrary sequences of microactions without shared invariants, namely, intentions. This implies that what we referred to as developmental disorders in “theory of mind” (Baron-Cohen et al., 1985) or “mentalizing” (Frith, 2003), and “mindblindness” (Baron-Cohen, 1995) would be explained uniformly by the difference in granularity of the typical and atypical populations.

Failures in granularity matching have a significant influence on language acquisition as well. In all natural languages, the world is categorized in a similar way, with some variations due to geographical and historical conditions, into “natural categories” at the “basic level” (Rosch, 1975; Rosch, Mervis, Gray, Johnson, & Boyes-Braem, 1976). The basic level is determined by the balance between the environmental complexity of the particular world one lives in and the capacity of one's cognitive system to make predictions and exercise control of the world. In other words, the natural categories of a certain language are evolutionarily and historically formed by the majority with a typical cognitive granularity. This implies that those with atypical cognitive granularity, such as individuals with autism, would have difficulty and inefficiency in learning and using the language of the standard granularity. This would also be applicable to virtually all artifacts, such as architecture and social conventions.

Even in the neurotypical majority, such a mismatch in granularity may occur to various degrees. For example, when we deal with complex systems, such as computers, or natural phenomena, such as earthquakes, we often try to understand the systems' behavior in terms of human mental terms, such as want, hate, happy, angry, and tired. This general tendency to personification, known as “media equation” (Reeves & Nass, 1996), could also be explained by cognitive granularity. When a system is too complex to understand in terms of causal mechanisms, we step back to get a macroscopic view, as illustrated in Figure 2, whereby we see mental and social analogues for efficient description and prediction.

Meanwhile, some people have a stronger tendency to empathetically interpret the behavior of a system in terms of human mental states, while some other people show the opposite tendency to systematically interpret the same system in terms of causal and deterministic mechanisms. Baron-Cohen (2002) hypothesized that individual brains can be plotted on the “empathizing” and “systemizing” dimensions, and that autism would be an extreme case of the “systemizing” brains. This diversity of cognitive predisposition could be explained directly by individual variance in cognitive granularity.

Implications for autism therapy

Individuals with autism, having atypical cognitive granularity, struggle continuously against the mismatched granularity of the surrounding social world. However, they often show good performance on manipulating toys and simple machines, the causal and deterministic behavior of which are easily understood at the level of microscopic granularity. This could be why autistic children often obsessively engage in interactions with such physical objects, while showing indifference to the people nearby. A good granularity matching would give them predictability, controllability, and cognitive efficiency, and thus a sense of security.

We believe that the preferable therapeutic intervention for children with autism is to build a bridge between the predictable physical realm and the as yet unpredictable social realm. One possible approach for this bridging is to use a simple robot, whose complexity in behavior (determined by the software) and appearance (determined by the hardware) can be tuned to a desirable or acceptable level for each autistic child. In other words, the cognitive granularity of the robot can be tuned to that of each individual. Also, by appropriately changing the level of the robot's cognitive granularity, we can provide each child with a “zone of proximal development” to foster his or her adaptation to the social realm.

In fact, our pilot study (Kozima, Michalowski, & Nakagawa, 2009) found that children with autism preferred interacting with a simple but communicative robot, called Keepon, whose behavior was carefully designed to match the cognitive granularity of the individual children. The children spontaneously approached the robot and engaged in dyadic interaction with it in a playful mood, which then extended to triadic interactions where they exchanged with adult caregivers the pleasure and surprise they previously found in the dyadic interaction. Most of the autistic children naturally understood the social meanings of the robot's actions; this is probably because every action of the robot is designed to be almost identical at both microscopic and macroscopic levels.

The above study suggests that children with autism possess the motivation to share mental states with others, which is contrary to the commonly held position, for instance, by Tomasello (1999), that this motivation is impaired in autism. We believe that children with autism possess an intact motivation to relate to others, but because of the granularity mismatch they fail to connect their motivation with what they microscopically perceive from the surroundings. In conclusion, we recognize autism to be an extreme case in the spectrum of cognitive styles characterized by cognitive granularity, which may stem from abnormalities in the cortical density.