Structural information theory

Structural information theory (SIT) is a theory about human perception and in particular about perceptual organization. which is the neuro-cognitive process that enables us to perceive scenes as structured wholes consisting of objects arranged in space.. SIT was initiated, in the 1960s, by Emanuel Leeuwenberg^[1][2][3] and has been developed further by Hans Buffart, Peter A. van der Helm, and Rob van Lier. It has been applied to a wide range of research topics,^[4] mostly in visual form perception but also in, for instance, visual ergonomics, data visualization, and music perception.

SIT began as a quantitative model of visual pattern classification. Nowadays, it includes quantitative models of symmetry perception and amodal completion, and is theoretically sustained by a perceptually adequate formalization of visual regularity, a quantitative account of viewpoint dependencies, and a powerful form of neurocomputation.^[5] SIT has been argued to be the best defined and most successful extension of Gestalt ideas.^[6] It is the only Gestalt approach providing a formal calculus that generates plausible perceptual interpretations.

The simplicity principle

Although visual stimuli are fundamentally multi-interpretable, the human visual system usually has a clear preference for only one interpretation. To explain this preference, SIT introduced a formal coding model starting from the assumption that the perceptually preferred interpretation of a stimulus is the one with the simplest code. A simplest code is a code with minimum information load, that is, a code that enables a reconstruction of the stimulus using a minimum number of descriptive parameters. Such a code is obtained by capturing a maximum amount of visual regularity and yields a hierarchical organization of the stimulus in terms of wholes and parts.

The assumption that the visual system prefers simplest interpretations is called the simplicity principle.^[7] Historically, the simplicity principle is an information-theoretical translation of the Gestalt law of Prägnanz,^[8] which was based on the natural tendency of physical systems to settle into stable minimum-energy states. Furthermore, just as the later-proposed minimum description length principle in algorithmic information theory (AIT), a.k.a. the theory of Kolmogorov complexity, it can be seen as a formalization of Occam's Razor in which the best hypothesis for a given set of data is the one that leads to the largest compression of the data.

Structural versus algorithmic information theory

Since the 1960s, SIT (in psychology) and AIT (in computer science) evolved independently as viable alternatives for Shannon's classical information theory which had been developed in communication theory.^[9] In Shannon's approach, things are assigned codes with lengths based on their probability in terms of frequencies of occurrence (as, e.g., in the Morse code). In many domains, including perception, such probabilities are hardly quantifiable if at all, however. Both SIT and AIT circumvent this problem by turning to descriptive complexities of individual things.

Although SIT and AIT share many starting points and objectives, there are also several relevant differences:

• First, SIT makes the perceptually relevant distinction between structural and metrical information, whereas AIT does not;
• Second, SIT encodes for a restricted set of perceptually relevant kinds of regularities, whereas AIT encodes for any imaginable regularity;
• Third, in SIT, the relevant outcome of an encoding is a hierarchical organization, whereas in AIT, it is only a complexity value.

Simplicity versus likelihood

In visual perception research, the simplicity principle contrasts with the Helmholtzian likelihood principle,^[10] which assumes that the preferred interpretation of a stimulus is the one most likely to be true in this world. As shown within a Bayesian framework and using AIT findings, the simplicity principle would imply that perceptual interpretations are fairly veridical (i.e., truthful) in many worlds rather than, as assumed by the likelihood principle, highly veridical in only one world.^[11] In other words, whereas the likelihood principle suggests that the visual system is a special-purpose system (i.e., adapted to one specific world), the simplicity principle suggests that it is a general-purpose system (i.e., adaptive to many different worlds).

Crucial to the latter finding is the distinction between, and integration of, viewpoint-independent and viewpoint-dependent factors in vision, as proposed in SIT's empirically successful model of amodal completion.^[12] In the Bayesian framework, these factors correspond to prior probabilities and conditional probabilities, respectively. In SIT's model, however, both factors are quantified in terms of complexities, that is, complexities of objects and spatial relationships, respectively. This approach is consistent with neuroscientific ideas about the distinction and interaction between the ventral ("what") and dorsal ("where") streams in the brain.^[13]

SIT versus connectionism and dynamic systems theory

On the one hand, a representational theory like SIT seems opposite to dynamic systems theory (DST). On the other hand, connectionism can be seen as something in between, that is, it flirts with DST when it comes to the usage of differential equations and it flirts with theories like SIT when it comes to the representation of information. In fact, the analyses provided by SIT, connectionism, and DST, correspond to what Marr called the computational, the algorithmic, and the implementational levels of description, respectively. According to Marr, such analyses are complementary rather than opposite.

What SIT, connectionism, and DST have in common is that they describe nonlinear system behavior, that is, a minor change in the input may yield a major change in the output. Their complementarity expresses itself in that they focus on different aspects:

• First, DST focuses primarily on how the state of a physical system as a whole (in this case, the brain) develops over time, whereas both SIT and connectionism focus primarily on what a system does in terms of information processing (which, in this case, can be said to constitute cognition).
• Second, according to both SIT and connectionism, this information processing relies on interactions between pieces of information in distributed representations, that is, in networks of connected pieces of information. In this respect, however, connectionism focuses on concrete interaction mechanisms (i.c., activation spreading) in a prefixed network that is assumed to be suited for any input, whereas SIT focuses on the nature of the outcome of the interactions which are assumed to take place in transient, input-dependent, networks.

Modelling principles

In SIT, candidate interpretations of a stimulus are represented by symbol strings, in which identical symbols refer to identical perceptual primitives (e.g., blobs or edges). Every substring of such a string represents a spatially contiguous part of an interpretation, so that the entire string can be read as a reconstruction recipe for the interpretation and, thereby, for the stimulus. These strings then are encoded (i.e., they are searched for visual regularities) to find the interpretation with the simplest code.

In SIT's formal coding model, this encoding is modelled by way of symbol manipulation. In psychology, this has led to critical statements of the sort of "SIT assumes that the brain performs symbol manipulation". Such statements, however, fall in the same category as statements such as "physics assumes that nature applies formulas such as Einstein's E=mc² or Newton's F=ma" and "DST models assume that dynamic systems apply differential equations". That is, these statements ignore that the very concept of formalization means that potentially relevant things are represented by symbols - not as a goal in itself but as a means to capture potentially relevant relationships between these things.

Visual regularity

To obtain simplest codes, SIT applies coding rules that capture the kinds of regularity called iteration, symmetry, and alternation. These have been shown to be the only regularities that satisfy the formal accessibility criteria of

• (a) being so-called holographic regularities that
• (b) allow for so-called hierarchically transparent codes.^[14]

A crucial difference with respect to the traditional, so-called transformational, formalization of visual regularity is that, holographically, mirror symmetry is composed of many relationships between symmetry pairs rather than one relationship between symmetry halves. Whereas the transformational characterization may be better suited for object recognition, the holographic characterization seems more consistent with the buildup of mental representations in object perception.

The perceptual relevance of the criteria of holography and transparency has been verified in the so-called holographic approach to visual regularity.^[15] This approach provides an empirically successful model of the detectability of single and combined visual regularities, whether or not perturbed by noise.

Cognitive architecture

The transparent holographic regularities have been shown to lend themselves for transparallel processing. This means that, in the process of selecting a simplest code from among all possible codes, O(2^N) codes can be taken into account as if only one code of length N were concerned.^[16] This supports the computational tractability of simplest codes and, thereby, the feasibility of the simplicity principle in perceptual organization.

To enable transparallel processing, SIT's formal model gathers candidate codes (of only the input at hand) in special distributed representations called hyperstrings (a form of feature binding). These hyperstrings can be seen as formal counterparts of transient neural assemblies which signal their presence by firing synchronization of the neurons involved. This gives rise to a concrete picture of flexible cognitive architecture implemented in the relatively rigid neural architecture of the brain. That is, these temporarily synchronized assemblies can be called "gnosons" (i.e., fundamental particles of cognition) whose synchronization might well be a manifestation of transparallel feature processing.^[17]

See also

• Neural processing for individual categories of objects
• Principles of grouping
• Theory of indispensable attributes
• Simplicity theory

References