Bottom-Up and Top-Down Processing, and Pattern Recognition

Definition and concept of perception; perceptual organization — influence of past experiences.

Official Syllabus: Paper I · Unit 5 Sensation, Attention and Perception

“Perception involves going beyond the immediately given evidence to construct a representation of the external world.”

— Jerome Bruner (1957), in On Perceptual Readiness

Recognising a face, a word or a cup requires information to travel in two directions. Upward from the retina come edges, colours and contours. Downward from memory come expectations about what is likely to be there. Perception is the meeting of the two, and the long argument in this field has been over how much of the work each direction does.

Bottom-up processing

Bottom-up or data-driven processing begins with the stimulus and builds upward: receptors register elementary features, features are combined into shapes, shapes into objects, objects into scenes. Nothing in the process depends on the perceiver’s knowledge or expectations — the same input produces the same analysis in anyone with a working visual system.

• Its direction is periphery to centre, part to whole.
• Its content is supplied entirely by the stimulus.
• It is fast, automatic and obligatory: you cannot choose not to analyse the edges in front of you.
• It is necessary: without it there is nothing to perceive, which is why perception of an entirely novel object is still possible.
• Its evidence is physiological — the feature detectors of Hubel and Wiesel, which respond to oriented edges regardless of what the observer expects.
• Its limitation is that it is slow and error-prone with degraded input, and by itself cannot resolve ambiguity, because the retinal image is consistent with many interpretations.

Top-down processing

Top-down or conceptual-driven processing begins with knowledge, expectation and context, and uses them to interpret, complete and disambiguate the incoming data.

• Its direction is centre to periphery, whole to part.
• Its content comes from memory, context, motive and set.
• It is fast where it applies and can run ahead of the data, generating a hypothesis the data then confirm or refute.
• Its evidence is everywhere in everyday perception: reading bad handwriting, understanding speech in a noisy room, recognising a friend at a distance from gait alone.
• Its limitation is that it imports error. Where expectation is wrong, perception is wrong — which is what perceptual set, illusion and the proof-reader’s blindness to typographical errors demonstrate.

A caution worth stating, because the library treatments often get it backwards: top-down does not mean breaking a whole into parts. It means using higher-level knowledge to shape lower-level interpretation. The common illustration of “learning what a dog is by noting legs and tail” is bottom-up, not top-down.

The two contrasted

Gibson against Gregory

The theoretical dispute is worth stating because the two positions are genuinely incompatible at their strong forms.

Gregory’s constructivist theory holds that the retinal image is impoverished and ambiguous, so perception must be hypothesis-testing: the brain forms the best guess consistent with the data and past experience. Illusions are then the strongest evidence — they are hypotheses that happen to be wrong, and they persist after explanation because the inference is not under voluntary control.

Gibson’s theory of direct perception holds the opposite. The optic array available to a moving observer is extremely rich, containing invariants — texture gradients, optic flow, horizon ratios — that specify the layout of the world unambiguously. Perception therefore requires no inference: information is picked up directly. Gibson added the influential concept of affordances, the action possibilities a surface or object offers the perceiver, arguing that we perceive what things are for rather than their abstract properties.

The evaluation. Gibson is strongest where he looked — a moving observer in a richly textured natural environment, which is the condition his theory was built for and the condition under which perception is almost error-free. Gregory is strongest where he looked — brief, impoverished, static, unusual displays, which is where inference is forced to work hardest. The dispute is partly an artefact of different experimental conditions, and the honest conclusion is that the relative contribution of the two directions varies with the quality of the input rather than being fixed.

How the two combine

The modern position is that the two operate simultaneously and interactively, not in sequence, and three lines of evidence establish it.

• The word superiority effect. A letter is identified more accurately when presented within a word than when presented alone, or within a nonsense string. This is decisive: if processing were purely bottom-up, surrounding letters could not help, since each letter’s own data are identical in both conditions. Word-level knowledge is reaching down to assist letter-level identification.
• The phoneme restoration effect. A speech sound deleted from a sentence and replaced with a cough is heard as present, and listeners cannot say which sound was missing. Which phoneme is “heard” depends on the rest of the sentence, including words occurring after it — so top-down completion is operating on material already past.
• Context effects on ambiguous forms. The same shape is read as the letter B or the number 13 according to whether the surrounding characters are letters or digits, without the perceiver noticing any ambiguity.

McClelland and Rumelhart’s interactive activation model (1981) formalises this. Processing units at feature, letter and word levels are connected by excitatory links between consistent units and inhibitory links between inconsistent ones, with activation flowing both up and down at once. Evidence for a word excites its letters, which excite their features, while the features excite the letters and the letters the word. The system settles into the interpretation best supported from both directions, which reproduces the word superiority effect naturally rather than stipulating it.

Combining the two in everyday life

The practical question is how to use both deliberately, and the general rule is that top-down processing should be trusted in proportion to the reliability of your expectations, and bottom-up processing relied on exactly where it is not.

• Reading and studying. Survey headings and summaries first to build a framework — top-down — then read the detail into it. Reading an unfamiliar technical text with no framework is laborious precisely because it must be done bottom-up.
• Medical and technical diagnosis. The expert’s pattern knowledge is what makes diagnosis fast, and it is also what produces premature closure — settling on a familiar pattern and ceasing to look. The discipline of deliberately re-examining the raw data when a case does not quite fit is a deliberate switch back to bottom-up.
• Proof-reading. Here top-down processing is the enemy, because knowledge of what the sentence should say overrides what it does say. The counter-measures are all devices to defeat it: reading backwards, reading aloud, changing the font, leaving the text overnight.
• Eyewitness testimony. Expectation fills gaps and produces confident error, which is why interview protocols are designed to elicit free recall before any suggestion is introduced.
• Learning a skill. Early learning is effortfully bottom-up — a learner driver attending to each control separately; expertise is the accumulation of top-down structures that let the whole situation be read at once.
• Design. Good design supports both: clear bottom-up signals such as contrast and spacing, together with conventions the user already knows, so expectation and data agree. Confusion arises when a familiar form carries an unfamiliar function.

The three stages: selection, organisation, interpretation

Processing sensory input into a percept is conventionally described in three stages, and the two directions of processing operate throughout all of them rather than belonging to any one.

• Selection. Of the vast input arriving at the receptors, only a fraction is processed. Bottom-up selection is stimulus-driven — intensity, contrast, movement and novelty capture attention involuntarily. Top-down selection is goal-driven — the perceiver’s set, interests and current task determine what is sought. The two compete continuously, which is why a salient irrelevance can derail a deliberate search.
• Organisation. What has been selected is structured into coherent units: figure separated from ground, elements grouped by proximity, similarity, continuity, closure and common fate, and the whole stabilised by the constancies. Organisation is largely bottom-up and automatic — the grouping principles operate without instruction and in infancy — but past experience reorganises it, which is why a meaningless pattern becomes organised the moment its identity is known and never looks the same again.
• Interpretation. The organised unit is matched to a category in memory and given meaning. This is where top-down processing dominates, since the categories, the expectations and the context that resolve ambiguity all come from the perceiver. The output is a recognised object rather than a shape.

The stages are not strictly sequential. Interpretation feeds back to selection, so recognising something as important draws further attention to it; and organisation is influenced by tentative interpretation, which is how an ambiguous figure can reorganise when a different reading is suggested. The three-stage account is a useful decomposition of one continuous, interactive process, not a description of three separate operations in series.

Pattern recognition

Pattern recognition is the process by which a stimulus is matched to a category in memory. Three families of account exist, and the examinable comparison is between the first two.

Template matching

The simplest account. Memory holds a template — a stored whole-pattern copy — for each recognisable pattern, and an input is compared against the stored set until a match is found.

Its appeal is simplicity, and it is the correct account of machine recognition in constrained settings: the magnetic characters on a cheque and the standardised numerals on a postal code are deliberately designed so that template matching will work, which tells you what the theory needs in order to succeed.

Its failures as a theory of human recognition are severe.

• Variability. We recognise the letter A in countless fonts, sizes, orientations, handwritings and degrees of degradation. A separate template for each is required, which is combinatorially impossible.
• Storage. The number of templates needed is unmanageable.
• Novelty. A pattern never seen before has no template and should be unrecognisable, yet we recognise novel handwriting immediately.
• Normalisation does not save it. Proposing a pre-processing stage that rotates and scales the input to standard form only pushes the problem back: the system must already know the object’s orientation to correct it.
• No account of similarity. Matching is all-or-none, so template theory cannot explain why we confuse some patterns and not others — why an O is confused with a Q and not a W.

Feature detection

The dominant account. A pattern is analysed into a set of component features, and recognition is the identification of a combination of features rather than a whole.

Selfridge’s Pandemonium model (1959) gave it its classic architecture, in four layers of “demons”: image demons register the raw input, feature demons each shout according to how strongly their own feature is present, cognitive demons listen for the feature combinations corresponding to their pattern and shout proportionally, and a decision demon selects the loudest. Crude as the metaphor is, it is a genuinely parallel, evidence-weighted architecture.

Its evidence is strong and converges from three directions.

• Physiological — Hubel and Wiesel’s cortical cells respond to precisely the oriented edges the theory requires.
• Behavioural — visual search is slower when distractors share features with the target, and confusion errors follow feature overlap, so letters sharing features are confused more often.
• Developmental and comparative — feature-based analysis appears early and is widespread across species.

Its limitations are real.

• It does not specify the feature set. Which features are primitive is largely stipulated after the fact.
• It underweights relations. A set of features is not a pattern: the same features arranged differently make different letters, and feature lists do not capture arrangement.
• It is bottom-up by construction, so it cannot by itself explain the word superiority effect or context effects.
• It does not scale to objects, which is the grandmother-cell problem.

Template matching and feature detection compared

The verdict is that feature detection is clearly the better account of human pattern recognition, and that template matching survives as a description of the constrained machine case and as a useful contrast that shows what any adequate theory must explain. Feature theory’s own weakness — the neglect of arrangement — is what the next family addresses.

Prototype and structural accounts

Prototype theory holds that memory stores not every exemplar nor a rigid template but an abstracted central tendency — an average of the instances encountered. Recognition is judged by similarity to the prototype, which explains why typical instances are recognised faster than atypical ones, and why people confidently “recognise” a prototype they have never actually seen. It handles variability as template theory cannot while preserving the whole-pattern matching that feature theory loses.

Biederman’s recognition-by-components theory supplies the structural element. Objects are decomposed into a modest set of basic volumetric primitives, geons — blocks, cylinders, wedges, cones — and an object is represented by its geons and the spatial relations among them. A cup is a cylinder with a curved handle attached at the side; moving the handle to the base makes a different object from identical parts. This captures arrangement, explains why line drawings are recognised as readily as photographs, and predicts the finding that degrading the junctions between parts impairs recognition far more than removing equivalent amounts of contour elsewhere.

The modern position combines them: features and their relations, matched against flexible category representations, with top-down context contributing throughout.

Criticisms and limitations

• The bottom-up/top-down distinction is clearer than the processing it describes. The two are so thoroughly interactive that assigning any particular effect wholly to one direction is usually arbitrary, and the terms function better as a descriptive vocabulary than as a theory.
• The Gibson–Gregory dispute was never settled empirically, because each theory was tested under the conditions favouring it. Neither side generated a decisive crucial experiment.
• Gibson’s invariants and affordances are underspecified. The theory asserts that information sufficient for direct perception exists without always demonstrating what it is, which makes it difficult to falsify.
• Constructivism struggles with speed. Perception is fast and generally accurate, which sits awkwardly with an account requiring hypothesis generation and testing on every occasion.
• Pattern recognition research has been dominated by letters. Alphanumeric characters are an unusual class — two-dimensional, high-contrast, culturally standardised, learned by explicit instruction — and the generalisation from them to natural object recognition is weaker than usually acknowledged.
• Feature theories do not say what a feature is. Without an independent criterion, the feature set can be adjusted to fit any result, which costs the theory much of its predictive force.
• Most of this work is visual and Western-laboratory. Pattern recognition in hearing, touch and smell is comparatively unstudied, and the role of literacy in shaping the letter-based findings is rarely considered.