Visual search and feature binding Lecture 7

Illusory Conjunctions (from Treisman 1988, 1992)

if attention is not properly deployed, participants experience illusory conjunctions. For example, when a display containing a blue triangle and a red circle is displayed only very briefly, participants cannot accurately report whether they saw a blue triangle and red circle, or a red triangle and blue circle. That is, they report all the separate features correctly, but do not bind them together correctly. Illusory conjunctions have, consistent with FIT, been demonstrated under conditions when there is insufficient attention, e.g. when focused attention is absent and participants don't have prior knowledge about the objects (e.g., a banana is yellow), when spatial attention is diverted away to another part of the screen, or when the features to be bound are displayed in peripheral vision (while the subject is focusing on the centre).(relevant stored knowledge absent, spatial attention is diverted, display is presented in peripheral vision)

Negative priming Tasks (Tripper, 1985)

it has been criticised that FIT is (like other models of focused attention discussed in the last lecture) and early selection model, i.e., it assumed that selective attention and attentional selection operate on the basis of early perceptual features (e.g. colour or orientation). Evidence against this early selection assumption comes from studies using negative priming tasks (Tipper, 1985). The classic negative priming task uses hybrid objects in two different colours, where attention is only paid to objects in one colour (e.g., name the red object and ignore the blue one). What is typically found is that ignoring an object on one trial (N) makes us slower at naming this same object on the next trial (N+1), so ignoring something seems to have aftereffects. Studies using semantically related objects (e.g., cats and dogs) have demonstrated that such negative priming occurs even on the level of semantic categories, i.e. ignoring a cat on trial N causes slower reaction times for naming a dog on trial N+1. This finding suggests that we process the meaning (semantics) of unattended objects which we should have excluded from higher-level processing according to FIT, because selection already happens on the level of perceptual features (colours).

Feature integration: the role of top-down knowledge

Illusory conjunctions are less likely when people use their prior knowledge about objects to make a decision. For example, Treisman & Souther (1986) used three shapes under brief viewing conditions where illusory conjunctions were frequently reported. However, they found that when they told participants before the experiment that the three objects were a carrot, a lake and a tire, no illusory conjunctions were made. Treisman concluded that prior knowledge can play an important role in facilitating object perception even at this rather early feature binding stage.

FIT evaluation

STRENGTHS: An important attempt to explain what is happening within the attentional spotlight Influenced thinking on a variety of topics from early sensory encoding to later attentional control WEAKNESSES: Conjunction searches are faster than predicted The similarity of distractors is influential Neglect/Extinction patients have problems with both conjunctive and single-feature targets

Visual Search

needed every day to find the things that are relevant for a given task at hand in a clutter visual world.

FIT Stages of visual search

-A rapid initial parallel process is utilized to identify features, this process in attention independent. -Then a slower, serial process is utilised to form objects from combining features. Features can be combined by focused attending to the location of the object (with focused attention being the "glue"). Feature combination can be influenced by prior knowledge (e.g., bananas are yellow)

Dual Pathway Model

As an alternative to FIT, Wolfe (1998) proposed the Dual Pathway Model (related to his earlier model called Guided Search Theory). According to Wolfe, instead of assuming that the initial feature processing is parallel and subsequent processing is serial, he assumed that search can be more or less efficient depending on whether serial or parallel processes are deployed.

Evidence for FIT

Both studies showed that feature search performance (as measured by reaction times) was not affected by set-size, i.e., pop-outs led to very fast reaction times independent of the search set size. In contrast, RTs in the conjunctive search condition increased with change in the number of items in the search array (the set-size). These findings support the assumption that feature search can be based on parallel search mechanisms (and single feature targets cause pop-out effects in a bottom-up manner), while conjunction search requires serial attention to each spatial location.

Parietal Cortex and Feature Binding

Corbetta and colleagues (1995) conducted a number of fMRI studies finding that during conjunction search but not feature search, posterior temporal and parietal regions show increased activation, over baseline control conditions.These fMRI studies were first criticised because they left open the question whether this increased activity was simply driven by the fact that conjunction search is more difficult than feature search, and that every more difficult task might produce more parietal activation. However, Shafritz et al. 2002, conducted an fMRI study in which subjects had to attend either to one feature only (colour or shape), or both features of each object presented. The objects were followed by a mask, and they then had to decide whether the probe matched one of the previously presented objects. if parietal lobe responds to difficulty it should increase its activity in the "attend both" condition irrespective of conjunction or feature search. The authors replicated Corbetta et al.'s previous findings that parietal areas increase activity during conjunction relative to feature search. However, they also found that his increase during conjunction search only happens when binding across space is required, in the simultaneous condition, not when binding across time. This finding is consistent with the idea that parietal lobe plays a specific role for binding features across space, and supports the findings from neuropsychological studies.

Ponser's Attentional Systems

Endogenous System (top-down, goal driven): Controlled by the individual's intentions and expectations. Involved when informative central cues are presented. Exogenous System (bottom-up, stimulus driven): Automatically shifts attention. Involved when uninformative peripheral cues are presented. Stimuli that are salient or that differ from other stimuli are most likely to be attended.

overall about FIT

FIT clearly had an important influence on thinking in the field, and it was the first systematic attempt to characterize visual search. FIT has been criticised for overestimating search times for conjunction search (suggesting that parallel search might be involved to some degree during conjunction search), that it cannot explain why the similarity amongst distractors affects reaction times, and that it cannot explain why patients with impaired spatial attention have problems with both types of searches (feature and conjunction).

Evidence against FIT

FIT comes from a study by Duncan & Humphreys (1989). They criticise that it is hard to define a prior what exactly constitutes a "feature", before having found out which features pop-out during search. However, some evidence does not support this assumption.the authors demonstrated that searching for an L is easy if the Ts are rotated 180 and 270 degrees, but easier if the Ts are rotated 0 or 90 degrees. Therefore, even if the basic features (horizontal and vertical lines) are kept constant, visual search can be easier or harder based on the similarity between targets and distractors. Duncan and Humphreys also argued that the similarity between distractors themselves is important in determining how easy visual search is. This implies that some feature binding occurs rapidly and without attention, contradicting a basic assumption of FIT. (group objects together)

FIT limitations

FIT is also of limited usefulness in explaining the results from more ecologically valid, realistic search displays. In real world visual search, observers often know where to expect certain things. In the study by Ehringer et al. (2009), eye fixations were measured while participants search for the presence of a human figure in the pictures. The dots on the pictures represent the first 3 fixations of each participant. The data show that participants are very selective in focusing on locations where they realistically expect a human being. For example, they ignore the sky and trees, and instead focus on locations on the ground.

Parietal cortec and feature binding (Walsh et al.,1995) (Esterman et al., 2007)

Further support comes from a TMS study by Walsh et al. (1995), demonstrating that stimulation of the parietal lobe disrupts healthy participants' performance (reaction times) when they are doing conjunction search, but not feature search. Finally, Esterman and colleagues (2007) found that activating the intraparietal sulcus with TMS can reduce the occurrence of illusory conjunctions in healthy participants, again suggesting a role of this region for feature binding.

Summary: Parietal cortex

Neuropsychological findings consistently demonstrate that the parietal lobe plays an important role for spatial attention. Patients with parietal lesions experience problems with feature binding across space but not time. TMS on the parietal lobe affects feature binding across space, and can vary the level of illusory conjunctions. The deficits are more strongly linked to the top-down, rather than the bottom-up, parietal lobe system, but this dissociation is still under investigation.

(Single) Feature search

The target does not share any features with the other, distracting objects in the displays -> appears to "pop-out". Target has a unique feature that is not shared by other items in a display.

Conjunction Search

The target does share one or several features with the distractors; thus, the target cannot be detected by inspecting only colour, or only shape (letter), but we need to take into account a conjunction of the two features. Target has no unique feature, making visual search more difficult.

Binding via synchrony

The temporal correlation hypothesis. First proposed by C. von der Malsburg (1981). Singer & Grey (1995) suggested that gamma oscillations (~40Hz) might be particularly relevant for binding by temporal synchrony. Evidence to date is ambiguous: Some EEG studies found enhanced gamma oscillations in conditions that require more binding and spatial attention. Other studies (e.d. Bong et al., 2008) found that in monkeys, whether neurons were responsible to different features of the same object or of different objects had no effect on the synchronization of their firing pattern.

Treisman & Gelade (1980) and Treisman & Schmidt (1982) study (FIT)

They studied participants' reaction times in detecting the target under conditions of feature and conjunction search. In addition, they manipulated the number of items within the search array, the set size.

Feature Integration Theory (Treisman 1988, 1992)

Tries to explain why visual search is much harder under conjunction, conditions than under single feature search conditions. FIT assumes that all features of objects (like size, shape, orientation, colour) are processed in separate modules and are separable from the object itself (e.g., a ball is a ball, but it can have a certain colour, move in a certain direction etc.). The separate features are encoded by our perceptual system in parallel (i.e. simultaneously), even without paying attention. Therefore, when the target in the display has one unique feature that distinguishes it from all other objects, we can identify the target very fast and without paying attention to each of the objects in the display (pop-out). However, when the target shares features with the distractors, then several features of objects need to be integrated, which requires the allocation of spatial attention. Under these conditions of conjunction search we therefore need to search the display serially (object-by-object).

Deficits in Feature Binding: Balint's Syndrome

an attentional deficit that is characterized by deficits in binding the features of several different objects. In 1999, Treisman reported the case of patient RM who had an extensive bilateral stroke damaging large areas of hus occipito-temporal cortex. Among other impairements, has was unable to focus attention on more than one object at a time- simultanagnosia. the most prominent attentional deficit was that he made a large number of illusory conjunction errors even when objects were presented for up to 10 seconds. It has been replicated since that lesions of the parietal lobe can cause feature binding deficits. Interestingly, the increase in illusory conjunctions is only observed when objects are presented simultaneously but in different spatial locations (requiring binding across space), but not when objects are presented in the same spatial location one after the other (requiring binding across time). Similar impairments in binding have been reported after damage to certain sub-regions of the thalamus, suggesting that the thalamus also plays a (probably early) role in feature binding.

Guided Search Theory

assumes that every item in a display initially (and pre-attentively) produces an activation map, where each item has a certain level of activation. Objects that contain target features (e.g., red objects or A-shaped objects during a search for a red A) receive greater activation because they are relevant for our goal-directed search. Attention is then directed towards items on the basis of their initial level of activation, starting with those locations in the display that have the highest activation values. This explains longer search times when several items (even distractors amongst themselves) share features, because then many locations in the display have high activation. Guided search theory also explains why FIT often over-estimates search times in conjunction search: according to guided search, conjunction search can be more efficient that purely serial search because parts of the display (the parts containing low activation because the items don't share any features with the target) can be ignored straight away.

Regarding real world searches, the Dual Pathway Model also predicts that

there is a selective (rather serial) and a non-selective (rather parallel) pathway. The selective pathway helps us to search through a display in a goal-oriented way (e.g., find the red target we are looking for). However, the selective pathway has a bottleneck, and can thus only process a limited amount of information simultaneously. The non-selective pathway can take in much more information in parallel, and allows us to rapidly gain a gist of what the scene represents (e.g., a kitchen, a car park). This gist helps us to pre-activate locations where we would expect to find our target (e.g., humans more likely on the floor than in the sky).

illusory conjunctions are not just driven by guessing

they can occur with high confidence, and not under all circumstances that impair performance (e.g. just making the task harder to do). Also, illusory conjunctions are misbindings across space, but they never occur across time (e.g. if a red ball is shown first, and then quickly after the blue triangle, people correctly report the features).

Visual search paradigms

used to investigate visual search, where a target within a visual display has to be detected as rapidly as possible, i.e. participants decide whether the target is present or absent.

Summary

visual search studies suggest that attention is required to bind different features of objects together. Without spatial attention, we tend to misbind the features of objects. Feature integration can explain many but not all results from visual search tasks. And neuropsychological and imaging/stimulation studies suggest that the parietal lobe plays a particularly important role for binding features across space.