认知心理学2013-09-05 2:57 AM

(Pea)nuts and bolts of visual narrative: Structure and meaning in sequential image comprehension

Highlights • We examined processing of narrative structure and semantics in comic strips. • Semantic relatedness and narrative structure each offer behavioral advantages. • ERPs indicate that narrative structure may act independent of semantic relatedness. • Structure and semantics interact during the comprehension of visual narrative. • Sequential image comprehension has analogies to language comprehension. Abstract Just as syntax differentiates coherent sentences from scrambled word strings, the comprehension of sequential images must also use a cognitive system to distinguish coherent narrative sequences from random strings of images. We conducted experiments analogous to two classic studies of language processing to examine the contributions of narrative structure and semantic relatedness to processing sequential images. We compared four types of comic strips: (1) Normal sequences with both structure and meaning, (2) Semantic Only sequences (in which the panels were related to a common semantic theme, but had no narrative structure), (3) Structural Only sequences (narrative structure but no semantic relatedness), and (4) Scrambled sequences of randomly-ordered panels. In Experiment 1, participants monitored for target panels in sequences presented panel-by-panel. Reaction times were slowest to panels in Scrambled sequences, intermediate in both Structural Only and Semantic Only sequences, and fastest in Normal sequences. This suggests that both semantic relatedness and narrative structure offer advantages to processing. Experiment 2 measured ERPs to all panels across the whole sequence. The N300/N400 was largest to panels in both the Scrambled and Structural Only sequences, intermediate in Semantic Only sequences and smallest in the Normal sequences. This implies that a combination of narrative structure and semantic relatedness can facilitate semantic processing of upcoming panels (as reflected by the N300/N400). Also, panels in the Scrambled sequences evoked a larger left-lateralized anterior negativity than panels in the Structural Only sequences. This localized effect was distinct from the N300/N400, and appeared despite the fact that these two sequence types were matched on local semantic relatedness between individual panels. These findings suggest that sequential image comprehension uses a narrative structure that may be independent of semantic relatedness. Altogether, we argue that the comprehension of visual narrative is guided by an interaction between structure and meaning.

KEYWORDS

SHARE & LIKE

COMMENTS

ABOUT THE AUTHOR

认知心理学

0 Following 9 Fans 0 Projects 35 Articles

SIMILAR ARTICLES

Highlights • We compare English acquisition in internationally-adopted preschoolers and infants. • Like infants, preschoolers use one word utteran

Read More

Abstract The sound |faɪv| is visually depicted as a written number word “five” and as an Arabic digit “5.” Here, we present four experiments – two qua

Read More

Infants have a bandwidth-limited object working memory (WM) that can both individuate and identify objects in a scene, (answering ‘how many?’ or ‘what?

Read More

This paper examines the judgment of segmented temporal intervals, using short tone sequences as a convenient test case. In four experiments, we investi

Read More

Much of the research on mathematical cognition has focused on the numbers 1, 2, 3, 4, 5, 6, 7, 8, and 9, with considerably less attention paid to more

Read More

It is typically assumed that count nouns like fork act as logical sortals, specifying whether objects are countable units of a kind (e.g., that a whole

Read More

One important function of categories is to permit rich inductive inferences. Prior work shows that children use category labels to guide their inductiv

Read More

The article investigates the mechanisms of selecting and updating representations in declarative and procedural working memory (WM). Declarative WM hol

Read More

Losses were found to improve cognitive performance, and this has been commonly explained by increased weighting of losses compared to gains (i.e., loss

Read More

Three studies compared bilinguals to monolinguals on 15 indicators of executive processing (EP). Most of the indicators compare a neutral or congruent

Read More