视觉叙事理论国内外研究现状

… it in the theory of Narrative Grammar (specifically Visual Narrative Grammar for the visual-graphic … for a theory of narrative structure by describing the basic narrative categories and their …

… Visual Narrative Reader, brings together both older seminal essays in visual narrative theory … Cohn, himself, is most interested in the cognitive aspects of visual narrative theory or how …

Abstract The past decade has seen a rapid growth of cognitive and brain research focused on visual narratives like comics and picture stories. This paper will summarize and integrate this emerging literature into the Parallel Interfacing Narrative‐Semantics Model (PINS Model)—a theory of sequential image processing characterized by an interaction between two representational levels: semantics and narrative structure. Ongoing semantic processes build meaning into an evolving mental model of a visual discourse. Updating of spatial, referential, and event information then incurs costs when they are discontinuous with the growing context. In parallel, a narrative structure organizes semantic information into coherent sequences by assigning images to categorical roles, which are then embedded within a hierarchic constituent structure. Narrative constructional schemas allow for specific predictions of structural sequencing, independent of semantics. Together, these interacting levels of representation engage in an iterative process of retrieval of semantic and narrative information, prediction of upcoming information based on those assessments, and subsequent updating based on discontinuity. These core mechanisms are argued to be domain‐general—spanning across expressive systems—as suggested by similar electrophysiological brain responses (N400, P600, anterior negativities) generated in response to manipulation of sequential images, music, and language. Such similarities between visual narratives and other domains thus pose fundamental questions for the linguistic and cognitive sciences.

… there has been a ‘constant interaction between narrative intent’ and illusionist ‘pictorial realism’ … painting was predominantly narrative from the Renaissance to the nineteenth century. …

… – as in the visual nar… theory of Visual Narrative Grammar (VNG) (Cohn 2013b) overlaps with notions from film theories and can apply to films. Since VNG has been designed as a theory …

… effective methodologies for approaching visual artefacts so that it … visual narrative. We suggest that the framework provides an effective and general foundation for reengaging with visual …

Abstract Drawn sequences of images are among our oldest records of human intelligence, appearing on cave paintings, wall carvings, and ancient pottery, and they pervade across cultures from instruction manuals to comics. They also appear prevalently as stimuli across Cognitive Science, for studies of temporal cognition, event structure, social cognition, discourse, and basic intelligence. Yet, despite this fundamental place in human expression and research on cognition, the study of visual narratives themselves has only recently gained traction in Cognitive Science. This work has suggested that visual narrative comprehension requires cultural exposure across a developmental trajectory and engages with domain‐general processing mechanisms shared by visual perception, attention, event cognition, and language, among others. Here, we review the relevance of such research for the broader Cognitive Science community, and make the case for why researchers should join the scholarship of this ubiquitous but understudied aspect of human expression.

Visual narratives of sequential images – as found in comics, picture stories, and storyboards – are often thought to provide a fairly universal and transparent message that requires minimal learning to decode. This perceived transparency has led to frequent use of sequential images as experimental stimuli in the cognitive and psychological sciences to explore a wide range of topics. In addition, it underlines efforts to use visual narratives in science and health communication and as educational materials in both classroom settings and across developmental, clinical, and non-literate populations. Yet, combined with recent studies from the linguistic and cognitive sciences, decades of research suggest that visual narratives involve greater complexity and decoding than widely assumed. This review synthesizes observations from cross-cultural and developmental research on the comprehension and creation of visual narrative sequences, as well as findings from clinical psychology (e.g., autism, developmental language disorder, aphasia). Altogether, this work suggests that understanding the visual languages found in comics and visual narratives requires a fluency that is contingent on exposure and practice with a graphic system.

… narrative have been proposed from many perspectives and most of these nowadays promote further the notion that narrative … A multimodal discourse theory of visual narrative. Journal of …

… Narrative have increasingly featured papersand sessions on the intersections between scholarship on narrative … work by analysts of graphic narrative, including Jeanne Ewert's and Erin …

… photographs implied in light of narrative theory. When the Oklahoma City bombing occurred, Americans had to readjust and reevaluate the narrative that they believed about domestic …

Visual communication design (VCD) is a form of nonverbal communication. The application of relevant linguistic or semiotic theories to VCD education renders graphic design an innovative and scientific discipline. In this study, actual teaching activities were examined to verify the feasibility of applying narrative theory to graphic design courses. Matched group design was employed to equally divide 30 participants into experimental and control groups, who participated in distinct activities over a 4-week period. The results revealed that incorporating narrative theory into graphic design courses enabled increasing students’ poster design capabilities across various dimensions, including thematic concept, image creativity, and visual aesthetic. Narrative is a storytelling method. Applying narrative techniques to VCD not only facilitates the creativity of designers, but also elicits the audience’s visual memory, thereby encouraging a bidirectional communication between the two entities.

… This article introduces the EPIC framework for translating nuanced brand meanings into … visual content. Integrating the literatures on visual storytelling, archetypes, and critical visual …

While the consumption of visual information becomes a daily commodity integrated into our lives, data visualisation is dominated by dashboards and charts. The main contribution of this article is an advanced way to visualise data as a data story. We converged paradigms from digital storytelling, serious games, and data visualisation to turn data into useful insights. The creation, management, and analysis of data have been increasingly given more attention in industry and professional practices. However, the potential of packaging data and analytic results into easily digestible and visually explorable content intended for non-professional audiences has not yet been investigated to its full extent. We contributed towards overcoming the gap between data analytics and data presentation. By integrating a story-like environment and entertainment into data visualisation, we explore the possibilities of efficiently communicating data and insights to general audiences in a casual context. We present this modular approach to customising messages for visual data storytelling from an information and communication perspective, including a test prototype developed to illustrate our data storytelling framework.

Visual analytics usually deals with complex data and uses sophisticated algorithmic, visual, and interactive techniques supporting the analysis. Findings and results of the analysis often need to be communicated to an audience that lacks visual analytics expertise. This requires analysis outcomes to be presented in simpler ways than that are typically used in visual analytics systems. However, not only analytical visualizations may be too complex for target audiences but also the information that needs to be presented. Analysis results may consist of multiple components, which may involve multiple heterogeneous facets. Hence, there exists a gap on the path from obtaining analysis findings to communicating them, within which two main challenges lie: information complexity and display complexity. We address this problem by proposing a general framework where data analysis and result presentation are linked by story synthesis, in which the analyst creates and organises story contents. Unlike previous research, where analytic findings are represented by stored display states, we treat findings as data constructs. We focus on selecting, assembling and organizing findings for further presentation rather than on tracking analysis history and enabling dual (i.e., explorative and communicative) use of data displays. In story synthesis, findings are selected, assembled, and arranged in meaningful layouts that take into account the structure of information and inherent properties of its components. We propose a workflow for applying the proposed conceptual framework in designing visual analytics systems and demonstrate the generality of the approach by applying it to two diverse domains, social media and movement analysis.

Research findings in biomedical science are often summarized in statistical plots and sophisticated data presentations. Such visualizations are challenging for people who lack the appropriate scientific background or even experts who work in other areas. Scientists have to maximize knowledge dissemination by improving the communication of their findings to the public. To address the need for compelling and successful infographics in biomedical science, we propose a new theoretical framework for Visual Storytelling and illustrate its potential application through two visual stories, one on vaccine safety and one on cancer immunotherapy. In both examples, we rely on solid data and combine multiple media (photographs, illustrations, choropleth maps, tables, graphs, and charts) with text to create powerful visual stories for the selected target audiences. If fully validated, the proposed theory may shed light into non-traditional techniques for building visual stories and further the agenda of creating compelling information visualizations.

Big Data research, and the development of sophisticating data mining methods requires besides algorithm research, also finding better ways to convey information contained in data sources to the end-user. Through improving data visualization methods, we are able to create visual presentations of data sources allowing users to gain a deeper understanding of data. For several millennia, stories where the major medium to communicate complex messages between humans. Within this research work we attempt to converge both, storytelling and data visualization through the development of an interactive data storytelling framework. Our goal is to communicate knowledge contained in data to a general audience and utilize Australia's energy consumption data as an exemplary case for visualizing aspects as energy consumption and production. We describe the essential elements and theories contributing to this framework, and give special attention to user-centered design aspects. We identify end-user requirements, and illustrate the practical application of the overall framework through a prototype implementation.

A growing number of scientific publications is available to promote sustainable river management. However, these publications target researchers rather than water management professionals who are responsible for the implementation of management practices. To bridge this science-to-practice gap, we conceptualize and propose a series of steps to prepare effective storylines targeted at a practitioner audience. We developed this approach within a research program that supports integrated and collaborative river management. We prepared three storylines, each based on one scientific publication. The storylines combined text and interactive visuals using the ESRI StoryMaps tool to make them available online. Via focus groups with 44 participants from research and practice, we evaluated the perceived usefulness of and engagement with the content and design. We collected feedback from participants using a survey as well as via audio and screen recordings. Our findings show that we should narrow down the audience of the storylines by tailoring them to the needs of project managers rather than specialized advisors. Therefore, the content should offer more than a visual summary of the research by showing examples of the management application. A more engaging sequence with a clear protagonist is further required to better relate to the problem and the potential application. Although visuals and interactive elements were considered attractive, a multi-disciplinary editorial team is necessary to better complement the visuals’ design to the text. The level of detail of participants’ feedback shows that involving project managers to co-create storylines can be an important step for improvement.

ABSTRACT Visual and digital storytelling methods can reposition research participants as coproducers of knowledge, foster engagement and collaboration with marginalized peoples, and offer greater depth of self-expression. However, these methods are constituted in complex terrains of power. Without continual attenuation to power imbalances, the methods will contribute to the silencing and erasure of marginalized communities. This study outlines how reflexivity as a methodological tool and part of the Cultured-Centered Approach can enable the interrogation of terrains of power, allowing for the continual opening of democratic possibilities and community ownership of visual and digital storytelling infrastructures. Excerpts from the “Poverty Is Not Our Future” campaign illustrate the argument. The campaign's cocreated audio-visual advertisements communicate everyday stories of poverty among residents living in a poor suburban site in Auckland, Aotearoa New Zealand, and serve as a visual narrative of resistance to dominant structures. This study contributes to critical theorizing of culture and communication and the coconstruction of visual stories.

ABSTRACT In this article, I review considerations and techniques for approaching cartographic design as visual storytelling. Stories, like maps, are a method for documenting and explaining, for meaningfully abstracting our experiences, for communicating and sharing, and for asserting a particular worldview. I argue that visual storytelling offers an entry point for hybridization in cartography, uniting technology with praxis, product with process, and design with critique while opening rich new avenues for transdisciplinary research and design. I begin by introducing influences on map-based visual storytelling and review ten recurring themes that make visual storytelling different from traditional perspectives on cartographic design. I then offer three of potentially many ways to articulate and organize the design space for map-based visual storytelling: foundational narrative elements and their adaptation to geographic phenomena and processes, visual storytelling genres delineating different story experiences, and visual storytelling tropes used to advance narratives across text, maps, images, and other multimedia. I conclude with a call for future research on visual storytelling in cartography, including visual design, visual ethics, and visual literacy.

Qualitative analysis is often a textual undertaking. However, it can be helpful to think about and represent study phenomena or narrative accounts in nontextual ways. In this article, we share our unique and artistic process in developing and employing pictorial narrative mapping as a qualitative analytic technique. We recast a nontextual, artistic–analytic technique by combining elements related to narrative mapping and narrative art. This technique involves aesthetic attunement to data and visual representation through pictorial design. We advanced this technique in the context of a narrative study about how arts-informed dissemination methods influence health-care practitioners’ delivery of care. We found that the Pictorial Narrative Mapping process prompted an aesthetic and imaginative experience in the analytic process of qualitative inquiry. As an analytic technique, Pictorial Narrative Mapping extends the inquiry process and enhances rigor through artistic means as well as iterative and critical dialogue. Additionally, pictorial narrative maps can provide a holistic account of the phenomenon under study and assist researchers to make meaning of nuances within complex narratives. As researchers consider employing Pictorial Narrative Mapping, we recommend that they draw upon this technique as a malleable script yielding to an organic process that emerges from both their own data and analytic discussions. We are further curious about its imaginative capacities in social and health science literature, its possibilities in other disciplinary contexts, and the prospects of what Maxine Greene refers to as becoming more wide awake—in our case, in future research analytic endeavors.

Abstract This article proposes the methodological framework of visual narrative analysis through the study of images and narratives. We are interested in the appeal of political storytelling. In applying an approach of layered interpretation, we study images and slogans to consider the more complex underlying narratives in their political and cultural context. Our exploratory case studies draw on material from right-wing populist parties, namely election campaign posters from Germany and the UK as material for the analysis. We find that narratives operate with a ‘fantasmatic logic’, which adds fantasy to politics, to depoliticise and camouflage their radical intent and gain approval by making consent desirable. We identify two exemplary narratives (honest men under threat; proud mothers) that entrench traditional gender roles in accordance with patriarchy and nationalism. Theoretically, our approach contributes to debates in IR on cultural underpinnings in international politics and the construction of collective identities through shared/divided narratives. Visual narrative analysis provides a promising methodological tool for analysing visual representations in their productive relationship with text. This perspective foregrounds the power of political storytelling through fantasmatic appeal and fosters a better understanding of the global rise of populism.

… Instead, I argue for the centrality and the specifically narrative roles of the pictorial model. To … of an inside observer, it enters into narrative patterns such as plot and characterization, as …

… Our analysis is underscored by comparisons with other idiosyncratic artists, from Max Beckmann in exile in Amsterdam to RB Kitaj confronting his critics in London. Artworks that are '…

In our everyday discourse we make frequent reference to pictorial narratives. We exclaim on the hunt scene in the cave painting, the frenzy unfolding in the graffiti, the adventure of the baby in the book illustration, and the disintegration of a marriage in the oil painting. Yet a more precise question concerning narrators and their relation to these so-called pictorial narratives remains overlooked. Theoretical debates in narratology are still primarily focused on literary narratives and so pictures remain relatively neglected as a class. Kendall Walton is an exception. He argues that the literary narrator is necessary to provide access to the story as ‘he mediates the reader’s access to the rest of the fictional world’. He says that pictorial narratives cannot sustain narrators akin to literary counterparts. But this seems to be at odds with how we understand paintings such as Marriage A-la-Mode, where events are arguably recounted to the viewer with wicked humour. This paper has two main aims. The first is to set out what is meant by ‘pictorial narrators’ by providing a succinct and up-to-date guide to the discussions that have touched on this issue. The second is to explore the possibility that pictorial narratives imply pictorial narrators.

… analysis doesn't derive from the extension of the categories used to analyze the pictorial representation of spatial objects to the description of the ways how narrative texts are …

… see, also involves other dimensions of analysis. It is not within our scope to enter into the question of art historians’ choices of analysis nor of the definition of narrative modes. For the …

… is then woven together into a coherent narrative (Plot, which may … This will be explained further in the section on “Narrative … The reason why our linguistic analysis of the narrative text …

… This article focusses on an analysis of selected instances of painting on religious themes, … into stories' and, subsequently, that'a narrative impulse is inscribed in the creation of images'(p…

… detailed manner, by continuous pictorial narrative, we mean a … analysis for each of the categories considered was then carried out, reporting the raw data, as we shall see in the analysis …

Abstract Unacknowledged by its practitioners, narratology has often been revisionary rather than descriptive when categorizing narratives. This is because definitions, expert judgment and personal intuition, traditionally the main tools for categorization, are vulnerable to media blindness and to being theory loaded. I argue that to avoid revisionary accounts of ordinary everyday practices such as narrative or gameplay of which non-experts have a firm understanding, expert categorizations have to be tested against folk intuitions as they become apparent in ordinary language. Pictorial narrative in single pictures is introduced as a specific case of categorization dispute and an experiment laid out in which non-experts assess if different pictures tell stories. As the chosen pictures correspond to different criteria of narrative to varying degrees, the experiment also serves as an implicit test of these criteria. Its results confirm monochrony compatibilism, the position that single monochronic pictures can autonomously convey stories. While the pictures rated high in narrativity correspond to traditional criteria of narrative, I argue that the way in which these criteria are usually interpreted by narratologists is problematic because they exclude these pictures from the realm of narratives. It is argued that the way marginal phenomena are categorized is essential for a sound understanding of even the most paradigmatic objects of a domain because categorizations influence definitions and definitions ultimately guide interpretations.

… Holmes’s skills in evidence analysis, though it is fictional … narrative features of story-telling are not only applicable to textual information. They can be extended to tell a story in pictorial …

… narrative processes, including those of focalization. Yet although there is growing critical interest in graphic narratives,… , graphic narratives rely on an even blending of semiotic modes to …

… One may conclude that graphic narratives range between … at: according to semiotic approaches, graphic narratives are … approaches, the graphic narratives' semiotic types, word and …

… study of Jarvinia, focusing on her semiotic decision making when she designed her graphic … graphic narratives because the compositions were neither a comic book nor a graphic novel. …

… the mixing of different semiotic resources – … of semiotic modes in the comics medium and their dread of racial mixing. As I argue in the second part of this article, my approach to semiotic …

Abstract The paper is a study of how graphic narratives (graphic novels and the comics) represent time in external visual space as well as in inner (mental) representations. Peirce’s semiotics is the main tool of research. After a survey of various approaches to the study of time in narratives in general and in graphic narratives in particular, an outline of the various aspects of the embodiment of time in space in general is given before the forms of the embodiment of time in the space of graphic narratives is examined in detail. Signs of time are signs that represent time as their object and create mental representations of time as their interpretant. On this semiotic premise, the paper proposes a reinterpretation of Genette’s theory of time in narratives. A comparison with narratives in prose reveals that narrative time in graphic narratives evinces marked discontinuities in their spatial representation. The paper distinguishes between time as continuity, time as age, and points in time. It concludes that time as continuity is typically embodied in the form of spatial diagrams whereas time as age and points in time are mainly of the nature of indexical signs. The insights into the forms of embodiment of time in space derived from these premises are applied to examples from recent graphic novels and some traditional comics.

ABSTRACT An empathetic patient-practitioner relationship has been shown to improve treatment and recovery among patients, and decrease the likelihood of malpractice suits among medical professionals. It is, therefore, puzzling why an empathetic approach to medical education through medical humanities has not been more widely adopted. The interdisciplinary field of medical humanities includes topics such as graphic narrative. Operating from a discourse analytic perspective, this paper presents a dissection of the place of graphic narrative in medical education grounded in the visual semiotic theories of Kress and Van Leeuwen (2001) and Jewitt and Oyama (2001). Graphic medicine is a form of graphic narrative that gives those suffering with, treating, or caring for loved ones with an illness, a form of expression that isn’t available through verbal communication. Through semiotic analysis of representational, interpersonal, and compositional metafunctions of four graphic narratives composed by medical students at Penn State College of Medicine, this paper explores the significance of empathy in the doctor–patient interaction, revealing that an empathetic approach to medicine is valuable to patients and their families, as well as to doctors and medical students, and should be further implemented beginning in medical school.

… in the graphic narratives composed by 12-year-old students. The student graphic narratives featured in … metalanguage that describes the semiotic resources of the medium of comics. …

… the contributions made in narrative semiotics. One approach … the causal cohesion of the semiotic discourse. Although I am … representative of the semiotic approach to the narrative. Ideally…

… across strata (ie context, discourse semantics, lexicogrammar and … multimodal discourse. The SF-MDA approach has, for example, led to the study of the functionality of language, visual …

Multimodal Discourse Analysis of Advertising Language In 2009, Zhang Delu constructed an integrated framework for multimodal discourse analysis. This framework includes four levels: cultural level, context level, content level and expression... How To Do Critical Discourse Analysis A Multimodal … A Multimodal Introduction. David Machin and Andrea Mayr. 3. Presenting Speech and Speakers: Quoting Verbs. DISCOURSE AND TECHNOLOGY Multimodal Discourse … Multimodal Discourse Analysis The subtheme of GURT 2002, multimodal discourse analysis, was intended to high-light the recognition discussed in many of the papers in this volume that all discourse is multimodal. That is, language in … Multimodal Discourse Analysis of “Union Is Strength” from the ... anings of visual grammar to explore how images reproduce meaning through various symbols. The content studied in this article includes multimodal discourse encompassing various symbolic resources such as images, sounds, Multimodal Discourse The Modes And Media Of … Using a multimodal discourse analysis approach, this dynamic collection examines various discourses, modes and media in circulation during the early stages of the pandemic, and how these have impacted our daily lives in terms of the various meanings they express. Intersemiotic Complementarity: A Framework for Multimodal … In this chapter a descriptive framework for the analysis of page-based multimodal texts is introduced and applied to a multimodal text extracted from the Finance department of The Economist magazine. Multimodality, Systemic Functional-Multimodal Discourse … Systemic Functional–Multimodal Discourse Analysis (SF-MDA) is a theoretical approach that analyzes scenarios involving the production of meanings through the combination of socially constructed resources to meet the communicational needs of individuals. Multimodal Discourse

This chapter is dedicated to the critical analysis of multimodal texts - i.e., texts that incorporate semiotic resources beyond verbal language. Our focus here is, in particular, on the relationship between the verbal and the visual mode of communication. In a first section, we touch briefly on the ubiquity of multimodality in contemporary society and provide definitions of core concepts. We then continue, in a second section, to systematically develop an argument for the relevance of multimodality within critical disocurse analysis (CDA). The third section summarizes a number of exemplary studies that have adopted a critical approach in multimodal discourse analysis. These studies cover a variety of issues and areas of scholarly inquiry, and therefore aptly demonstrate that multimodal CDA can be applied in various forms. In the fourth section, we introduce - as a practical example - one particular methodological appraoch in more detail, and illustrate its various analytical steps on the basis of two selected multimodal texts. We close with a brief reflection and some concluding remarks.

… the visual and verbal modes, to explainjust what features make multimodal text visually-… of pagebased multimodal texts is introduced and applied to a multimodal text extracted from …

The area of discourse analysis has long neglected the value of images as a semiotic resource in communication. This paper suggests that like language, images are rich in meaning potential and are governed by visual grammar structures which can be utilized to decode the meanings of images. Employing a theoretical framework in visual communication, two digital images are examined for their representational and interactive dimensions and the dimensions’ relation to the magazine advertisement genre. The results show that the framework identified narrative and conceptual processes, relations between participants and viewers, and symbolic attributes of the images, which all contribute to the sociological interpretations of the images. The identities and relationships between viewers and participants suggested in the images signify desirable qualities that may be associated to the product of the advertiser. The findings support the theory of visual grammar and highlight the potential of images to convey multi-layered meanings .

… the construction of multimodal mental models which encode information derived from visual – including mediatised visual – experience, then the increased visual information we receive …

Multimodal discourse analysis is the analysis of different symbolic modes within a text, which breaks through many limitations of traditional discourse analysis to a great extent. This paper takes the visual grammar of Kress and Leeuwen as the theoretical framework, which gives a good explanation of the reproducing meaning, interactive meaning and composition meaning of image discourse, which is also suitable for the analysis of news picture discourse. This paper expounds how other symbolic resources interact with each other, so as to construct a complete text with linguistic symbols, and then convey more social interactive meaning. The results show that visual grammar is feasible and operational in the analysis of multimodal news texts. The background and text of news discourse can be effectively supplemented and explained, and it is of great significance to improve readers' pictures’ reading ability.

… Multimodal Discourse Analysis approach, to study the underlying semiotic mechanisms through which visual … of specialized discourses such as physics and mathematics is reduced by …

This paper explores the visual agency in political campaign discourse in southwestern Nigeria by analysing some visual images that are used for political communication in the political rally context. Data for the study were obtained from 12 political rallies, two each of the six south-western Nigerian states. Systemic functional multimodal discourse analytical framework and Barthes’ conception of the interaction of signs encapsulated in the concepts of “anchorage” and “relay” provide the theoretical perspective for the analysis of the data. Data analysis reveals that the discourse participants use semiotic artefacts such as vests, ankara and head wears to communicate messages relating to the visions, ideals, abilities and promises of the political parties and politicians to the electorate in order to gain their acceptance. The paper reveals further that the visual signifiers in the discourse were also used to give aesthetic and affective appeal to the discourse. The paper concludes that the visual resources communicated political meanings and highlighted certain cultural and social factors in the Nigerian environment.

… This multimodal discourse study examines visual images in selected online news discourse on the 2023 presidential elections in Nigeria to identify the various meanings which the …

ABSTRACT

Abstract Mental model theory is taken by scholars like Teun van Dijk as the fundamental idea of epistemic representations about the situations, events, and actions of the natural and social world. Modified by dynamic as well as situationally variable cognitive context (communication) models, the structures of such models, and therefore of the world knowledge, are claimed to multiply affect the production and interpretation of text and talk— e.g., in storytelling, argumentation, expository discourse, the level and specificity of characterisation, (non-)metaphoric blends, etc. Hence, the theory of context models also forms the basis of pragmatics. In this article, it is argued that visual and multimodal discourse will benefit from adopting van Dijk's mental model theory as the basic model for analysis. This will demand adaptation of the model, which has hitherto been primarily employed to examine verbal mass-communication or interaction, to multimodal mass-communication. The article offers tentative suggestions on how this can be done, using a corpus of political cartoons and Internet memes.

… for multimodal analysis, which we exemplify in a case study of how neoliberal management discourse … Placing this within the existing sub-fields of multimodality that have emerged, we …

In addition to words, the symbols, colors, sculptures, photographs, music, etc. are also frequently employed by participants to express themselves in communication. Advertising is closely related to sounds, colors, picture animations and other symbols. This paper aims to present how semiotics acts effectively to realize the real business purpose to reflect the unique significance of the multimodal discourse analysis. Based on Visual Grammar, this paper analyzes the 2014 Brazil World Cup advertisements from the perspective of representational meaning, interactive meaning and compositional meaning, this research means to prove that different modes within an advertisement depend on each other and have an interdependent relationship. And these relationships have different roles in different contexts.

… between the visual and the rhetorical realization of multimodal discourse supports the adoption … form a visual entity but lack functional cohesion or, conversely, are visually disconnected …

… Since research on visual or multimodal argumentation often challenges traditional conceptions of what an argument is, O’Keefe discusses, conceptualises and defines a notion of …

… To sum up and connect these narrative cultures I believe what we are pre… narrative motifs and images of the wild northern man. They are in turn set forth through oral and visual narrative …

… images can be narrative and propose an account of what makes certain images narrative … necessary and paradigmatic characteristics of narrative images, I will use examples that are …

Preview this article: Images, Words, and Narrative Epistemology, Page 1 of 1 < Previous page | Next page > /docserver/preview/fulltext/ce/58/8/collegeenglish9010-1.gif

… of absorbed narrative experiences contained many image schema related utterances. … observed construction of absorption in fictional narrative is consistent with an account of narrative …

Constituent structure has long been established as a central feature of human language. Analogous to how syntax organizes words in sentences, a narrative grammar organizes sequential images into hierarchic constituents. Here we show that the brain draws upon this constituent structure to comprehend wordless visual narratives. We recorded neural responses as participants viewed sequences of visual images (comics strips) in which blank images either disrupted individual narrative constituents or fell at natural constituent boundaries. A disruption of either the first or the second narrative constituent produced a left-lateralized anterior negativity effect between 500-700ms. Disruption of the second constituent also elicited a posteriorly-distributed positivity (P600) effect. These neural responses are similar to those associated with structural violations in language and music. These findings provide evidence that comprehenders use a narrative structure to comprehend visual sequences and that the brain engages similar neurocognitive mechanisms to build structure across multiple domains.

Reading fiction for pleasure is robustly correlated with improved cognitive attainment and other benefits. It is also in decline among young people in developed nations, in part because of competition from moving image fiction. We review existing research on the differences between reading or hearing verbal fiction and watching moving image fiction, as well as looking more broadly at research on image or text interactions and visual versus verbal processing. We conclude that verbal narrative generates more diverse responses than moving image narrative. We note that reading and viewing narrative are different tasks, with different cognitive loads. Viewing moving image narrative mostly involves visual processing with some working memory engagement, whereas reading narrative involves verbal processing, visual imagery, and personal memory (Xu et al., 2005). Attempts to compare the two by creating equivalent stimuli and task demands face a number of challenges. We discuss the difficulties of such comparative approaches. We then investigate the possibility of identifying lower level processing mechanisms that might distinguish cognition of the two media and propose internal scene construction and working memory as foci for future research. Although many of the sources we draw on concentrate on English-speaking participants in European or North American settings, we also cover material relating to speakers of Dutch, German, Hebrew, and Japanese in their respective countries, and studies of a remote Turkish mountain community.

Narrative is fundamental to the ways we make sense of texts of … the form and function of narrative in MILES by developing … concepts of narrative guidance and narrative construction and …

… Storytelling is an ancient art rooted in our common human culture, as well as in our physiology and psychology. (Bran Ferren, cochairman and chief creative officer of Applied Minds, Inc.…

Abstract Visual storytelling comes in many forms (e.g. films, comics, photographs, commercials) and is used for a range of purposes (e.g. to entertain, inform, persuade). Technological advances are enabling non-specialists to be consumers and producers of these works. Although many people are growing up surrounded by visual works, this does not mean that they carefully attend to images. Education must prepare students to navigate the changing visual landscape. This study, which investigates an undergraduate Visual Narratives course taught in spring 2017 in the United States, focuses on students’ uses of art and design elements. A content analysis of 124 course documents shows a wide range of art and design elements at work in students’ visual narrative analyses (27 elements) and original compositions (26 elements), with many elements overlapping (21 shared). These results suggest that teaching a wide range of art and design elements can help students acquire a flexible toolkit for reading and composing different kinds of visual texts, expanding their visual literacy. Rather than serving as an end goal, this foundational knowledge offers a focused way of looking that could be combined with other lenses (critical race theory, feminist theory, etc.).

… the essence of nonverbal communication, supports the idea of visual storytelling. Visual storytelling is a method of promoting or conveying a message or idea through visual elements, …

This study aims to describe visual storytelling in the animated film Leaving Home as a nonverbal communication medium that provides education to the audience. The power of nonverbal communication as a form of visual storytelling in the animated film Leaving Home is the researcher's motive to examine how a nonverbal animation can be interpreted universally. The nonverbal communication of the animated film Leaving Home embodies writing into moving images. An animator must have the ability to visual storytelling to make the imagination in telling stories (synopsis-text) into scripts (visual-written). An animator, also an animation film creator, must understand the rules of scriptwriting to communicate the message implied in his work. An excellent visual understanding will help an animator visualize the picture's language into an exciting story. The technique of conveying visual storytelling stories in animation sometimes finds obstacles in understanding the story due to improper use of visual language. This research uses qualitative methods to uncover the phenomenon of nonverbal communication through visual storytelling in the animated film Leaving Home to explain in more detail. The results showed that nonverbal communication is omnipresent, including nonverbal aspects of each communicative action. In the interactions seen in the story of the animated film Leaving Home, all its nonverbal channels come into play. This study also represents the imagination visualized in animated films as more hyperbolic. After conducting a study of nonverbal visual storytelling communication in the animated film Leaving Home, it can be concluded that films with animation techniques can be presented with visual storytelling packaged in an artistic visual language. So visual storytelling can be a literacy animation technique in conveying information and opening up opportunities for animation creators to communicate nonverbally to the audience.

This research develops a conceptual framework for telling visual stories about science using short-format videos, termed SciCommercial videos, that draw upon marketing communication. The framework is illustrated by an exemplar, the Good Whale Watching video, which is explained using a visual rhetoric keyframe analysis. Finally, the effectiveness of the video is evaluated as a science communication tool using an empirical online survey with 1698 respondents. The results highlight the benefits of using video for storytelling about science by using our framework formula, modified from marketing practices, to produce videos that are Simple, Unexpected, Concrete, Credible, Emotional, Science Storytelling (SUCCESS).

Before the days of neon lights and marquee signs, business owners had to get creative when it came to advertising their establishments. In fact, wine bars in ancient Rome used to hang a bunch of vine leaves outside their door as a nod to the God of Wine, Bacchus. And when weather conditions left them with a short supply of vine leaves, barkeeps turned to bushes inns called bush & bushes, still exist today.

Storytelling is an effective way of communicating information and knowledge, and it is widely adopted in heterogeneous contexts, from education by improving critical thinking and enhancing learning practice, to journalism by encouraging coherent stories of news supported by graphical representations. However, storytelling platforms seem to be targeted to a specific audience without showing how they can be adapted to heterogeneous needs, from class support in education to mechanisms to overcome the syndrome of the white page. In this article, we propose Novelette, a digital storytelling environment, and we show how it can be applied in heterogeneous contexts and by the different target audience. We present Novelette operating mechanisms, its architecture, and we overview different use cases, from tales creation Rodari style to data- and media-stories. By use-cases, we desire to make evident that the same platform can generate stories engaging for any target audience.

… Visual communication can facilitate this engagement—particularly when we aim to show … the benefits of visual storytelling and identifies key techniques of visual communication. …

… Although we are still generally very geared toward communication using words (and numbers) rather than visual representations, we foresee that a specific language that produces …

… spatial presence and visual coherence (Daghriri et al., … visual literacy theory guided data collection by focusing on learners’ interpretation of visual details, emotional cues, and narrative …

… , children's visual literacy becomes … of visual literacy; as students make their thinking visible and are guided to reflect upon these visual expressions, not only is their visual literacy …

Our world is overwhelmingly visual in nature and information is exchanged in many forms. Communication has shifted from reading words on a page to navigating a wide range of text sources. Messages are incomplete without visuals and therefore we must expand our definitions of literacy and what it means to be literate. It takes more than the abilities to read and write to be a literate person navigating the places in which we live and learn. Being literate must include the ability to analyze and represent ideas through visual communication. Literacy learning is not limited to reading, writing, listening, and speaking; we must include visual communication in our instruction. However, a universal definition for visual literacy has not yet been established. Diverse disciplines including art, education, linguistics, media, philosophy, semiotics, and technology contribute to an ever-evolving definition of visual literacy. Indeed, the term visual literacies has been used to represent the multiplicity of perspectives and applications appropriate for the field. Regardless of discipline, most trace their working definition of visual literacy to John Debes' seminal keynote at the inaugural convening of the International Visual Literacy Association in 1969: "Visual literacy refers to a group of vision-competencies a human being can develop by seeing and at the same time having and integrating other sensory experiences. The development of these competencies is fundamental to normal human learning. When developed, they enable a visually literate person to discriminate and interpret the visible actions, objects, symbols, natural or man-made that he encounters in his environment. Through the creative use of these competencies, he is able to communicate with others. Through the appreciative use of these competencies, he is able to comprehend an enjoy the masterworks of visual communication" (Debes, 1969, p. 27). Although Debes delineates competencies for both viewing and constructing visuals, a definition of visual literacy for education must move beyond "creative uses" and the ability to "enjoy the masterworks." A useful definition for education capitalizes on the thousands of images we traverse every day in our in-school and out-of-school lives and should not be constrained by language of any one discipline (e.g., art, technology, media). Visual literacy is defined as the abilities to critically analyze and communicate through visual texts. This broad definition accommodates the many visual mediums used to convey information within the various disciplines relevant to education. The term visual text is used intentionally to be inclusive of the kinds of representations that may appear within disciplines such as drawings, photographs, illustrations, icons, graphic organizers, paintings, infographics, maps, charts, and memes to name a few. Visual texts are expressions of thinking that use pictorial features to communicate meaning and facilitate understanding. Visuals are a language with their own semantic and syntactic intentions, often specific to their disciplines. The field of visual literacy not only focuses on the many ways we read and interpret images, but also the many ways we use visuals to convey meaning. Being visually literate include the abilities to analyze and construct visual texts. However, being literate is more than a set of competencies; it is an identity and "the visual images encountered every day play an important role in how we make sense of the world and how we see ourselves" (Serafini, 2014). This is exemplified in the way visual registers change in social context as seen in the range of images viewed while riding a bus or illustrating a complex concept. Although demonstrations of literate behaviors are still dominated by linguistic definitions of literacy, calls for an expanded literacy curricula across varied disciplines and educational contexts are ever increasing (Callow, 2005; Kedra, 2018; Serafini, 2011). Two important principles must be considered as we move conversations about visual literacy curricula forward: visual texts deserve equal status to written texts in educational contexts, and visual literacy competencies and abilities must be taught intentionally. It is essential to recognize the important roles images play in a world inundated with visual texts communicating a vast array of ideas. Images are no longer relegated to a complementary role where they elaborate or emphasize written text. Nor are they merely supplemental enhancing words on a page. Visual texts carry essential information for understanding and communicating meaning. Further, learners must be prepared to work in a world that is increasingly visual; one that could not be adequately described with words alone. As such, visual texts deserve a "privileged status, especially in relation to conventional text-based literacy" (Kedra, 2018, p. 69). To be clear, recognizing the critical roles visual texts play in communicating ideas does not in any way lessen the significance of the traditional linguistic literacies. Visual texts provide another option for demonstrating knowledge. In some cases, visual texts are the most effective communication tool to capture certain kinds of information. Many of us can navigate a map, follow steps in a diagram, and comprehend basic information in a textbook illustration. However, most of us interact passively with these powerful visual messages without focused attention. A definition of visual literacy must include intention, when viewers and visualizers make thoughtful choices and decisions about the ways they communicate ideas. Visual literacy skills are too often an afterthought or taken for granted because it is assumed learners are naturally effective consumers of visual information. However, visual literacy competencies must be taught in multi-disciplinary contexts. The expectation to include more visuals in instruction has implications for instruction. It should shift the way we think about literacy and the language arts, just as text has shifted from words on a flat surface to multidimensional and multimodal messages. Literacy researchers have long acknowledged the role of visuals in the language arts. The Standards for the Language Arts, jointly published by the International Reading Association and the National Council of Teachers of English in 1996 refer to the language arts as reading, writing, listening, speaking, viewing, and visually representing (italics added). However, little attention has been paid to the roles of viewing and visually representing ideas as literacy. More recently, the College and Career Readiness Standards (CCR) added support for enlarging our definitions of literacy to include visual texts. The Common Core State Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects requires students to create and analyze an "extensive range of print and nonprint texts in media forms old and new" (NGACBP & CCSSO, 2010, p. 4). These standards may guide efforts to support students as they develop the abilities they need to interpret and communicate complex understandings visually. The Common European Framework of Reference for Visual Literacy (CERF_VL) also calls for visual literacy instruction. Published in 2016, this edited volume highlights the relationship between visual literacy and twenty-first-century skills and provides curricula guidelines focused on metacognition and reflection. A framework was chosen over standards to provide advice and guidance to both art and general education teachers as they plan curricula. The Australian Curriculum Assessment and Reporting Authority (ACARA) provides more specific leveled expectations of visual knowledge focussing on "students understanding how visual information contributes to the meanings created in learning area contexts" (2016, n.p.)

… I have tried to show that we have good reasons for being very skeptical about the need for a specifically visual literacy as a prerequisite for understariding the kinds of visual conventions …

Abstract Visual imagery and composition inherently have the power to shape comprehension and interpretation beyond the literal. Today's students increasingly inundated with a steady stream of imagery from multimedia platforms including the Internet (i.e., social media – Facebook, Instagram, Twitter), television, film and advertisements ( White, 2012 ) are global consumers of media in their everyday lives yet they often lack the skills necessary to move beyond passive receivers of visual media messages. Visual literacy is vital for 21st Century learners and those who teach. Classrooms can become spaces for students to effectively communicate in and contribute to analytical and global dialogue for discussions of race and diversity, multicultural life and history thus encouraging students to become active deconstructionists of visual grammar.