Multimodal Projects in Research,Pedagogy, and Publication - Digital Futures Institute (DFI)

About Digital Futures Institute (DFI)

“The Teachers College Digital Futures Institute (DFI) is an interdisciplinary and research-driven endeavor that harnesses the expansive capacities of digital media and technologies for inquiring into, shaping, and transforming educational, health, and psychological outcomes worldwide. DFI is framed by a joint scholarship and service approach in which excellence in service to the TC community will be matched by excellence in innovation about digital futures research, pedagogy, and public engagement.”

Workshop Description

Have you thought about taking a multimodal approach to research, pedagogy, or publication, but want more support to get your project started? Have you started experimenting with multimodal course assignments, research projects, or publications and want to talk through next steps? In this week-long workshop, we will take a project-based approach to supporting multimodal pedagogy, research, and publishing. Scholars will be introduced to introductory mediamaking tools, and we will address methodological and theoretical background for multimodality and models and possibilities for multimodal academic work.

Today was the first day to the workshop. I wasn’t quite sure what the workshop was specifically about, but as multimodality is a big part of my research on visual literacy and pedagogy - I was curious to how other scholars were carrying out their quantitative/qualitative research with multimodality.

First thing I noticed - there were a lot of scholars from different disciplines. Animators, musicians, public school teachers and professors, faculties at Teachers College, etc. It was refreshing to see these people coming together under multimodality. Goes to show that multimodality is truly multimodal.

We first went around and shared our goals for the workshop. I didn’t know the agenda was a roughly finished project, and also took a long time to figure out why I was there.

My goal was to explore how to create multimodal curriculum in public school art classes that emphasizes students to be fluent readers AND also creators of multimodality.

Some participants wanted to discuss the theoretics behind multimodality, and some wanted to learn iMovie for their multimodal project. How do you welcome such diverse and disparate agendas into one coherent field? Unless it’s focused, you can’t teach teachers how to navigate iMovie AND dive deep into multimodality frameworks all in three workshop sessions…

Also, multimodality can be endangered into becoming such a throw away phrase if not careful. So we briefly went around and discussed what multimodality meant for each person. For me, multimodality means the relationship between images, text, graphs, audio, typography, etc. in one visual field. Here I wanted to distinguish clearly the difference between multimodality of mediums vs. multimodality of technologies. For example, in an art class I can teach my students how tiktok videos are comprised of visuals, music, and texts, and how these elements are in relationship with each other. I could also teach my students how to navigate around iMovie, Photoshop, etc. Multimodality can function within one visual field or between different technologies.

Below are some of the reasons why multimodality was important to the attendees.

Why is multimodality important?

1) Meta cognitive - what are the pictures/audio/etc. doing?

2) Communication that traverses different languages and cultures

3) Engagement

4) Accessibility

5) Multi-sensory meaning making & understanding including kinetic, audio/music

Problems

1) What to focus when teaching multimodality? Either technology or medium?

2) Multimodality requires at least superficial understanding of different technologies involved (video editing, photography, writing, audio editing, etc.) How do teachers or non artists create multimodality when they don't have knowledge of these apps and softwares?

3) Definition defers from discipline to discipline

4) Different from culture to culture