24-29 March 2024
BHSS, Academia Sinica
Asia/Taipei timezone

Exploration on Using AI-Generated Scenario-Based Videos on English Learning: A Focus on ESG Initiatives

Mar 27, 2024, 2:20 PM
20m
Conf. Room 1 (BHSS, Academia Sinica)

Conf. Room 1

BHSS, Academia Sinica

Oral Presentation Track 4: Social Sciences, Arts and Humanities (SSAH) Applications Humanities, Arts & Social Sciences Application

Speaker

Yu-Xuan Dai (National Taipei university of technology)

Description

This research investigates the impact of AI-generated scenario-based videos on the English-speaking learning experience and motivation, particularly within the context of Environment, Society, and Governance (ESG) initiatives. Recognizing the global significance of ESG initiatives for long-term corporate value and sustainability, the study emphasizes the need to integrate theoretical frameworks, practical insights, and diverse knowledge systems. In the Taiwanese context, English education has traditionally been influenced by cultural norms and exam-centric approaches, emphasizing written expression and reading skills over oral communication. However, effective English communication is crucial for advancing environmental protection initiatives, necessitating a shift in educational focus. Current research indicates a significant gap between engineering tasks and the predominant need for writing and communication in professional settings. Scenario-based learning is proposed to bridge this gap, fostering 21st-century skills such as problem-solving, communication, critical thinking, and creativity.

The research aims to assess whether AI-generated scenario-based videos, enriched with animation and realistic static images, can enhance the learning experience in English-speaking scenarios. Utilizing the AI video generation tools Steve.ai and Flex Clip, the study employs ChatGPT to create contextually rich English-speaking learning content. The content is structured using a prompt template from ChatGPT and adapted from MovieFactory, a robust system for generating high-resolution cinematic images and multi-modal films based on natural language input. The study employs a within-subject experimental design to evaluate the effectiveness of AI-generated videos, comparing the outcomes of the same content regarding picture quality, audio, voices, and animations. The evaluation of the learning experience is based on individual survey questionnaires gauging learning engagement and motivation. Individual differences in prior knowledge significantly influence the preference for static images or animations within specific domains of knowledge or skills. The hypothesis is that individuals with proficient English skills would better comprehend AI-generated animation content. The participants in this pilot study are thirteen graduate students with experience working and speaking English with foreigners, including six men and seven women from 22-27 years old. The experiment procedure contains three main parts, including the first interview part, which is about the difficulty of speaking English. The second part of the interview uses videos to learn English speaking. The third part is watching two types of visual representation AI-generated videos for learning English speaking in the scenario of ESG. The findings from the descriptive analysis and interview transcriptions indicate that individuals with advanced English proficiency (C1 and C2 levels) exhibit better comprehension of the animation learning material. This finding is attributed to the clarity, coherence, and interactivity facilitated by the characters' actions and eye contact in the AI-generated video. In contrast, participants possessing intermediate proficiency (B2 level) indicate that the primary issue with videos featuring static AI-generated images lies in the inconsistency between the content and the static images. On the other hand, those with lower proficiency (A2 and B1 levels) prefer realistic images, emphasizing the reduced cognitive load and enhanced focus on subtitles. Additionally, they note that static real images serve as visual aids, aiding content comprehension. However, the AI-generated images may need to align better with the learning material.

Future work will explore the impact of AI-generated videos on learning motivation using the ARCS model, which will specifically explore the relationship between input and output prompts, providing insights for future scenario-based learning video generations. Overall, the study contributes valuable findings of prior knowledge that can lead to different learning experiences of using AI-generated scenario-based videos for English-speaking learning experiences.

Primary authors

Ryan Sheng-Ming Wang (Department of Interaction Design, National Taipei University of Technology) Yu-Xuan Dai (National Taipei university of technology)

Presentation materials

There are no materials yet.