Chair: Nick Wong
Hong Kong University of Science and Technology
This colloquium presents the transdisciplinary collaborative journey of developing the Global Englishes AI-assisted Virtual Reality Interactive System (GAVIS), a QEF eLAFP-funded EdTech project addressing critical challenges in Hong Kong’s high-stakes English speaking assessment (HKDSE Paper 4). Through three interconnected presentations, we document how multiparty co-learning among experienced teachers, educational technologists, and applied linguists drove innovation across technical, pedagogical, and theoretical dimensions.
It contributes conceptual, methodological, and practical insights for researchers, educators, and technologists developing AI-enabled assessment tools, demonstrating that sustainable innovation requires progressive collaboration stages enabled by shared vulnerability, reciprocal knowledge exchange, and problem-focused thinking that transcends disciplinary boundaries.
Keywords: Transdisciplinary collaboration, multiparty co-learning, critical AI literacy, automated speaking assessment, virtual reality, high-stakes examination, HKDSE, educational technology
Presentation 1: Fluidity and Transpositioning in the researcher-practitioner partnership – The development of GELT-informed learning and assessment materials
Roy Chan Marymount University
Nick Wong Hong Kong University of Science and Technology
This talk outlines a collaborative research project established via a researcher-practitioner partnership (RPP) within a Global Englishes (GE) theoretical framework. While the ethnographic case study was used to examine the effectiveness of a GELT-informed learning and assessment materials (with the use of AI-generated avatars) embedded within a self-directed speaking training app for secondary and tertiary students in Hong Kong, translanguaging and transpositioning were used as the methodological framework and analytical tool for understanding research partnership in the current study. The study adopted a qualitative research design, collecting data from 6 school representatives through expert interviews, and via weekly diaries documenting key meeting discussion items in 12 months. In terms of the study findings, this study revealed that AI-generated avatars were considered an effective and positive affordance for students to increase their GE awareness because students in Hong Kong were unlikely to be exposed to a wide range of accents in regular homogenous classrooms. In terms of RPP, this study also demonstrated how researcher and teacher were two fluid identities and these identities could sometimes co-exist and some other times contradictory. Distinctive transpositioning happened depending on the decisions made in the different stages of the project by each member but an agreement was reached when our translanguaging spaces were congruent, meaning that the common goals were aligned. These include discussions in planning and implementing theoretical and pedagogical frameworks into the curriculum design. This study highlights the practical challenges and solutions of our research partnership and contributes to the partnership research in understanding the fluidity in teacher-researcher identities.
About the speakers
Roy Chan is an adjunct professor at Marymount University. He holds a Ph.D. in Applied English Linguistics from the Chinese University of Hong Kong. His recent journal articles appeared in English World-Wide, Journal of Multilingual and Multicultural Development, and Journal of Universal Language. His research focuses on language attitudes, pronunciation teaching, and teacher education.
Nick Wong is an experienced practitioner in English for Specific Purposes (ESP) and an applied linguist specializing in translanguaging and multilingualism. Currently, he is the principal investigator of an EdB eLAFP project, developing an AI-assisted virtual reality English-speaking program for secondary students (HK$30,365,700.00). He has received multiple teaching awards, including the SHSS Teaching Award and the CLE Teaching Award at HKUST. He is also a Senior Fellow of the Higher Education Academy (SFHEA) and a Fellow of the Lancaster University China Centre.
Presentation 2: Designing a VR System for Secondary School English Speaking Practice with LLM-Driven Conversational Agents and AI Feedback
Kento Shigyo
Hong Kong University of Science and Technology
In Hong Kong secondary schools, students often have few chances to practice English speaking in immersive or authentic group discussion settings, such as those required for the HKDSE English Speaking Exam (Part A). Classroom limitations—including time constraints, uneven participation, and the difficulty of recreating realistic discussion scenarios—make it challenging to provide effective speaking practice and individualized feedback.
This talk introduces the design and development of a virtual reality system created as part of the GAVIS project to support English speaking practice for secondary school students. Working with a multidisciplinary team, we integrated pedagogical insights with technical innovations such as large language model–driven conversational agents and AI-based speech feedback. The system enables students to engage in dynamic, interactive group discussion scenarios—such as those in the HKDSE English Speaking Exam (Part A)—within an immersive VR environment.
To ensure the system’s usability and pedagogical effectiveness, we conducted iterative user studies with students and teachers, gathering feedback that informed continuous refinement of system features, interaction design, and in-VR learning scenarios.
The talk will include a live demonstration showing how a student can practice participating in a group discussion using the VR system. We will also share key lessons learned throughout the project, offering practical insights for researchers, educators, and developers interested in applying VR and AI technologies to language learning.
About the speaker
Kento Shigyo is currently a Postdoctoral Fellow in the Department of Computer Science and Engineering at the Hong Kong University of Science and Technology (HKUST), working under the supervision of Prof. Huamin Qu. He is the team leader of VR Development Team in GAVIS QEF Project. His research interests lie at the intersection of Data Visualization, Human-Computer Interaction and immersive technology for education as well as a medical domain. One of his publications at ACM Multimedia received the Best Paper Honourable Mention Award.
Presentation 3: Bridging Pedagogy and Engineering: Developing a Unified, Resource-Efficient AI Framework for Multimodal Language Assessment
Sicheng Song
Hong Kong University of Science and Technology
Developing an AI assessment engine for group discussions (HKDSE Paper 4) presents a unique engineering paradox: while pedagogical validity requires sophisticated multimodal analysis, real-world deployment is strictly bound by data scarcity, sample imbalance, and infrastructure limitations. This presentation details the AI model architecture of the GAVIS project, demonstrating how we navigated these constraints to build a resilient, scalable system.
To resolve these, we engineered a solution across three core dimensions:
- Tiered Verbal Analysis Pipeline: Rather than relying solely on black-box models, we architected a layered scoring engine. We utilize SpeechAce as the foundational layer for acoustic benchmarking. These raw metrics are processed through Machine Learning (ML) regressors to perform precise score transformation and mapping to HKDSE sub-scores. Finally, we integrate a Large Language Model (LLM) layer for semantic fine-tuning, specifically to evaluate high-level context-sensitive criteria like “Ideas & Organization” which simpler models cannot capture.
- Dual-Stream Non-Verbal Alignment: To support cross-platform consistency, we developed two parallel detection pipelines. For the VR environment, we extract skeletal data directly from headset sensors and controllers ; for mobile/web users, we leverage TensorFlow.js (TFJS) for real-time, client-side pose detection via standard webcams. A critical engineering achievement is our data alignment protocol, which normalizes these heterogeneous inputs (VR telemetry vs. 2D camera coordinates) into a unified vector space, enabling consistent skeletal analysis across devices.
- AI Speaking Lab & Diarization: To extend assessment to offline, multi-party interactions, we designed the AI Speaking Lab, which captures group dynamics using distributed camera arrays. A unique technical challenge in this physical setting is acoustic overlap. We addressed this by integrating an automated Speaker Diarization pipeline, which accurately segments and attributes audio streams to individual speakers among co-located peers, ensuring precise individual assessment within the group context.
About the speaker
Sicheng Song is currently a Postdoctoral Fellow in the Department of Computer Science and Engineering at the Hong Kong University of Science and Technology (HKUST), working under the supervision of Prof. Huamin Qu. He is the team leader of AI Development Team in GAVIS QEF Project. He received his Ph.D. from East China Normal University, supervised by Prof. Changbo Wang and Prof. Chenhui Li. His research interests lie at the intersection of visualization and artificial intelligence, with a focus on AI-for-VIS, Human-Computer Interaction, and AI-driven educational technologies. He has published his works in top-tier venues such as IEEE TVCG, ACM CHI, UIST, EMNLP, and IEEE VR.
Panel Discussion:
Multiparty Co-Learning in Action: Past Challenges, Present Implementation, and Future Directions
Nick Wong1, Kento Shigyo1, Sicheng Song1, Roy Chan2, Jimmy Tse3, Jason Lo4
1 Hong Kong University of Science and Technology
2 Marymount University
3 Caritas Fanling Chan Chun Ha Secondary School
4 Trumptech Digital Education Services Ltd
Developing AI-enabled EdTech tools requires more than technical innovation or pedagogical expertise in isolation. This presentation examines how critical AI literacy emerges through a deliberate journey of unlearning, multiparty co-learning, and transdisciplinary collaboration. Rather than viewing AI literacy as individual competency, we demonstrate that it develops relationally—through the willingness of experienced teachers, technologists, and researchers to challenge disciplinary assumptions, question AI’s affordances and limitations, and co-construct shared understanding across boundaries.
This panel embodies the multiparty co-learning framework that enabled GAVIS development—positioning practitioners, technologists, and researchers not as separate experts presenting findings, but as co-learners engaging in real-time perspective-taking, role-shifting, and collective knowledge construction. By bringing diverse voices into dialogue around past challenges, present realities, and future possibilities, the panel demonstrates how transdisciplinary collaboration functions as both developmental process and sustainable practice.
Previous presentations had already revealed how role-shifting—acting FROM rather than merely understanding other perspectives—enabled breakthrough solutions during GAVIS development. Implementation has surfaced new questions requiring continued perspective-taking. The panel explores how transdisciplinary co-creation can address these challenges while supporting students with varying language backgrounds, learning differences, and accessibility requirements through personalized AI-powered solutions.
The discussion hopes to challenge participants to identify problems in their own contexts requiring transdisciplinary innovation and consider how role-shifting and perspective understanding can generate solutions that respect both technical possibilities and human educational values.
About the panel discussants
Jimmy Tse Chim Lui is an experienced and dedicated English teacher with over 15 years of expertise in teaching the language. Known for his proactive and curious nature, Jimmy is always ready to learn and implement new methods to enrich his teaching practices.
In addition to his teaching role, Jimmy serves as the Vice Prefect of the Language Across Curriculum Team, where he plays a vital part in promoting interdisciplinary language learning. He is also the Teacher-in-charge of the “Teaching English with I.T.” task group, demonstrating his passion for integrating technology into education to enhance student engagement and learning outcomes.
Jimmy’s forward-thinking approach and eagerness to embrace innovation make him a valued discussant.
Mr. Jason Lo is an accomplished Technical Director at Trumptech, bringing over 20 years of experience in the EdTech sector. He excels in designing and developing platforms that promote learning in mathematics, coding, problem solving, and languages. Jason’s expertise in scalable cloud architecture and the application of AI in education enables him to create impactful solutions that enhance student engagement.
As a leader in educational technology, Jason is dedicated to pushing the boundaries of learning experiences. His innovative mindset and deep understanding of pedagogical needs position him as a crucial advocate for integrating technology seamlessly into the classroom, inspiring both educators and learners alike.
Perhaps something that is related to VR and AI chatbot development? Or any other topics you find appropriate