浅析基于情感表达的三维动画角色肢体语言研究
肢体语言的情感编码理论与符号学叙事机制
该组文献奠定了研究的理论基础,探讨肢体动作如何作为非语言符号承载情感信息。涵盖了心理学识别机制、角色弧光理论、拟人化设计手法以及默片表演技巧,重点分析动画师如何通过肢体语言构建角色身份与叙事深度。
- Research on Interaction between Dancers’ Motion Capture and Robotic Arm(Po-Chih Lin, Feng-Cheng Lin, 2023, 2023 International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan))
- Animations and the Role of Emotions(Mihir Makwana, Makwana Ranveersingh, 2024, International Journal For Multidisciplinary Research)
- Rethinking Bodily Expression in Human-Robot Communication: Insights from Sculpture.(Belinda J. Dunstan, G. Hoffman, 2024, Interaction Design and Architecture(s))
- 情绪粒度对身体表情识别的影响:共情与人格的中介作用(屈妍妍, Unknown Journal)
- Emotional interfaces in performing arts: the Callas project(M. Bertoncini, I. Buonazia, 2008)
- Embodied Expression: Cognition, Technique, and Memory in the Art of the Silent Actor(Pali Vecsei, 2025, Theatrical Colloquia)
- 人物弧光理论视角下系列动画电影的角色设计研究——以《哪吒》为例(刘晓峰, 邵钰清, 沈宣吟, 应 群, 2025, 设计进展)
- 拟人化手法在动画角色设计中的运用研究——以《深海》为例(吴昆灵, 殷 俊, 2023, 设计进展)
- Approaches to the Representation of Human Movement: Notation, Animation and Motion Capture(T. Calvert, 2014, No journal)
- Agent Ali: Exploring Emotional Elements in Story Development with Artificial Intelligence(R. Zainal, Mohd Asyiek Mat Desa, 2024, PaperASIA)
- 虚拟偶像中表演者问题及运营者相关权利行使探究(李彦青, 2024, 争议解决)
- “Panawa” animation movement design: Rat character with human personality(E. Aulia, C. Aditya, 2017, 2017 4th International Conference on New Media Studies (CONMEDIA))
- 符号学视角下的三维游戏元素与叙事结构研究(匡星瑜, 马嘉阳, 2024, 设计进展)
基于运动特征提取与拉班分析的情感量化研究
该组文献侧重于肢体语言的科学量化与实证分析。重点应用拉班移动分析(LMA)理论和矢量空间建模,提取人体运动的特征参数,探讨运动模式与人格特质(OCEAN模型)、情感分类及音乐节奏之间的映射关系。
- EmoDescriptor: A hybrid feature for emotional classification in dance movements(Junxuan Bai, Rongtao Dai, J. Dai, Junjun Pan, 2021, Computer Animation and Virtual Worlds)
- Feature extraction for human motion indexing of acted dance performances(A. Aristidou, Y. Chrysanthou, 2014, 2014 International Conference on Computer Graphics Theory and Applications (GRAPP))
- From Motions to Emotions: Classification of Affect from Dance Movements using Deep Learning(Sukumar Karumuri, Radoslaw Niewiadomski, G. Volpe, A. Camurri, 2019, Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems)
- The EMOTE model for effort and shape(Diane M. Chi, Mónica Costa, Liwei Zhao, N. Badler, 2000, Proceedings of the 27th annual conference on Computer graphics and interactive techniques)
- Expressing Digital Character Personality through Motion capture: A study of body movements and personality traits(Anthie Kolokotroni, Marina Stergiou, Dimitris Baltas, Vilelmini Kalampratsidou, Panagiotis Kyriakoulakos, S. Vosinakis, 2025, Proceedings of the 3rd International Conference of the ACM Greek SIGCHI Chapter)
- Temporal relationship between dancer’s body movements and music beats in classical ballet(Takuya Wakiyama, Yurina Tsubaki, M. Kuno-Mizumura, Y. Sakaguchi, 2025, Scientific Reports)
- Vector Space Modeling of Classical Chinese Dance Movements Based on Motion Capture Technology(Miaogu Liu, 2024, Applied Mathematics and Nonlinear Sciences)
三维动作捕捉、建模技术与表现力增强优化
该组文献关注技术的底层实现与视觉效果的提升。涵盖了从Mocap(动作捕捉)到骨骼权重绑定、动作重定向,以及从二维草图到三维姿态的转换工作流,旨在通过技术优化解决‘恐怖谷效应’并增强动画的实时交互表现力。
- Application of three-dimensional image technology in the context of the metaverse in the production of emotional contrast and special effects in animation(Guo Yan, Hong Xin, Zhang Kuan, 2023, Multimedia Tools and Applications)
- Sketch-Based Virtual Human Modelling and Animation(Chen Mao, S. Qin, D. Wright, 2007, No journal)
- BLUI: a body language user interface for 3D gestural drawing(Arthur W. Brody, C. Hartman, 1999, No journal)
- Physical World to Virtual Reality – Motion Capture Technology in Dance Creation(Wu Zhen, L. Luan, 2021, Journal of Physics: Conference Series)
- Landing place: remapping motion capture of dance movement to objects and environments(Vita Berezina-Blackburn, Bebe Miller, B. Windsor, 2005, No journal)
- Exploring Visualizations in Real-time Motion Capture for Dance Education(Georgios Tsampounaris, K. E. Raheb, A. Katifori, Y. Ioannidis, 2016, Proceedings of the 20th Pan-Hellenic Conference on Informatics)
- Characterization and Emotional Expression of Digital Modeling in Film and Television Art(Zhenqian Xu, 2025, Journal of Combinatorial Mathematics and Combinatorial Computing)
- Exploring Choreography Through Body-Part Puppeteering in VR(Asako Soga, Misha Sra, 2025, Proceedings of the 2025 ACM Symposium on Spatial User Interaction)
- Sensing the Inside Out: An Embodied Perspective on Digital Animation Through Motion Capture and Wearables(Katerina El-Raheb, Lori Kougioumtzian, Vilelmini Kalampratsidou, A. Theodoropoulos, Panagiotis Kyriakoulakos, S. Vosinakis, 2025, Sensors (Basel, Switzerland))
- 基于恐怖谷效应的数字虚拟角色设计研究(神雨丹, Unknown Journal)
- Enhancement of 3D character animations through the use of automatically generated guidelines inspired by traditional art concepts(Jose A. S. Fonseca, Denis Kravtsov, Anargyros Sarafopoulos, J. Zhang, 2016, ACM SIGGRAPH 2016 Posters)
- 3D Character Posing from 2D Sketches for Storyboarding(Sophia Mouajjah, Cédric Plessiet, 2021, 2021 7th International HCI and UX Conference in Indonesia (CHIuXiD))
AI驱动与跨模态生成的情感动画前沿技术
该组文献代表了行业的最前沿趋势,聚焦于利用人工智能自动生成情感动作。包括多模态驱动(音乐驱动舞蹈)、生成式AI虚拟演员(Actotron)以及人机交互中的智能情感反馈系统。
- Intelligent Construction of Animation Scenes and Dynamic Optimization of Character Images by Computer Vision(Nan Zhang, Han Meng, Mingyu Ju, 2024, Computer-Aided Design and Applications)
- “虚拟数字人”的“今生”和“来世”(毛秀凤, 2024, 电子商务评论)
- Music-Aligned Holistic 3D Dance Generation via Hierarchical Motion Modeling(Xiaojie Li, Ronghui Li, Shukai Fang, Shuzhao Xie, Xiaoyang Guo, Jiaqing Zhou, Junkun Peng, Zhi Wang, 2025, ArXiv)
- 有情感的机器人:人造代言的情绪表达与交流(田倍嘉, 刘宏艳, 胡治国, Unknown Journal)
- The Actotron(Justin Matthews, Angelique Nairn, 2024, M/C Journal)
传统文化遗产保护与交互表演艺术的应用实践
该组文献探讨肢体语言在特定文化语境下的应用,如传统舞蹈(中国古典舞、泰舞等)的数字化建模与保护、闽文化等遗产的IP开发,以及在交互式剧场和虚拟现实空间中的美学呈现与叙事翻译。
- Translation process of ThaiDanceXML into a 3D animation representation(Yootthapong Tongpaeng, Tansita Somyawakath, P. Sureephong, 2018, 2018 International Conference on Digital Arts, Media and Technology (ICDAMT))
- UniQNatyam: An Approach Towards Non-Repetitive Dance Generation(Vibha Murthy, Vidisha Chandra, Vishakha Hegde, Rayudu Srishti, K. Srinivas, 2023, 2023 IEEE 4th International Conference on Pattern Recognition and Machine Learning (PRML))
- DANCING INTO THE DIGITAL AGE: PRESERVING INDIAN CLASSICAL DANCE NARRATIVES THROUGH VIRTUAL AVATARS(Suma Dawn, M. Bhattacharya, 2025, ShodhKosh: Journal of Visual and Performing Arts)
- WhoLoDancE: Towards a methodology for selecting Motion Capture Data across different Dance Learning Practice(Antonio Camurri, K. E. Raheb, Oshri Even-Zohar, Y. Ioannidis, Amalia Markatzi, Jean-Marc Matos, Edwin Morley-Fletcher, Pablo Palacio, Muriel Romero, Augusto Sarti, S. Pietro, Vladimir Viro, Sarah Whatley, 2016, Proceedings of the 3rd International Symposium on Movement and Computing)
- The Transformation of Dance Analysis: How Motion Capture and AI Are Changing the Art Form(Ayush Kumar Ojha, 2021, Journal of Humanities,Music and Dance)
- Towards Cultural Preservation of Traditional Motion Knowledge through Automated Annotations with MoRTELaban(Roberto Perez-Martinez, Alberto Casas-Ortiz, Olga C. Santos, 2025, Adjunct Proceedings of the 33rd ACM Conference on User Modeling, Adaptation and Personalization)
- Research on Motion Capture Technology of Peking Opera Performing Arts and Its Practicality Exploration in Opera Physical Training(Jue Hou, 2025, Applied Mathematics and Nonlinear Sciences)
- "I Fregi del Ceppo”: when artificial intelligence and geomatics meet theatre(P. Bartolini, A. Conti, Lidia Fiorini, G. Tucci, 2025, Virtual Archaeology Review)
- Catching the ghost: the digital gaze of motion capture(Vanessa Chang, 2019, Journal of Visual Culture)
- Reconstructing the Body and Space: A Comparative Study of Biped and Xihe Jianqi in Digital Interactive Dance(Xinhuan Zeng, 2025, Global Review of Humanities, Arts, and Society)
- Design and Production of a 3D Animated Short Film Using Blender Exploring Superstitious Beliefs(Erika D. Marca, Mszeey R. Montalbo, Jay Lynn C. Villanueva, John Rover R. Sinag, R. A. Maaño, Donabell S. Hernandez, Rodrigo C. Belleza, 2023, 2023 IEEE International Conference on Computing (ICOCO))
- Visual analysis of 3D characters and animations used in the presentation of cultural heritage at museums(Özlem Vargün, 2023, JOURNAL OF ARTS)
- Practical Research on Virtual Reality and Augmented Reality Technology in Min Cultural Heritage Digital IP Character Design Innovation(Honglin Li, Jiankai Weng, 2025, Applied Mathematics and Nonlinear Sciences)
- Reconstructing Traditional Mythological Character Images in Animation Character Design from a Semiotic Perspective : Focusing on the Central Characters in the Chinese Animated Film Series ‘Ne Zha’(Fang Han, Won-jun Chung, 2025, Korea Institute of Design Research Society)
最终分组结果构建了一个从理论到技术再到应用的完整研究框架:首先通过符号学与心理学确立肢体语言的情感编码逻辑;其次利用拉班分析法实现情感特征的数学化建模;随后深入探讨了动作捕捉、实时渲染等表现力增强技术;并整合了当前AI驱动与跨模态生成的前沿进展;最后将这些研究成果应用于文化遗产保护、交互表演及特定叙事题材中。整体呈现出跨学科融合、技术精度与文化深度并行的发展态势。
总计51篇相关文献
人造代言人正在融入人类的生活,本文从人造代言情绪表达的主要方式(面部和姿势表情)、人类对人造代言的情绪识别/评价及影响因素两个方面,对人造代言与人类的情感化交流的研究现状进行了探讨。未来研究可以从情绪类型多样化、情绪和非情绪线索整合、目标用户和情境适用性等方面进一步深化。
拟人化手法是指将非人类形象赋予人类的特征和形态,使其更具人性化的表现。在动画角色设计中,拟人化手法使角色造型更具有个性和可爱度,增加观众亲和力,也使角色动作神态更加传神,更能够准确表达角色的情感和心理状态。动画电影《深海》运用拟人化手法设计出了大量优秀的动画角色,本文通过分析动画《深海》的角色设计,论述拟人化手法在动画角色设计中的应用,从造型、动作、表情等方面描述如何用拟人化手法创造出优秀的动画角色。
为探讨情绪粒度对身体表情识别的影响及共情的中介作用,本研究通过两项递进式研究展开实证检验。研究一(N = 201)采用问卷调查法,分析情绪粒度与人格特质、共情的关系;研究二采用行为实验法,考察情绪粒度对静态与动态身体表情识别正确率和反应时的影响,并检验共情的中介效应。结果显示:1) 积极情绪粒度(PEG)与消极情绪粒度(NEG)呈中等正相关(r = 0.367, p p p p = 0.092),与恐惧情绪识别反应时显著交互(β = 0.160, p p < 0.001)。3) 共情关注在NEG与消极身体表情识别正确率之间存在部分中介作用(中介效应值 = 0.069,95%CI = [0.012, 0.156])。研究表明,情绪粒度是独立于人格的认知情感能力,其对身体表情识别的影响具有情绪类型与呈现方式特异性,共情仅在消极情绪加工路径中发挥中介作用,为情绪识别的双通路理论提供了实证支持。
三维游戏作为新兴媒介,受到众多年轻群体的追捧,其融合了视觉、听觉等多种互动元素,具有独特的内容表现和情感表达能力,在当今艺术设计领域发挥着重要作用。本研究基于符号学理论,借由对《双人成行》游戏设计中的角色造型、场景布置及互动方式进行逐一审视,从符号表现和叙事内涵的视角揭示游戏体验过程中用户对身份转化、意义模仿,以及脉络演进的内涵。本研究从多重维度探讨了符号学在三维游戏中的作用,剖析符号学对用户情感共鸣所产生的影响,旨在为未来的游戏创作提供新方向。
本研究基于恐怖谷理论观点,探讨数字虚拟角色形象设计中,拟真度高低与原型物种差异对虚拟角色认同感、亲和力和喜爱度的影响。采用量化研究方法,以2 (拟真度高/低) * 2 (是否人类)二因子线上实验法进行数据收集,共收集有效样本342份,使用SPSS进行数据分析。研究结果发现,在数字虚拟角色的认同感上,高拟真度组与低拟真度组没有显著区别,但人类形象组与非人类形象组两者的平均分数有显著差异。针对人类形象角色,高拟真度的数字虚拟角色,其认同感、喜爱度均显著高于低拟真度的数字虚拟角色。然而,针对非人类形象角色,低拟真度的数字虚拟角色,其亲和力、喜爱度均显著高于高拟真度的数字虚拟角色。文章同时测量了恐怖谷效应研究中经常混用的因变量,数据结果表明人们对于认同感、亲和力和喜爱度的理解和评分存在明显区别。研究表明,人们会受到拟真程度高低以及原型物种差异的影响而对数字虚拟角色产生不同的认同感、亲和力和喜爱度感知,此前的恐怖谷效应研究因变量存在混淆使用和模糊表意情况,此实证研究厘清了恐怖谷效应测量变项,同时为数字虚拟角色设计与相关数字产品开发提供理论支持和设计实践建议。
随着人工智能和计算机图形学的飞速发展,虚拟数字人技术逐渐成为研究与应用的热点。该技术通过创建逼真的数字化人物形象,实现与用户的自然交互,广泛应用于娱乐、教育、客服等领域。虚拟数字人不仅能模仿人类的动作和语音,还具备一定的智能和情感表达能力,为人们提供更加便捷和丰富的个性化服务体验。然而,该技术的快速发展也带来了隐私泄露、就业市场冲击等问题,引发了社会的广泛关注。因此,在享受虚拟数字人带来的便利的同时,我们也需要关注其潜在的风险和挑战,加强相关法规的建设和技术的监管,以确保该技术的健康、可持续发展,并最大限度地发挥其在各个领域的积极作用。本研究丰富了现有对于“虚拟数字人”的研究,并详细分析了“虚拟数字人”发展的“今生”以促进“虚拟数字人”发展的“来世”。
随着元宇宙和虚拟偶像在行业领域中的兴起,有观点提出给予虚拟偶像以著作权法中表演者的身份来给予其相应的保护。这表面上解决了其保护问题,但其实并没有对虚拟偶像的产生方式进行分类分析并且违背了著作权法的基本原理。本文通过对表演概念进行辨析、梳理表演者权成立须具有的构成要件,并以此为大前提来分析动作捕捉类虚拟偶像、程序类虚拟偶像以及声音同步类虚拟偶像中表演者是否存在以及如何存在,得出动作捕捉类虚拟偶像中“中之人”是表演者,但其权利应由运营者代为行使;在其余两类虚拟偶像中不存在所谓的表演者的结论。运营者对内要协调好与“中之人”之间的权利分配,对外要进行规避侵权与维权。对于运营者的权利保护应基于对作品的著作权以及录像制品的邻接权,但是在必要时可以对视听作品进行改造即删除“已固定”要件,从而对他人未经许可而进行现场直播的行为进行规制。
随着饺子导演的《哪吒》系列电影持续热映,观众对哪吒这一角色的设计成功愈发关注。从角色弧光的角度出发,分析前后两部动画的塑造方式,可以发现它们高度重视角色弧光的稳定性与连贯性。在此基础上,进一步探讨角色的视觉与行为符号,推导出符号应随弧光曲线的变化而调整的规律。即在弧光的不同阶段,角色的视觉与行为符号需与其内心成长相呼应,这种紧密联动正是角色深受观众喜爱的关键因素之一。反过来,该规律也可为系列动画的角色设计提供有益借鉴。
No abstract available
No abstract available
Digital characters are imaginary or realistic-looking entities created with computer programs. It can be used in cartoons and animations, as well as in many different areas such as games, advertisements, and the cinema sector. Avatars that represent real identities on digital platforms such as the metaverse today also consist of digital characters. Modeling digital characters that can replace real identities in the digital world is relatively easy. However, there are many paradigms in modeling a historical character in which the cultural heritage is reflected and therefore requires a difficult and laborious process. Information about the period in which a historical character lived is very limited, it is not possible to know exactly the culture, lifestyle, eating habits, speech, tone of voice, behaviour, and body movements of the period. Moreover, this character’s breaking away from history and interacting with today’s human is equivalent to interacting with an alien. Therefore, when it comes to cultural heritage, the characters either do not speak and have very limited movements, or if they interact, there are examples modelled from recent history. Today we live in a digital age, our tendency to communicate with visuals and watch events with moving images is dominant. In this case, the date and the information about the date are interesting if they are visualized. For this reason, it is necessary to speak the language of the digital age in order to reflect the cultural heritage and present history to the youth. Interaction and reflection of experience-oriented history with augmented reality, virtual reality or augmented reality has become the determinant of the post-digital age, not digital any more. They have become a necessity for history, culture, and archaeology. Virtual reality applications, storytelling with digital characters, have become a tool that encourages participation and motivation for today’s user. In this research, the scale in the table of Spyros Vosinakis “Dimensions, definitions and possible values related to the cultural heritage application use of digital characters” was developed and used. A historical character modelled in 3D according to this scale is “design: which era, clothing accessories and body covering preferences”, “reality: virtual, augmented mixed reality”, “credibility: movement, facial expressions, gaze, personality, emotion transfer” “environment: Criteria such as the credibility of the environment in which the character is presented, “interaction: interaction with the user, the use of artificial intelligence, decision-making, responding”, “function: showing, telling a story”, “reflecting history: what information is given” are examined. The characters of Mona Lisa, John Calvin, Uruk City, Hagia Sophia, Lady Ada Byron and Salvador Dali, which were taken into the scope of the research with the purposeful sampling method, are analysed by content analysis method. According to the findings, it was concluded that the details, clothing, accessory colours, and coatings in the models play an important role in increasing the believability, the ability of the characters to move, the voice acting, and the interaction increase the reality. When augmented reality and virtual reality experiences are compared, it can be said that the reality perception of the user who is teleported to the historical place increases more with the complete isolation of virtual reality from the external environment. Finally, it is thought that the inclusion of human characteristics such as movement, verbal skills, mimics, and personality in the design, as well as realistic character design, plays an important role in increasing the number and interaction of participants, due to their reality.
Filipino culture has a lengthy history of superstitions, but their significance has diminished, particularly among the younger generation. An animated short film was produced to inform, educate, and entertain people about Filipino superstitions. Examining the influence of superstitious beliefs on topics such as marriage, fortune, life, and mortality in Quezon Province, the study employed animated short films to tell stories, convey emotions, and provide real-world scenarios. The process included the animation's pre-production, production, and post-production phases, including composing and recording original soundtracks, importing background music and sound effects, and evaluating the short-animated film for audio and animation, synchronization, and transition. The 3D characters and backgrounds were modeled, rigged, animated, and lit using Blender software. The collected data from research and interviews impacted the film's concept, theme, music, and three-dimensional animation technique. Feedback indicated that the superstitions animated short film was compelling. The 3D film about superstition entertained the viewers with its contemporary language, humorous tone, and astounding 3D animation. The animation appeals to a broader audience due to its humorous and engaging visuals. The audience feedback and evaluation were followed by releasing the animated short film “Ay Weh? Tru ba?” on YouTube and Facebook.
Highlights: The reuse of 3D reality-based digital data in theatre productions supports the cross-valorisation of cultural activities. The frieze of the Ospedale del Ceppo: from digital documentation for preservation to scenic design and performance. The integration of digital technologies, avatars and artificial intelligence in contemporary theatre scenography. Abstract: The integration of three-dimensional (3D) digital technologies into cultural heritage and theatre is transforming how historical works are preserved and experienced. This paper focuses on the performance I Fregi del Ceppo, which exemplifies this trend by using 3D data to bring the Renaissance friezes of the Ospedale del Ceppo in Pistoia, Italy, to life. Originally digitised for conservation, the friezes served as the foundation for a theatrical production. The project used artificial intelligence (AI) tools to analyse and animate the frieze characters’ postures and relationships. The performance incorporated a 180º multi projection system that synchronised human actors with digital projections, merging live performance with digital heritage. This work highlights the broader trend of integrating AI and digital tools into theatre. Body scans, motion tracking, and emotion recognition enable new storytelling methods, while virtual characters and avatars allow performers to explore identity and interaction in novel ways. The fusion of AI with performance art is pushing the boundaries of creativity, generating dialogues, analysing performances, and enabling real-time interaction with human actors. I Fregi del Ceppo demonstrates how digital heritage can enhance theatre, extending the life of historical works and offering new cultural experiences. It also points to a future where AI and 3D technologies will play an increasingly central role in shaping the performing arts.
Emotional research in film and animation has long been implemented in the West. This is because Elements of emotion are essential in an animated film to generate an affective impression and emotions in the audience. Malaysia's animation industry has increased recently, and Malaysian animators have produced many animation products. However, previous studies stated that the current situation of local animated films needed a more potent storytelling technique. There is a West scholar who argues that good storytelling can evoke the emotions of the audience. Applying emotions in animation through computer technology is complicated compared to live-action films that can control emotions through the actors' acting techniques. Thus, this study identified the elements of emotion in developing the case study of storytelling Agent Ali in the movie. The mixed method was used to identify aspects of emotion in Agent Ali the Movie, such as Freytag's Pyramid model and Hume AI. As a result, this study found that aspects of emotion exist in the story development process in Agent Ali the Movie, such as happiness, sadness, and anger in every three acts of structure (exposition, conflict, and resolution). The existence of emotion has proved that animated films in Malaysia need to be focused to overcome the weak storytelling technique. At the end of the discussion, this study also found that Artificial intelligence (AI) technology like Hume AI could speed up the animation production process, especially in identifying the facial expressions and body language of characters in the process of storytelling development.
Introduction – The Advent of the Actotron Imagine a movie production where leading actors are not bound by human limitations, and digital entities render every emotion, movement, and line with breathtaking precision. This is no longer a conceptual idea but is becoming more possible with the increased integration of artificial intelligence (AI) into screen production activities. Essentially, we are at the dawn of the Actotron era. These advanced virtual actors, equipped with artificial intelligence, could transform not just how movies are made, but who makes them and what stories they tell. The Actotron promises to redefine the creative landscape, challenging our perceptions of artistry and authenticity in the digital age. The potential of the Actotron marks a milestone at the intersection of artificial intelligence, performance, and technology. This virtual human represents both a technological leap and a cultural shift that may revolutionise entertainment globally. Synthesising advancements in AI, motion capture, and voice synthesis, the Actotron enables autonomous performance, raising questions about creativity, copyright law, and the ethics of digital personalities. The capability for real-time learning and interaction pushes boundaries beyond CGI and deepfakes. Driven by AI algorithms and real-time graphics, the Actotron simulates nuanced human emotions, allowing dynamic interaction with human actors in media. Using future studies, we consider the potential emergence of the Actotron as the next step in digital actors and the place of artificial intelligence in the screen production industry. Method: Future Studies and Futurecasting To explore the potential and implications of the Actotron, this article employs methodologies from Future Studies and Futurecasting. These approaches are suited to assessing the Actotron due to their focus on creating plausible scenarios that envision future technological and societal shifts (Brown). Future Studies, as outlined by Miller, provides a structured way to consider potential outcomes and how current trends might evolve, utilising the "possibility-space" approach to explore future scenarios (Miller, "Futures"). This method allows us to escape the constraints of conventional forecasting, which relies heavily on past trends, limiting creative exploration of more impactful future scenarios. Exploring the Actotron's impact within a non-ergodic context—where historical precedents do not dictate future results—is useful. Miller explains that in unpredictable environments, traditional forecasting methods falter by not accommodating radical changes and emergent patterns (Miller, "From Trends"). This insight is vital for navigating uncertainties and recognising that the past may not be a reliable guide for future developments. Understanding this is critical for assessing how technologies like the Actotron could reshape media and entertainment, fostering a more adaptable approach to future possibilities. Futurecasting, as elaborated by Steve Brown, involves modelling future possibilities not to predict changes definitively but to prepare strategically for potential new realities. This approach aligns with the innovative essence of the Actotron—aimed at transforming performance landscapes and interactive experiences by anticipating shifts in technology and audience engagement dynamics. Miller highlights the critical role of anticipation in shaping decisions, emphasising its impact on developing technologies like the Actotron ("Futures"). By transitioning from trend-based forecasting to futures literacy, we can explore a wider array of possibilities beyond traditional prediction methods. By integrating Future Studies and Futurecasting and applying insights from the non-ergodic context and possibility-space approach, this analysis not only predicts but also prepares for a strategic future by providing a robust framework for understanding the societal impacts of technologies like the Actotron. CGI, Deepfakes, and Digital Actors The inception of Computer-Generated Imagery (CGI) revolutionised visual storytelling in cinema. Starting with simple wire-frame graphics in the 1970s, exemplified by Westworld (1973), CGI evolved into today's complex imagery. The 1980s and 1990s saw landmark films like Tron (1982), Terminator 2: Judgment Day (1991), and Jurassic Park (1993), demonstrating CGI's potential to create realistic environments and characters that enhanced narrative depth (Das). In the late 1990s and early 2000s, digital actors or "synthespians" emerged. Films like Final Fantasy: The Spirits Within (2001) and The Polar Express (2004) used full CGI and motion capture technologies to create human-like characters. Advances in motion capture, translating human actions into digital models, were critical in developing digital actors that convincingly emulate real human emotions and interact with live actors on screen (Gratch et al.). Building on earlier developments, this period saw significant advancements in digital doubles, which are highly realistic digital replicas of actors created using motion capture and digital modelling techniques. This progress was exemplified by The Matrix Reloaded (2003) and The Curious Case of Benjamin Button (2008). These films leveraged sophisticated motion capture to create detailed digital replicas of actors, refining digital doubles in mainstream cinema (Deguzman). Characters like Gollum from the Lord of the Rings trilogy showcased this technology's peak by combining motion capture with digital modelling to perform complex emotional roles alongside live actors (Patterson). Alongside these developments was the exploration of Autonomous Digital Actors (ADAs), integral to virtual actors and interactive media, extensively documented in research. ADAs represent significant advancements in digital media and interactive entertainment, offering novel methods for creating and animating 3D characters (Perlin and Seidman). These virtual actors can perform complex scenes autonomously, using procedural animation to respond to dynamic directions without pre-scripted motions, enriching interaction and storytelling (Iurgel, da Silva, and dos Santos). This technology allowed for cost-effective and versatile character animation, potentially transforming industries from gaming to educational software by enabling more nuanced and emotionally responsive character interactions. From 2017 onwards, deepfake technology captured public attention for convincingly—if controversially—manipulating video and audio, serving as both a precursor and foundational element for more sophisticated digital actors (Sample). Originally, deepfake technology focussed on manipulating video and audio recordings. Utilising machine learning and sophisticated algorithms, deepfakes could alter facial expressions, sync lips, or replace faces entirely (Pavis 976). This required understanding the video's three-dimensional space to apply realistic modifications, conducted during lengthy post-production workflows involving multiple VFX artists. In The Book of Boba Fett (2021), deepfake technology enabled the realistic portrayal of a youthful Mark Hamill as Luke Skywalker. The technique merged over 80 shots of deepfakes, CG heads, a body double, and Hamill's own performance to seamlessly depict his younger self (Bacon; Industrial Light & Magic). From pioneering CGI in the 1970s to sophisticated digital doubles in the early 2000s, the trajectory of visual storytelling has led to the advent of the Actotron. This technology has become a mainstay in visual effects and digital character generation, offering means to modify appearance, and age, or enable actors to fulfil different characters within a production (Xu 24). Synthesising these advancements through futurecasting, we consider the Actotron a virtual human tool that democratises filmmaking. To understand how this future operates, we turn to the fictitious but possible scenario of Alex, an imaginative director who harnesses the Actotron to bring cinematic visions to life. The Actotron Scenario Imagine a near future where film production has been revolutionised and democratised by the advent of Actotron technology—an advanced form of virtual human capable of comprehensive autonomous performance. We follow a day in the life of Alex, an aspiring young director with a passion for storytelling and a flair for technology. Alex's day begins in the quiet of her home studio, illuminated by the glow of dual screens. Today, Alex will create the lead character for an upcoming short film. Opening a sophisticated software portal, Alex interacts with a generative AI engine designed to craft an Actotron. Alex inputs desired traits and styles—courageous, empathetic, with a hint of mystery. The artificial intelligence proposes several faces; Alex selects one with captivating eyes and a resolute expression. Next, they sculpt the body—athletic and poised for action. Alex then tests different voice samples presented by the AI, blending them to forge a unique voice that mirrors their character's essence—a calming tone with a resilient undertone. With the character finalised, Alex uploads the script. The Actotron, "Kai", analyses it, intelligently querying to grasp the character's motivations fully. Content with Kai's comprehension, Alex moves to the virtual set. Alex commands, "Action!" and Kai begins the scene. Observing how Kai's expressions shift authentically with each line, Alex notes the performance. After a take, Alex suggests prompt changes—"Let's try it with more surprise on discovering the clue"—and Kai adapts seamlessly. This process repeats, with Alex refining Kai's performance until it aligns with her vision. As the day progresses, Alex introduces more Actotrons into different scenes. She directs interactions between Kai and other virtual actors, creating complex, dynamic exchanges that would be costly and challenging to shoot in a traditional setting. By dusk, Alex
No abstract available
Similar to language and music, dance performances provide an effective way to express human emotions. With the abundance of the motion capture data, content‐based motion retrieval and classification have been fiercely investigated. Although researchers attempt to interpret body language in terms of human emotions, the progress is limited by the scarce 3D motion database annotated with emotion labels. This article proposes a hybrid feature for emotional classification in dance performances. The hybrid feature is composed of an explicit feature and a deep feature. The explicit feature is calculated based on the Laban movement analysis, which considers the body, effort, shape, and space properties. The deep feature is obtained from latent representation through a 1D convolutional autoencoder. Eventually, we present an elaborate feature fusion network to attain the hybrid feature that is almost linearly separable. The abundant experiments demonstrate that our hybrid feature is superior to the separate features for the emotional classification in dance performances.
In this work, we explored methods for expressing dance narratives in a sequence on a virtual avatar, particularly in the context of Indian classical dance styles. This study has many different goals. First of all, it would aid in the preservation of multiple dance practices that are rapidly disintegrating since there aren't enough learners to impart lessons to them. Second, it may inspire a younger audience to be more engaged in discovering more about their cultural heritage by recasting it into a format that is easily accessible to them and can be valuable. Furthermore, this research will facilitate the organization of dance subtleties' morphology as well as their translation, retrieval, and storage processes. The aforementioned requirements, which are ontological and resource-intensive, would aid in digitizing vintage dance forms, making it easier to navigate information heading onward, including comprehending meaning, translating it into other languages, using rhythm, and other labour-intensive operations. This paper includes an examination of several approaches and strategies that are currently in use and could potentially implemented in future endeavours for the translation, conservation, and preservation of dance narratives.This research investigates the process of transforming dancer-choreographed stories into inanimate objects and virtual characters. It aims to explore animation techniques for converting Indian dance forms into animated narration, preserving cultural heritage, understanding emotions in animated sequences, and mapping body movements to capture dance structures' essence.
CALLAS project aims at designing and developing an integrated multimodal architecture able to include emotional aspects to support applications in the new media business scenario with an “ambient intelligence” paradigm. The project is structured in three main areas: the "Shelf", collecting multimodal affective components (speech, facial expression and gesture recognition); the "Framework", a software infrastructure enabling the cooperation of multiple components with an easy interface addressed to final users; and three "Showcases" addressing three main fields of new media domain: AR art, Entertainment and Digital Theatre, Interactive Installation in public spaces and Next Generation Interactive TV. INTRODUCTION – THE CALLAS PROJECT CALLAS Conveying Affectiveness in Leading-edge Living Adaptive Systems is an Integrated Project founded by the European Commission within the 6 Framework Programme Information Society Technologies priority, in the strategic objective Multimodal Interfaces (2.5.7). The project started in November 2006 and will end in May 2010. The project consortium is composed of universities and private research laboratories working on multimodal applications, together with artists, broadcasts and theatres, involved as final users [1]. MULTIMODAL AFFECTIVE INTERFACES: OBJECTIVES AND DOMAIN In everyday life, human communication combines speech with gestures, movements, and non-verbal expressions: each of those communication channels is affected by emotions. Taking in consideration the role of emotions and affectiveness is therefore fundamental to enrich naturalness also in human-machine interaction and communication. The CALLAS project face the challenges of implementing innovative affective interfaces to comprehend emotional input within the domain of interactive media. Affective and emotional interfaces are generally concerned with the real-time identification of user emotions to determine system response. They usually rely on Ekmanian emotions such as anger, fear, sadness, enjoyment, disgust and surprise. The domain of interactive new media, such as interactive narratives, digital theatre or digital arts, involves different ranges of emotions on the user’s side, some of which correspond to responses to aesthetic properties of the media, or characterize the user experience itself in terms of enjoyment and entertainment. To identify these ranges of EVA 2008 London Conference ~ 22-24 July Massimo Bertoncini Irene Buonazia _____________________________________________________________________ 20 emotions, more complex articulations of modalities are required across semantic dimensions as well as across temporal combinations. For instance, input from emotional language and paralinguistic speech (laughter, cries) must be categorized as indicators of user attention and must be integrated across interaction sessions of variable durations, instead of analyzing a single emotional status in real-time. The first scientific CALLAS objective is to advance the state-of-the-art in Multimodal Affective Interfaces, by creating new emotional models, able to take into account a comprehensive user experience in Digital Arts and Entertainment applications, and by developing new modalities to capture these new emotional categories. The main technological research (developed by the universities and research labs participating in the project) consists in the development and integration of advanced software components for semantic recognition of emotions. Those components will be available through a “living” repository, called the CALLAS “shelf”. On the other side, the project (mainly with the effort of the software and engineering companies of the consortium) aims at establishing a software methodology for the development and the engineering of a "framework" for Multimodal Interfaces that will make their development accessible to a larger community of users (even without a deep understanding of the theories of Multimodality), represented in the consortium by theatres, broadcasts and digital artists. Finally, the effectiveness of CALLAS approach will be validated developing research prototypes in the domain of digital media, arts and entertainment. In recent years, new media has developed largely in terms of richness of digital contents (combining text, video and sound) and of technical sophistication. On the other hand, emerging technologies, such as ubiquitous computing, augmented and virtual reality, human-computer interaction, and context and location-awareness, are making possible a paradigm embracing users’ natural behaviour as the centre of humancomputer interaction. Most New Media are actually interactive, and rely on digital content for which user interaction plays a central role. The domain of digital cultural content (digital theatre, mixed reality arts, ubiquitous systems supporting interactive storytelling and TV) is specially challenging, involving a wide range and combination of sophisticated users’ emotions and feelings. This particular domain chosen by CALLAS imposes to advance the understanding of emotional interaction, taking into account also non-Ekmanian emotional categories. CALLAS COMPONENTS: THE SHELF The Shelf consists of a dynamic pool of advanced multimodal interface technologies, selected taking into special account efficiency and robustness, in order to guarantee a consistent performance for many contexts and scenarios, specially for the use in uncontrolled “production” scenarios, while many technologies are developed and tested in controlled settings. CALLAS shelf components include: Emotional Speech Recognition: combines keyword spotting in utterances with information about the emotional state of the speaker, according to correspondences with a list of Ekmanian and non-Ekmanian emotions (component specially developed by Faculté Polytechnique de Mons); Emotional Natural Language Understanding (developed by University of Augsburg): includes acoustic as well as linguistic features, relying on a corpus-driven approach; EVA 2008 London Conference ~ 22-24 July Massimo Bertoncini Irene Buonazia _____________________________________________________________________ 21 Sound Capture and Analysis (component developed by VTT): maps on emotional patterns speech, surrounding sounds (music, crowd cheering), to guarantee natural and adaptive interaction with physical and virtual environment, as well as in the creation of MR/AR environments; Video Feature Extraction: extracts contextual and emotional information about users, environment and media from video streams combining audio and visual information. The component especially analyses video streaming on wide spaces, tracking speed, direction and quantity of movement of items in the space; Gesture and Body Motion Tracking (developed by VTT): provides information about body movement and gestures interpreted as thresholds to different emotional states. The tracking, especially focussing on hands movements, will be performed with different sensors positioned from upper limb to the whole body; Haptic Tracking component is a 3D haptic tracker for virtual environment navigation, based on interpretation of force/tactile feedback (developed by Humanware); it will be further developed into Wearable Interfaces for Motion Capture, embedding different miniaturized transducers for gesture recognition and motion tracking; Multimodal Interpretation of User Experience (developed by University of Teesside) researches on the emotional categorisation of the user experience, aiming at defining a new paradigm for investigating emotions in multimodal interfaces; Affective Multimodal Interpreter / Facial Expression Recognition extracts expressivity from gaze detection, facial features (measured through coordinates of interest point in the face), and gesture recognition (measured through head-hands coordinates). Such components, operating on high-resolution images of frontal faces and signals coming from many sensors, are developed by ICCS. The output of such researches should be an Expressivity Synthesis, able to generate, from image sequences, sensors, history and personality details, an expressive model of user’s behaviour, to be performed by ECA. Emotional Natural Language Generation component (to be developed as a research output of the project by University of Augsburg) is responsible for generating natural language without disregarding the affective aspects of a conversation. It is based on an annotated corpus consisting of sentences that present typical expressions used in a conversation. The corpus has to be annotated with categories and topics that the sentences are about and also with the emotional state that they denote. Affective Music Synthesis (developed by University of Reading) component aims at enhancing musicality and music expression of virtual actors according to the user’s mood, making users' experience of sound and music less mechanical. Emotional Attentive ECA (Embodied conversational agents, based on achievements of ECA developed by University of Paris8), will investigate on three core capabilities of ECA in emotional and social context: emotional communication through gesture, facial expression, gaze, body; emotional expression in a social context by blending or masking emotions; modelling perceptual-attentive social behaviours that are a basis for interaction, such as mutual, joint and shared attention. THE CALLAS INTEGRATED APPROACH: THE FRAMEWORK The aim of the CALLAS project is to develop a system able to combine emotional components and features in new modalities, enabling different modes of integration, as required by various applications, offering pre-assembled, re-usable, and semantic fusion EVA 2008 London Conference ~ 22-24 July Massimo Bertoncini Irene Buonazia _____________________________________________________________________ 22 components. The CALLAS Framework is being designed as a software infrastructure that will allow a number of She
Following the footsteps of the times, an excellent and complete movie cannot be separated from the application of digital modeling. In this paper, we mainly use 3D modeling, motion capture, rendering and other related technologies to edit and produce the character's physique, proportion, contour, etc., design the character's expression, color and action, and build the film and television scenes in 3D space. Thus, it realizes the characterization and emotional expression in film and television. Will be through the traditional 2D film and television and three-dimensional film and television control experiments, from the experimental data can be seen, in the frame rate, three-dimensional modeling technology film and television than the traditional 2D film and television on average 14% to 20% higher. There is also a leading edge in the number of textures. The data color emotion analysis indicated that the color shift and strong contrast connects the plot and the audience's feelings. The quantitative survey of emotional experience through questionnaires shows that the audience in the 3D film and television group is higher than the traditional 2D film and television in terms of immersion experience, interaction experience and learning and enjoyment experience. Therefore, 3D modeling technology plays an important role in the creation of film and television art.
Storyboard is a critical stage in the 3D animated movie production field. It defines the breakdown, the shots, the main actions, and offers the first visual appearance of the upcoming movie. During this step, artists must enter into an ideation process in order to prototype the acting, expressions and poses of their characters and use them to develop the story line. By working exclusively on a 2D medium, storyboard artists have a partial view of the future 3D scene. The purpose of this study is to ease the transition from the storyboard to the 3D layout scene while using digital tools. In this paper, we focus mainly on the issue of character posing. Our goal is to set up a new workflow for story boarders, that will let the user create character poses in an expressive way by preserving the artist’s natural drawing gesture. This workflow will bring the user into an augmented co-creative process with digital tools.
As the global animation industry continues to expand, the dual forces of technological advancement and market demand have posed new challenges for traditional mythological animation in balancing cultural continuity with contemporary relevance. Focusing on character design as a key entry point, this study aims to explore strategies for creative transformation. Drawing on semiotic theory, it constructs a three-tier analytical model—comprising the Real System, Denotative System, and Connotative System—and analyzes the symbolic evolution of Nezha and Ao Bing in China's 3D animated Nezha film series. The study finds that modern technology has overcome the limitations of traditional visual carriers; under the premise of preserving core cultural anchors, symbolic elements are reconstructed to align with contemporary narrative contexts; and the resulting value transformation fosters the integration of traditional culture with modern ideological expression. This research further proposes a threefold dynamic mechanism—“technological empowerment, symbolic reconstruction, and value reconfiguration”—to reveal the logic of cultural symbol reproduction. The mechanism offers a methodological framework for the modernization of myth-based animation and contributes theoretical insight into the creative regeneration of cultural symbols.
Abstract With the development of virtual reality (VR) technology and augmented reality (AR), the communication media and expression of culture have been further expanded. In this paper, VR and AR technologies are applied to the digital IP character design of Min cultural heritage, focusing on the image generation and action recognition processes in the digital IP character design process. The LAFITE model is used to generate the digital IP image of Min cultural heritage, and after the pose representation of the digital IP character, the multi-class support vector mechanism is used to construct the action recognition model. The model tests proved that the FID and IS produced by the LAFITE model are superior to those produced by other traditional models by 30% to 316% and 6% to 48% respectively. The output images of the Min cultural heritage digital IP characters are also of better quality. The MSVM model exhibits a high recognition rate for various actions of the IP characters, with each index value exceeding 93%, thereby facilitating effective interaction and enrichment of digital IP characters. The image output and action recognition model proposed in the study can promote the innovative design of digital IP characters of Min culture and enhance the digital creative expression and interactive forms of Min culture.
. The aim of this article is to investigate an intelligent approach for constructing animation scenes and dynamically optimizing character images driven by computer vision, with a particular emphasis on its potential applications in Chinese animated film production. To accomplish this objective, the study integrates multi-modal emotion recognition techniques, enhancing the emotional impact of animations by simultaneously processing facial expressions and voice emotions of characters. Utilizing state-of-the-art convolutional neural networks (CNN) and long short-term memory (LSTM) models, the research intelligently analyzes and refines animation scenes and character depictions. The findings reveal that this strategy notably elevates the emotional ambiance of the animated scenes and enhances the dynamic expressiveness of the characters, thereby contributing to technological advancements in Chinese animated filmmaking. This study not only shows the potential of computer vision technology in animation production but also provides a new direction for the modernization process of animated films in China. Through the integration of technological innovation and artistic expression, it is expected that Chinese animated films will present a more wonderful visual feast in the future.
Sculpture offers a centuries-long tradition of techniques for expressing emotion and movement in a static form. Insights from this field present an opportunity to design robots that express not only through movement, but also via dynamic cues in their static positions. Such cues can suggest motion potential, emotion, and character. This paper presents three principles identified in sculpture techniques that can be applied to robot design: (a) depicting exposure and protection of emotional pivot points in the body, (b) weight distribution, and (c) the revelation of movement mechanisms and tension through flexible skins. We employ the first two of these principles in an interactive design and motion control environment to demonstrate the potential for application to the design of social collaborative robots. We illustrate the third principle via a robot design that uses a flexible fabric skin stretched over rigid and elastic actuation elements. Using insights from sculpture can promote the design of robots from a transdisciplinary perspective by increasing the readability of robot intent and affect even when the robot is not actively moving.
No abstract available
How can an animator capture and analyse emotion? Might the act of animating something itself be a way to access meaning that earlier studies were unable to do? Animation has the ability to both accentuate and hide emotions that are displayed through body language and gesture. We are exposed to both the spoken words and the subtle variations in body motions when we watch live action (human interview) documentary film. What may be gained by interpreting documentary video via the animator's personal and aesthetic perspective, or how much might be lost when it is converted into animation? This study investigates the outcomes of the initial of a series of animations produced utilising research via practise approach, drawing on my prior expertise as a game's animator. The process of manipulating images to provide them the appearance of motion is referred to as animation. By creating a series of images, or frameworks, with one frame being a bit distinct relative to the last, a representation of movement is produced. Cartoons are among the best forms of animation. Animated videos have always been hand-drawn, using a great deal of visuals made with minor adjustments using color as well as a sharpie. A lot of contemporary animation was produced using an arsenal of specialized software programs as computer technology and animation software proliferated.
Silent acting occupies an unusual position in performance studies: it is at once historically bounded by early cinema and yet persistently relevant as a model for understanding embodied expression. This article examines the silent actor as a site where cognition, nonverbal communication, and artistic technique converge. Drawing from film history, cognitive science, and performance theory, the study argues that silent performance functions as a laboratory for analyzing how bodies generate meaning and emotion in the absence of speech. After outlining the historical evolution of silent acting and the diverse traditions that shaped it, the article incorporates research in embodied cognition and spectator theory to explore how viewers infer intention, affect, and narrative from physical behavior. Technical aspects of silent performance, like gesture, posture, rhythm, facial micro- expression, and the actor’s relationship to the camera, are examined in detail to reveal how silent actors crafted expressive precision. Contemporary applications are analyzed through examples from modern cinema, animation, and motion-capture performance, demonstrating the enduring relevance of silent-era principles. Case studies of Charlie Chaplin, Buster Keaton, and Asta Nielsen illustrate how individual actors developed distinctive expressive vocabularies grounded in bodily form. The article concludes by arguing that silent acting not only enriches film and performance scholarship but also contributes to ongoing scientific discussions about perception, empathy, and the embodied mind.
No abstract available
Good animation movie needs to be communicative and deliver its meaning to the audience. One of the most important aspects is to design an effective movement for the animation. In movement, the thoughts and characteristics of a character are shown. Expression of a characters can be seen through their movement and body gestures. Body gestures show us an expression, emotions, traits, thoughts, and each different characters. Characters do t move effectively if the movement does t correspond to its characteristics. To understand the characteristics we need to understand each characters physiology, sociology, and psychology. In this research, the author explains and analyze the process of designing a rat movement with a human personality.
No abstract available
Dance is an art form involving body movements, expressions, and gestures to communicate emotions non-verbally. Choreographing a dance that aligns with the music is a complex process requiring time, money, and effort. This research paper aims to simplify choreography by exploring the relationship between music and dance motion. We propose using audio analysis, a cross-modal transformer, a reward model, and a 3D animated figure to generate unique and visually pleasing dances that accurately represent the music’s characteristics. The goal is to create an immersive experience without repetitive movements while conveying the intended emotions and story.
For centuries, the analysis of dance has relied on subjective observation and qualitative descriptions. The emergence of motion capture technology and artificial intelligence (AI) is revolutionizing this landscape, offering objective data and novel tools for analyzing and interpreting movement. This paper explores the transformative impact of these technologies on various aspects of dance analysis, including: Technique Assessment: Motion capture provides precise metrics for evaluating movement accuracy, alignment, and efficiency, unlocking new possibilities for personalized training and injury prevention. Choreographic Deconstruction: AI algorithms can identify patterns, motifs, and emotional arcs within complex dance pieces, offering deeper insights into choreographic intent and audience perception. Expressive Movement Analysis: By analyzing subtle nuances in body language and facial expressions, AI can shed light on the emotional and communicative dimensions of dance, enriching interpretation and fostering empathy with the performers. Historical Archival and Preservation: Motion capture can digitally preserve endangered dance forms and cultural traditions, facilitating cross-cultural exchange and ensuring accessibility for future generations. This paper examines the benefits and challenges of employing these technologies in dance analysis, addressing concerns about objectivity, authenticity, and the potential loss of the human element. It concludes by exploring potential future directions, arguing that motion capture and AI, when used thoughtfully and ethically, can enhance our understanding and appreciation of dance while fostering innovation and creative expression in this timeless art form.
There has been an increasing use of pre-recorded motion capture data for animating virtual characters and synthesising different actions; it is although a necessity to establish a resultful method for indexing, classifying and retrieving motion. In this paper, we propose a method that can automatically extract motion qualities from dance performances, in terms of Laban Movement Analysis (LMA), for motion analysis and indexing purposes. The main objectives of this study is to analyse the motion information of different dance performances, using the LMA components, and extract those features that are indicative of certain emotions or actions. LMA encodes motions using four components, Body, Effort, Shape and Space, which represent a wide array of structural, geometric, and dynamic features of human motion. A deeper analysis of how these features change on different movements is presented, investigating the correlations between the performers' acting emotional state and its characteristics, thus indicating the importance and the effect of each feature for the classification of the motion. Understanding the quality of the movement helps to apprehend the intentions of the performer, providing a representative search space for indexing motions.
This work investigates classification of emotions from MoCap full-body data by using Convolutional Neural Networks (CNN). Rather than addressing regular day to day activities, we focus on a more complex type of full-body movement - dance. For this purpose, a new dataset was created which contains short excerpts of the performances of professional dancers who interpreted four emotional states: anger, happiness, sadness, and insecurity. Fourteen minutes of motion capture data are used to explore different CNN architectures and data representations. The results of the four-class classification task are up to 0.79 (F1 score) on test data of other performances by the same dancers. Hence, through deep learning, this paper proposes a novel and effective method of emotion classification, which can be exploited in affective interfaces.
Experimentation in film, animation, music, and art has been marked by intermedial and transmedial play between various art forms, such as rendering acting and performing cues in digital characters. This work aims to shape future guidelines for digital character creation and define kinesiological patterns aligned with narrative development. It investigates kinesiological patterns linked to personality and whether actors’ personality traits, expressed through non-verbal gestures, postures and movements, are preserved and conveyed through digital characters. Experts and independent viewers assessed correlations between movement sections, qualities, and personality traits. The study setup is based on A Chairy Tale by Norman McLaren, which offers a rich movement vocabulary. Using character design theory like the ‘12 Principles of Animation’ and Laban’s 8 Efforts, movements were tagged and mapped to personality traits using the OCEAN model. Simulated via motion capture, scenes were evaluated by participants. Results showed a strong alignment between participants’ ratings and expert analysis, supporting personality-movement correlations.
Over the last few decades, digital technology has played an important role in innovating the pipeline, techniques, and approaches for creating animation. Sensors for motion capture not only enabled the incorporation of physical human movement in all its precision and expressivity but also created a field of collaboration between the digital and performing arts. Moreover, it has challenged the boundaries of cinematography, animation, and live action. In addition, wearable technology can capture biosignals such as heart rate and galvanic skin response that act as indicators of the emotional state of the performer. Such metrics can be used as metaphors to visualise (or sonify) the internal reactions and bodily sensations of the designed animated character. In this work, we propose a framework for incorporating the role of the performer in digital character animation as a real-time designer of the character’s affect, expression, and personality. Within this embodied perspective, sensors that capture the performer’s movement and biosignals are viewed as the means to build the nonverbal personality traits, cues, and signals of the animated character and their narrative. To do so, following a review of the state of the art and relevant literature, we provide a detailed description of what constitute nonverbal personality traits and expression in animation, social psychology, and the performing arts, and we propose a workflow of methodological and technological toolstowardsan embodied perspective for digital animation.
Abstract The development of motion capture technology provides the possibility of in-depth analysis of Chinese classical dance movements. In order to facilitate the measurement and calculation of classical dance movement data, this paper is divided into different human detection nodes. The TrignoIM wireless EMG wireless collector is selected to collect human electromyography data, and the data errors of the gyroscope, accelerometer, and magnetometer are analyzed and processed to construct the Chinese classical dance movement data set. Subsequently, Maya animation software was used to spatially model the character body and finger bone vectors of classical Chinese dance movements and combined with neural fusion shape techniques such as envelope deformation branching and residual deformation branching for bone building and skin weight binding. In order to test the effectiveness of the motion capture and modeling techniques in this paper, they are applied to the digital teaching of classical Chinese dance. The comparison group and the conventional group were selected to adopt different training methods, and a final assessment was conducted after completing the classical dance course. When it came to movement amplitude, the conventional teaching scored 4.06 points higher than the comparison group, and the comparison group performed better than the conventional group in all other areas. Classical Chinese dance movement vector space modeling based on motion capture technology is able to standardize classical dance movements.
HCI (Human-Computer Interaction) is a trend in technological art, it can build an immersive experience in an interactive field. This paper focuses on intelligently and dynamically capturing dancers' motion, quantifying the emotional representation of dancers. We define the gesture's emotional type (positive and negative), and the gesture's definitions given are expansion and contraction of the limb. Finally, the interaction between the 3D coordinates and the movement of the robotic arm can be captured by the pose of the dancer. Based on our best results, the image resolution is 960*540, and we achieve a real-time effect of up to 30fps.
Abstract In this paper, we obtain the form data of famous Peking Opera performers through inertial motion capture technology to construct a standard form library. Sensors are placed in key parts of the body of Peking Opera performers to capture position information, and specific 3D data is obtained after a series of calculations. The long and short-term memory network is also used to solve the problem of form timing, and the captured form data of teachers and students are presented intuitively and visually, so as to study the application of motion capture technology in theatrical form training. Comparative analysis shows that compared with the other two algorithms, the accuracy of the method in this paper has a mean value of 93%. When completing the form exercise “Qiba”, the rate of change of the right knee angle was 0.56 for the student and 0.22 for the teacher. As a result, the teacher’s voice is more consistent. The teacher’s rotational acceleration of the spine is higher than that of the students in the “Shake by Flag” form, showing his/her greater strength. In the movement analysis of “Harrier Turn”, the teacher’s spine vertical angle is bigger.
After more than 100 years of development, motion capture technology has been advanced greatly. In the initial stage, the motion capture technology was only capable of capturing a rough range of body movements. But now it is sensitive enough to capture the movement of the whole body. Motion capture technology has also been developed into different types: optical and non-optical systems including inertial, magnetic, and mechanical. There are also hybrid and depth sensing camera-based systems. Nowadays, with the combination of the technology and arts, computer software and internet technology have become a new territory for dance performance. Motion capture plays an increasingly important role in dance creation. Meanwhile, the combination of motion capture and virtual reality technology enables the audience to freely choose different viewing angles and moving positions in a totally virtual space, which can present stronger sense of tension and cultivate more immersive experience than live dance performances do. On the Internet, virtual reality dance theaters connect dance performers and audiences from different geographical spaces. The virtual reality dance heaters build up virtual performance venues, paralleling with the real theater performance in the physical world.
Created with digital motion capture, or mocap, the virtual dances Ghostcatching and as.phyx.ia render movement abstracted from choreographic bodies. These depictions of gestural doubles or ‘ghosts’ trigger a sense of the uncanny rooted in mocap’s digital processes. Examining these material processes, this article argues that this digital optical uncanny precipitates from the intersubjective relationship of performer, technology, and spectator. Mocap interpolates living bodies into a technologized visual field that parses these bodies as dynamic data sets, a process by which performing bodies and digital capture technologies coalesce into the film’s virtual body. This virtual body signals a computational agency at its heart, one that choreographs the intersubjective embodiments of real and virtual dancers, and spectators. Destabilizing the human body as a locus of perception, movement, and sensation, mocap triggers uncanny uncertainty in human volition. In this way, Ghostcatching and as.phyx.ia reflect the infiltration of computer vision technologies, such as facial recognition, into numerous aspects of contemporary life. Through these works, the author hopes to show how the digital gaze of these algorithms, imperceptible to the human eye, threatens individual autonomy with automation.
Well-coordinated, music-aligned holistic dance enhances emotional expressiveness and audience engagement. However, generating such dances remains challenging due to the scarcity of holistic 3D dance datasets, the difficulty of achieving cross-modal alignment between music and dance, and the complexity of modeling interdependent motion across the body, hands, and face. To address these challenges, we introduce SoulDance, a high-precision music-dance paired dataset captured via professional motion capture systems, featuring meticulously annotated holistic dance movements. Building on this dataset, we propose SoulNet, a framework designed to generate music-aligned, kinematically coordinated holistic dance sequences. SoulNet consists of three principal components: (1) Hierarchical Residual Vector Quantization, which models complex, fine-grained motion dependencies across the body, hands, and face; (2) Music-Aligned Generative Model, which composes these hierarchical motion units into expressive and coordinated holistic dance; (3) Music-Motion Retrieval Module, a pre-trained cross-modal model that functions as a music-dance alignment prior, ensuring temporal synchronization and semantic coherence between generated dance and input music throughout the generation process. Extensive experiments demonstrate that SoulNet significantly surpasses existing approaches in generating high-quality, music-coordinated, and well-aligned holistic 3D dance sequences.
Movement disciplines like dance or martial arts are carriers of cultural knowledge, identity, and tradition. However, oral traditions and video recordings make the preservation of this knowledge susceptible to being lost. Expert movement notation, in turn, holds the potential for precise capture and knowledge inheritance. However, motion notation approaches are not widespread, the process is often time-consuming, and the movements are hard to visualize without expert knowledge. In this work, we use Labanotation and Laban Movement Analysis (LMA), a notation system and method originally developed for dance, as a symbolic, interpretable framework for motion representation and preservation. Our contribution resides in the expansion of an existing annotation system, the LabanEditor, to handle full-body motion and data from multiple sources, and support the work of experts in annotating the movements. Our development, called MoRTELaban, supports motion-to-notation and inverse mapping from notation to keyframes, enabling exchange between video, motion capture, and Labanotation formats. This allows for the documentation and reconstruction of traditional motion practices using expert-readable scores and 3D skeletons.
No abstract available
No abstract available
No abstract available
No abstract available
Musicality is one of the most essential aspects of dance performance: Dancers control their bodies to the music, and audiences appreciate dances beautifully harmonized with music. The present study explores the physical reality of the dance musicality from the perspective of synchronization between music beats and body movements. Specifically, we investigated the temporal relationship between dancer’s body movements and metronome beats when a dancer performed four basic classical ballet movements (i.e., changement, passé, jeté, and tendu) to the metronome. We measured body movements of 10 ballet dancers using an optical motion capture system and force plates, and analyzed what movement reference points of dancer’s body motion (e.g., movement endpoints and ground reaction force peaks) occurred on or close to the beat and offbeat. Specific reference points coincided with the beat timing common to most dancers, but the different reference points were synchronized with the beat depending on the movements. These reference points were consistent with those reported in previous studies of the temporal relationship between music and body movements. Therefore, the present result suggests that humans have a set of common movement features that can serve as reference points for music-motion synchronization, and dancers select appropriate ones according to the target movements.
This study explores how digital technology reshapes the aesthetics of dance performance by analyzing two representative works: Biped (1999) by Merce Cunningham and Xihe Jianqi (2020) produced by the China Dancers Association. Through a comparative framework, the research investigates the interaction between body, technology, space, narrative, and audience experience within digital interactive dance. While Biped utilizes motion capture and computer graphics to abstract and depersonalize the body in a non-narrative structure, Xihe Jianqi integrates LiDAR and depth-sensing technologies to create a poetic fusion of traditional Chinese sword dance and digital responsiveness. The study reveals that digital dance is not a singular aesthetic trend but manifests diverse cultural and philosophical approaches. Key findings highlight the dual role of technology as both an abstracting tool and an emotional medium, the reconfiguration of the stage as a responsive ecosystem, and the transformation of the audience from passive spectators to interactive participants. This research contributes to the discourse on digital performance by proposing a refined analytical model that integrates body, technology, and space, and offers a culturally grounded perspective on the evolution of digital dance aesthetics.
We present a VR system for generating novel body motions by controlling partial body parts with standard VR handheld controllers. The system enables dancers and non-dancers to create full-body movement by manipulating tracked hands and head motion, bypassing the need for full motion-capture setups. We developed three puppet modes and conducted a preliminary user study with dancers and non-dancers. Our early findings suggest that even with minimal input, users can experience a sense of agency and create expressive motion, offering new possibilities for choreography and creative expression, especially for users with limited mobility.
最终分组结果构建了一个从理论到技术再到应用的完整研究框架:首先通过符号学与心理学确立肢体语言的情感编码逻辑;其次利用拉班分析法实现情感特征的数学化建模;随后深入探讨了动作捕捉、实时渲染等表现力增强技术;并整合了当前AI驱动与跨模态生成的前沿进展;最后将这些研究成果应用于文化遗产保护、交互表演及特定叙事题材中。整体呈现出跨学科融合、技术精度与文化深度并行的发展态势。