Behind the curtain: composing acousmatic sounds and 3D CGI

This commentary details the methods and ideas involved in creating a portfolio of audiovisual works that form a practice-based PhD completed at De Montfort University. The overall aim of this research project is to explore possible ways of combining acousmatic sounds with three-dimensional computer-generated imagery (3D CGI) so as to achieve balance between the two materials. Balance between sounds and images is a recurrent issue in audiovisual composition. This portfolio of works tackles this issue taking into consideration the particularities of acousmatic sounds and 3D CGI. In particular, this project addresses the following interlinked research questions: ‣ how can visual dominance (i.e. the natural tendency for vision to dominate over hearing) be tackled in the context of audiovisual composition of 3D CGI and acousmatic sounds? ‣ how can a unified method for the composition of 3D CGI and acousmatic sounds be developed so as to avoid the mere layering of the two elements and at the same time obtain audiovisual balance?

This has been tackled through the following objectives: ‣ outline the particular mimetic properties of 3D CGI rendered with physically based techniques (i.e. techniques that imitate how light behaves in the real world); ‣ define the potential issues of visual dominance and outline possible solutions in the context of 3D audiovisual composition; ‣ contextualise this research’s portfolio within the tradition of audiovisual composition and propose a ‘diegetic’ compositional method where sounds and images derive from a common (3D) world and their causal relationships are manipulated to enhance their perceptual balance. The methodology for this research consisted of experimenting with compositional workflows, learning specialised software tools and reviewing relevant literature. The research activity has been practical and driven by compositional experimentation. This portfolio aims at offering a contribution to the field of audiovisual composition by presenting a possible framework for merging the acousmatic music tradition and 3D computer-generated imagery. This framework takes into consideration the inherent mimetic and spatial characteristics of 3D CGI and uses the mentioned ‘diegetic’ approach as a way of merging and balancing sounds and images.