Mr Christian Steinmetz

Profile

Project Title: End-to-end generative modelling of multitrack mixing with non-parallel data and adversarial networks

Abstract: Modelling the task of transforming a set of recordings into a professional mix remains a challenging task. While many different approaches have been investigated, they fail to generalize to the diversity and scale of real-world projects. Recently, deep models applied to the task of virtual analog effect modeling have demonstrated that this class of models can capture the behaviour of these processors in an end-to-end fashion. These findings motivate the design of an end-to-end model for the multitrack mixing process that operates directly at the waveform level to synthesize a mixture. This direction is of interest due to the inability to propagate gradients through the mixing console directly, and the absence of parametric mixing console data. The proposed research intends to extend the work currently being carried out in the master thesis, by addressing further areas of investigation: the design of an adversarial training regime to capitalize on non-parallel data, and the formulation of the learning problem within a generative framework to model mixing as a one-to-many mapping. While this work is largely exploratory, these methods have the potential to solve many of the short-comings of earlier approaches, providing a more sophisticated model of the mixing process.

Research