University of Sussex
Browse
paper97.pdf (228.41 kB)

Live coding machine learning and machine listening: a survey on the design of languages and environments for live coding

Download (228.41 kB)
conference contribution
posted on 2023-06-09, 17:01 authored by Chris KieferChris Kiefer, Thor MagnussonThor Magnusson
The MIMIC (Musically Intelligent Machines Interacting Creatively) project explores how the techniques of machine learning and machine listening can be communicated and implemented in simple terms for composers, instrument makers and performers. The potential for machine learning to support musical composition and performance is high, and with novel techniques in machine listening, we see emerging a technology that can shift from being instrumental to conversational and collaborative. By leveraging the internet as a live software ecosystem, the MIMIC project explores how such technology can best reach artists, and live up to its collaborative potential to fundamentally change creative practice in the field. The project involves creating a high level language that can be used for live coding, creative coding and quick prototyping. Implementing a language that interfaces with technically complex problems such as the design of machine learning neural networks or the temporal and spectral algorithms applied in machine listening is not a simple task, but we can build upon decades of research and practice in programming language design (Ko 2016), and computer music language design in particular, as well as a plethora of inventive new approaches in the design of live coding systems for music (Reina et al. 2019). The language and user interface design will build on recent research in creative coding and interactive machine learning, exemplified by the Rapid Mix project (Bernardo et. al., 2016, Zbyszynski et. al., 2017). Machine learning continues to be at the forefront of new innovations in computer music, (e.g. new sound synthesis techniques in NSynth (Engel et. al. 2017) and WaveNet (van den Oord, 2016)); the language will seek to integrate models based around these new techniques into live coding performance, and also explore the efficacy of live coding as an approach to training and exploiting these systems for analysing and generating sound. Existing live coding systems and languages are often reported on, describing clever solutions as well as weaknesses, as given, for example, in accounts of the development of Tidal (McLean, 2014), Extramuros (Ogborn et. al, 2015) and Gibber (Roberts and Kuchera-Morin, 2012). Researchers are typically reflective and openly critical of their own systems when analysing them and often report on its design with wider implications (Aaron 2011; Sorensen 2018). However, they rarely speculate freely and uninhibitedly about possible solutions or alternative paths taken; the focus is typically on the system described. Before defining the design of our own system, we were therefore interested in opening up a channel where we could learn from other practitioners in language design, machine learning and machine listening. We created a survey that we sent out to relevant communities of practice - such as live coding, machine learning, machine listening, creative coding, deep learning - and asked open questions about how they might imagine a future system implemented, given the knowledge we have today. Below we report on the questionnaire and its findings.

Funding

MIMIC: Musically Intelligent Machines Interacting Creatively; G2434; AHRC-ARTS & HUMANITIES RESEARCH COUNCIL; AH/R002657/1

History

Publication status

  • Published

File Version

  • Published version

Journal

Proceedings of the Fourth International Conference on Live Coding

Publisher

Medialab Prado

Event name

ICLC 2019 International Conference on Live Coding

Event location

Media Lab Prado, Madrid

Event type

conference

Event date

16-18th January 2019

ISBN

9788418299087

Department affiliated with

  • Music Publications

Research groups affiliated with

  • Centre for Research in Creative and Performing Arts Publications

Full text available

  • Yes

Peer reviewed?

  • Yes

Legacy Posted Date

2019-02-28

First Open Access (FOA) Date

2019-03-11

First Compliant Deposit (FCD) Date

2019-02-25

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC