Phalanger: Controlling Music Software With Hand Movement Using A Computer Vision and Machine Learning Approach

Kiefer, Chris, Collins, Nick and Fitzpatrick, Geraldine (2009) Phalanger: Controlling Music Software With Hand Movement Using A Computer Vision and Machine Learning Approach. In: New Interfaces For Musical Expression, 4th June - 6th June 2009, Pittsburgh, USA.

[img]
Preview
PDF
Download (361kB) | Preview
Official URL: http://www.nime2009.org

Abstract

Phalanger is a system which facilitates the control of music software with hand and finger motion, with the aim of creating a fluid style of interaction that promotes musicality. The system is purely video based, requires no wearables or accessories and uses affordable and accessible technology. It employs a neural network for background segmentation, a combination of imaging techniques for frame analysis, and a support vector machine (SVM) for recognition of hand positions. System evaluation showed the SVM to reliably differentiate between eight different classes. An initial formative user evaluation with ten musicians was carried out to help build a picture of how users responded to the system; this highlighted areas that need improvement and lent some insight into useful features for the next version.

Item Type: Conference or Workshop Item (Paper)
Keywords: gesture,computer vision,AI,machine learning, neural networks, support vector machines, music, instruments, HCI
Schools and Departments: School of Engineering and Informatics > Informatics
Subjects: M Music. Literature on music. Musical instruction and study > M Music
Q Science > QA Mathematics > QA0075 Electronic computers. Computer science
M Music. Literature on music. Musical instruction and study > MT Musical instruction and study
Q Science > QA Mathematics > QA0076 Computer software
Depositing User: Chris Kiefer
Date Deposited: 23 Jun 2009
Last Modified: 30 Nov 2012 16:53
URI: http://sro.sussex.ac.uk/id/eprint/2202
Google Scholar:5 Citations

View download statistics for this item

📧 Request an update