A wireless MIDI controller for dynamic motion and gesture articulation

Droṇā – A wireless MIDI controller for dynamic motion and gesture articulation by Vincent Yi, Srikanth Sugavanam, Sonia Boscolo is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is a MEng Final Year project that was implemented by Mr. Vincent Yi, supervised by Dr. Sonia Boscolo and myself at the School of Engineering and Applied Sciences (SEAS), Aston University, in 2019-20.
We thank Ableton for providing us with a free version of their Live Lite 10 software for this academic work.
Project implementation by – Vincent Yi.

Vincent Yi is an MEng student studying Electrical and Electronic Engineering at Aston University. His industrial placement was with Tespack, which was awarded the Best EU Start-up Award in 2017. His academic secondment was with the Nanyang Technological University in Singapore.
LinkedIn – https://www.linkedin.com/in/vincent-yi-4219a1162/
Project goal – to realise a light, compact, wireless motion and gesture sensing MIDI controller for use with music production applications.

MIDI controllers have come to be extensively used since the advent of powerful digital audio workstations (DAW) like Ableton, Cubase, Logic Pro, etc. Typically these devices take the form of pianos, allowing registration of polyphonic notes (i.e. chords). Newer generations incorporate more specific control, like mixing controls (volume, balance, effect sends), or launching of clips and samples.
Open source software and hardware have made the realisation of custom built MIDI hardware possible. Examples of these include fully customizable grid controllers, orientation based devices, and even gesture controlled devices. Such devices add an additional dimension to the music making experience. But these devices are bulky, and are often wired, limiting their ease of use. This gap in the market is slowly being discovered, with products like the Genki’s Wave and the Orba debuting in this space.
In his MEng level project at Aston University, Vincent Yi developed a wireless motion MIDI controller, aimed for mounting onto a violin bow. When in play, violin bows are highly dynamic, and intrinsically convey musical expression and emotion. The nuances of these aspects can be captured via motion sensors, e.g. using gyroscopes and accelerometers, and can be used to generate polyphonic note and control MIDI signals. Fast DSP is key for capturing all aspects of bow motion. Alongside, wireless capability will greatly increase artist freedom of expression.
The proposed project invoked concepts from electronics, programming, software development, and user experience design. The aim of the project was to realize a mountable, wireless controller that could work with the DAW Ableton.
The demonstrated device used two Arduino Nano 33 BLE Sense boards in a master-slave configuration.
Remote/Peripheral/Slave transmitter unit for mounting on instrument. Note additional controls (LDR, buttons, pot) are included for expanding device functionality. Master/Central Receiver unit connected to computer
The on-board 6-axis gyroscope was used to convert motion information to MIDI signals, with stabilisation implemented using complementary filtering.
Wireless communication between the master and slave units was implemented using Bluetooth. The software bridge Hairless MIDI enabled our Arduino devices to talk to the DAW Ableton.

Owing to the COVID pandemic, we couldn’t realize the finished mountable device, but we could arrive at a proof-of-principle demonstration of a wireless midi controller. Interestingly, even with the Bluetooth communication protocol and the complementary filtering in place, the observed latency was quite low. From what we have seen so far, the device shows promise for use with dynamic musical instruments, expanding their range of expressiveness.
Click HERE for downloading the codes – these are shared under the CC-BY-NC-SA license for academic re-use.
The project has scope for further development, and even for use in applications beyond music. If you are interested to work on this project further, get in touch!

Droṇā – A wireless MIDI controller for dynamic motion and gesture articulation by Vincent Yi, Srikanth Sugavanam, Sonia Boscolo is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.