The idea of this project is to create a collaborative music system with kinetic interfaces. The system consists of a conductor, several instruments, and leap motions as inputs, which are connected over the internet. To be more specific, the conductor monitors the music data from the instrument and controls the musical features of the instruments. For each musical instrument, the player controls it on a digital instrumental interface with a leap motion. In general, the project aims at creating a system by which users can create music cooperatively and explores how collaborative computing can do for musical or even artistic creation.
The conductor and the virtual instruments are connected over the internet. To achieve this, we utilized a processing library called oscP5. By experimenting multiple example connection methods provided by the library, we decided that the oscP5’s TCP connection is the most suitable for this project. Using the TCP connection methods, we are able to make the server and the clients to respond in real time. More specifically, the clients send pings repeatedly to the server (every 0.5 seconds), and the server acknowledges the pings and responses with the data of musical features to the clients. Meanwhile, this way of connection also allows the system to identify the existence of the clients. If a node stopped to ping the server for a certain long enough time, the server would delete it from its list, and modify the interface accordingly as well.
The instruments have two tasks. First, it provides the players with a digital interface of the instruments by which the players can play the instruments with a leap motion. Second, the players send pings to the server. The pings include the information of the name of the instruments such that the server can identify whether this is a new instrument that just came online, or it is an old one. This means that as long as there is a server online, the clients are free to come and go, thus enhancing the scalability of the system.
For now, the conductor has three functions. Firstly, it controls the instruments that are connected to them by sending them data of musical features. Currently, the musical features only contain the volume, but we are going to explore more on additional features that make sense to play with. Secondly, the conductor is aware of the size and the identity of the whole network. This has two advantages. First, we will be able to design each difference instrument’s feature and send them to the correct instrument. Second, the conductor’s interface can thus be dynamic. For instance, when there are two instruments connected, the interface will be split into two, and more there are more instruments connected. And if one or more instrument leaves the system, the interface can be changed accordingly.
This project is meant to be developed for the whole semester, and this midterm project is the first phase of it. For the rest of the semester, we are expecting the following improvements:
- Create additional interfaces for additional musical instruments.
- Explore the musical features that make sense to alter into more detail.
- Consider using Microsoft Kinect as the user input device for the conductor.
- Improve and beautify the interfaces for both the instruments and the conductor.
- We might try to extend the system out of LAN to achieve better flexibility and scalability.