Sometimes I just want a quick way to send some OSC (open sound control) messages around. I can jump into a Python command line and use simpleosc easily enough, I guess.
But I spend a lot of time at a *nix command line, and it would be nice to just run a command to send a message. Maybe something like this:
aturley@machine% args2osc "/stupidoscsynth" sf on 0.2 127.0.0.1 9596
The first argument (“/stupidoscsynth”) is the message address, the next argument (“sf”) says that this message has two arguments whose types are string and float, the next two arguments (“on” and “0.2″) are the message’s string and float arguments, and the last two arguments (“127.0.0.1″ and “9596″) are the destination host and port.
I probably could have just written a little Python script and been done with it.
import osc
import sys
def convert_element(in_element_type, in_element):
type_dict = {"i":int,
"f":float,
"s":str}
if type_dict.has_key(in_element_type):
return type_dict[in_element_type](in_element)
return None
if "__main__" == __name__:
osc.init()
osc.sendMsg(sys.argv[1], [convert_element(t, v) for t, v in zip(sys.argv[2], sys.argv[3:-2])], sys.argv[-2], int(sys.argv[-1]))
#include <sstream>
#include <iostream>
#include <string>
#include <vector>
#include "osc/OscOutboundPacketStream.h"
#include "ip/UdpSocket.h"
using namespace std;
int StringToInt(string inString) {
istringstream buffer(inString);
int outInt;
buffer >> outInt;
return outInt;
}
float StringToFloat(string inString) {
istringstream buffer(inString);
float outFloat;
buffer >> outFloat;
return outFloat;
}
vector< vector<string> > ZipStrings(const vector<string> &a, const vector<string> &b) {
vector< vector<string> > v;
vector<string>::const_iterator ia = a.begin();
vector<string>::const_iterator ib = b.begin();
for (;
ia != a.end() && ib != b.end();
ia++, ib++) {
vector<string> vt;
vt.push_back(*ia);
vt.push_back(*ib);
v.push_back(vt);
}
return v;
}
vector<string> CStringToStringVector(const char *str) {
vector<string> v;
int len = strlen(str);
for (int i = 0; i < len; i++) {
v.push_back(string(1, str[i]));
}
return v;
}
int main(int argc, char *argv[]) {
if (argc > 4) {
vector<string> typeVector = CStringToStringVector(argv[2]);
vector<string> argVector;
for (int j = 3; j < argc - 2; j++) {
argVector.push_back(argv[j]);
}
vector< vector<string> > typeArgPairs = ZipStrings(typeVector, argVector);
int bufferSize = 2048;
char buffer[bufferSize];
osc::OutboundPacketStream p(buffer, bufferSize);
UdpTransmitSocket transmitSocket(IpEndpointName(argv[argc - 2], StringToInt(string(argv[argc - 1]))));
p << osc::BeginMessage(argv[1]);
for (vector< vector<string> >::iterator i = typeArgPairs.begin(); i != typeArgPairs.end(); i++) {
if ((*i)[0] == "i") {
p << StringToInt((*i)[1]);
} else if ((*i)[0] == "f") {
p << StringToFloat((*i)[1]);
} else if ((*i)[0] == "s") {
p << (*i)[1].c_str();
}
}
p << osc::EndMessage;
transmitSocket.Send(p.Data(), p.Size());
}
}
Assuming you already have the oscpack library installed, you can put this in a file called args2osc.cpp (or anything you want) and compile it like this:
g++ -o args2osc args2osc.cpp -l oscpack -I/usr/include/oscpack/
This is a pretty nice tool for testing programs that receive OSC messages. Maybe later I'll add support for bundles.
// what does a bubble sort sound like?
10 => int noteCount;
if (me.args() == 1)
me.arg(0) => Std.atoi => noteCount;
SqrOsc osc1 => ADSR adsr => Gain g => dac;
SqrOsc osc2 => adsr;
0.5 => g.gain;
// create the unsorted array of MIDI notes
int notes[noteCount];
for (int i; i < noteCount; i++) {
Std.rand2(40, 90) => notes[i];
}
fun void bubbleSort(int l[])
{
for (int i; i < l.cap(); i++) {
for (int j; j < l.cap() - i - 1; j++) {
l[j] => Std.mtof => osc1.freq;
l[j + 1] => Std.mtof => osc2.freq;
adsr.keyOn();
0.2::second => now;
adsr.keyOff();
if (l[j] > l[j + 1]) {
l[j + 1] => int temp;
l[j] => l[j + 1];
temp => l[j];
}
}
}
}
bubbleSort(notes);
The sound is kind of fun, and because of the way bubble sort works, you end up with some structures that repeat. You might almost call it musical.
Going a little further with the idea of playing the output of a sort, I began to wonder if I could maybe be lazy and just use sorting algorithms that have already been implemented. Most sorting algorithms work by comparing two elements and rearranging them depending on which one is larger. If you are comparing numbers, the criteria for “larger” is fairly well understood. But if you are trying to sort by, oh, let’s say an employee id, and the id is some collection of numbers and letters, and those numbers and letters may mean something special to the way they are sorted, then you want to be able to specify what “larger” actually means. The qsort() function in the C standard library takes a comparison function as an argument. This function knows how to compare two elements in the list that is passed as one of the other arguments to qsort(). That way qsort() doesn’t have to know anything about the elements it is sorting. In Java, there is an interface called Comparable. Objects that implement this interface must provide a method called compareTo() that knows how to compare itself to another object. Each time a sorting function needs to compare one item to another, it uses this compareTo() method.
Can we hijack compareTo()? You bet. Here’s a simple class that implements compareTo():
class Buzz implements Comparable {
int val;
public int compareTo(Buzz b) {
return b.val - this.val;
}
}
What if, in addition to returning the difference between the two values we also sent the values out of the program to another program that would do something with them? Well, thanks to an OSC library called JavaOSC we can do this fairly easily.
I put together a Groovy program that creates a list of objects that implement Comparable and then sorts the list. Their compareTo() method sends OSC messages to the local host (127.0.0.1) on port 6767. These OSC messages specify which numbers are being compared.
import com.illposed.osc.OSCPortOut;
import com.illposed.osc.OSCMessage;
import java.net.InetAddress;
import java.util.Random;
class Buzz implements Comparable {
int val;
OSCPortOut sender;
public int compareTo(Buzz b) {
Object[] args = [new Integer(b.val), new Integer(this.val)];
OSCMessage msg = new OSCMessage("/comparing", args);
this.sender.send(msg);
sleep(200);
return b.val - this.val;
}
}
OSCPortOut sender = new OSCPortOut(InetAddress.getByName("127.0.0.1"), 6767);
list = [];
rand = new Random();
20.times {
list << new Buzz(val: 40 + rand.nextInt(51), sender: sender);
}
list.sort();
And here's a Chuck program that listens for the messages and plays the notes
I think the default list sorting algorithm in Java is a quicksort (edit: actually it looks like it is a mergesort, and it may be replaced in the future by timsort), but you can probably find others. And if you can't find anything, you can always write your own. Once you have a sorting algorithm that accepts a collection of Comparable objects, you can use the code I've provided to listen to it.
making some sort of “instrument” that a group of people could play together
making some horrible noise
My goal going into this was to create a system that would allow each user to create a circular sequencer in a web browser (most likely running on a smart phone) by dragging elements (like samples, and the sequencer itself) around. The sequencers would all play through a common sound system, and would all be displayed together in some sort of global view.
From the user’s perspective, what you have is a sequencer. You create your own sequence, other people create their own sequences, everybody sees all the sequences up on one big display. This is an instrument that people can play together. Like the world’s worst piano duet. On sequencers. With bad samples. As the video shows, I’ve gotten this to work with the iPhone, so you could, in theory, entice a room full of people with iPhones to lay down some self-organizing beats. I haven’t done much testing on the Palm Pre or Android, but I think it should work, assuming they support the standard browser mouse events. The user first clicks somewhere in the red box to draw the initial circle sequencer. She can then play with the circle. She can click on the edge of the circle and drag to change its size (which changes the amount of time that it takes to get around the circle, which effectively changes the speed of the sequence). She can click in the middle of the circle to move it. She can click outside the circle and drag to the edge of the circle to add a sample (more on that in a minute). And she can click on a sample and drag it to a different part of the circle. There are four samples that can be used, and the user chooses the sample to put on the circle by clicking in outside of the circle in one of four quadrants and dragging to the desired location on the circle. Imagine that the center of the circle is the point where two perpendicular lines meet, giving you four quadrants. The samples break down thusly:
upper-right = noise
lower-right = buzz
lower-left = beep
upper-left = tweet
So, lots of people can create these little sequencers using their smart phone web browsers. And then there’s a big screen that shows everybody’s sequencers, and shows the current location in each sequence with a little white circle. And there’s some sort of PA that’s blasting out the sequences. And the crowd goes wild.
From a technical perspective, this is an example of how to use a bunch of different technologies to cobble together a demo that explores some random ideas.
Let’s start at the browser. We have your run-of-the-mill HTML and JavaScript. Nothing too fancy here. I’m taking advantage of some of the iPhone’s touch-specific event handlers, but that’s about it. Oh, and I’m using JQuery, which in this day and age is pretty routine. So my HTML and JavaScript handle events and draw pretty pictures. And they send messages to the HTTP server. I’m using some JQuery AJAX and JSON functionality to send messages to the server when things (like circles or samples) get created or moved.
The web server is a bit of custom Python. If you’re familiar with BaseHTTPServer then you know that Python makes it very easy to write your own web server. If not, let’s just assume I’m awesome and I write web servers in between sky diving and fighting terrorists. You can see me starting the server in the video with this command:
sudo python ../polosc/polosc.py 80 cs.not cs.map
The server runs on port 80, and takes two types of input files. The .map files map HTTP request paths to actual files on the system. The .not files tell the server that when it receives a POST request for the specified path it should look at the posted JSON data and use it to send an OSC message (using simpleOSC) to the specified server. The message format is pretty simple. It looks like this:
{"address":ADDRESS, "data":[DATA1, DATA2, DATA3, ...], "type":[TYPE1, TYPE2, TYPE3, ...]}
The “type” list is optional, but it is used to get around the following facts:
the JavaScript JSON code sends values that look like ints even when I want them to be floats
the Python JSON libary sees these ints and treats them like ints
Chuck needs separate handlers for OSC messages that take different arguements, even if they go to the same address
So, now we have an OSC-to-HTML bridge. But where do these OSC messages go?
They go to the program that runs this whole show, circlesequencer.ck, which is written in Chuck. This program takes the OSC messages describing creation and update events, and uses them to construct a model of what’s going on. It then uses this model to play the sequences from each circle sequencer. And it updates the global view, which will be discussed shortly. I chose Chuck because I like the way it handles timing and audio. I probably could have used another language and imported a timing library, but the Chuck timing model is a joy to use. This almost makes up for the fact that Chuck’s data structures are horribly anemic. Almost. Between its insistence on exact type matches for OSC messages and a weak hash-map interface, I spent a lot of time fighting with Chuck to get it to do what I wanted. But I finally got it to work, so I guess I’ll consider that a victory. Earlier I called the sounds “samples”, but they’re actually just really simple waveforms with simple attack-decay envelopes. Chuck is powerful enough that you could put almost any kind of audio that you wanted in here, but I stuck with some minimalist sounds for the demo.
When a change occurs in the representation of the world, the Chuck program sends an OSC message to the global view server. “Global view server” is just a way of saying that I wrote a program that displays all the circle sequencers in one place and shows a little white circle for the current position of each sequence. The global view server is written in Processing and it knows how to handle the OSC messages that the Chuck program sends out.
Finally, to get the video working I had to use PD, because my video recording software wants to use Soundflower to capture application audio, but Chuck and Soundflower weren’t cooperating. So I piped the audio from Chuck out to Jack, got that audio in PD, then sent it out to Soundflower, which was picked up by the video software. And thus I had a video with sound, and lots of audio latency. I eventually got everything cleaned up in iMovie.
I’ll post a little postmortem and post some code later.
(later)
Implementation
I’m going to present this code as-is. It was exploratory, so there are lots of global variables and functions that should have been methods floating around. Things came up, I threw in a quick solution. If you dig around the code, you’ve been warned.
The best way of understanding the system is to think of it as following an MVC pattern. The web frontend acts as the controller, the Chuck layer acts as the model, and the Processing program acts as the view. Things are actually a little messier than that, but thinking in terms of MVC helps orient the discussion.
Controller
I’m not much of a web programmer, so I’ve probably done lots of stupid things here. At a high level, the controller takes mouse or touch events (which we can generally call “up events” and “down events”), figures out where they are relative to the canvas element, decides what to do, and redraws the canvas. The “decides what to do” can be broken down like this:
If no circle sequencer exists and a down event occurs, create one where the event occurred.
If a down event occurs within the circle and a little way from the edge, move the circle to the point where the up event occurs.
If a down event occurs near the edge of the circle but not an a sample marker, change the radius to the point where the up event occurs.
If a down event occurs near the edge of the circle on a sample marker, move the marker to the point on the circle point where the up event occurs.
If a down event occurs outside of the circle, place a sample marker on the circle where the up event occurs.
At the end of each of these actions, some JSON-encoded data is send to the HTTP server using an XMLHTTPRequest call, which is taken care of by JQuery’s ajax() function.
I generate a random circle ID every time the page loads. A better solution would be to generate the ID the very first time the page loads and store that in a cookie, then reuse it. An even better solution would be to have the server generate the ID and store that as a cookie. Maybe later.
HTTP-OSC Bridge
Before talking about the model itself, I will take a moment to talk a little more about the HTTP server and the way it converts JSON data to OSC messages. I wrote the HTTP server because I felt like I was constantly rewriting different versions of it anyway for my various web projects, and I thought it might be nice to abstract everything out into an HTTP-OSC bridge. It is written in Python and takes advantage of lots of powerful modules that already exist. Mostly my code just glues these modules together. Just to be up front, it requires Python 2.6 because it uses the json module. You might be able to use an older version of Python and include the stand-alone json module, but I can’t really offer any help there.
I talked a little bit about the format of the JSON messages earlier, but here’s a refresher on what they look like:
{"address":ADDRESS, "data":[DATA1, DATA2, DATA3, ...], "type":[TYPE1, TYPE2, TYPE3, ...]}
Let’s say I wanted to the following OSC message (broken into plain text parts so as to avoid confusion):
This would tell the model to create a circle with an id of "A7CC9D8" and a radius of 20.0 at location (80.0, 30.0). To do this, I would send the following JSON data:
{"address":"/circlesequencer/circle/create", "data":["A7CC9D8", 20, 80, 30], "types":"sfff"}
I will talk a little more about the OSC messages when I describe the model.
The OSC forwarding behavior is specified in a .not file (short for "notification"). The file consists of comma-separated entries that look like this:
/command,127.0.0.1,6464
This tells the server that whenever a HTTP POST has a path of "/command", it should generate an OSC message from the posted data and send it to the server "127.0.0.1" on port 6464. The server uses a similar mechanism to specify how to map request paths to filesystem paths.
This is a general-purpose program. The only thing that ties it to the circle sequencer system is the configuration files. So if you want to build something else and you need a way of turning HTTP messages into OSC messages, I would encourage you to consider using this. You know, like if you wanted to play a set using Ableton Live, and you wanted to give the audience control over some synth parameters from their iPhones (using the OSC functionality of the coming-really-soon-but-not-yet Max/MSP integration).
Model
The model is the brains of the operation. It is written in chuck, and it keeps track of all the sequencers and is responsible for updating the current sequencer position. It also generates the sequencers' sounds. When it receives an OSC message it update's it's version of the world. It accepts the following OSC messages:
/circlesequencer/circle/create,sfff circle_id radius x y
/circlesequencer/circle/update,sfff circle_id radius x y
It also sends the messages on to the view after it has updated itself. If the update fails, no message is sent to the view. For example, a client might send it a message saying that a marker has been added to circle "XYZ", but the message will be ignored if the model has not received a "/circlesequencer/circle/create" message for circle "XYZ". In addition to the creation and update OSC messages, it sends messages to the view whenever the sequencer's position has changed (it's position in the sequence, not it's position on screen).
View
The view displays all of the sequencers that the model has told it about. It is written in Processing and uses the oscP5 library (yes, that appears to be blinking text on the page) to handle the OSC messages coming from the model. The implementation is pretty simple.
Postmortem
It Worked!
I started off with a somewhat vague idea, and just started writing code. This was a good exercise in exploratory programming. The only part that was really well thought out beforehand was the HTTP-OSC bridge, and that was a consequence of having written more or less the same code several times in other projects. I knew what I wanted, and I knew how to get it. As I said before, most of these things grew organically, based on immediate needs, so it lacks organization. But ugly code that works is better than a bunch of UML diagrams.
Order of Implementation
I started off writing the HTTP-OSC bridge. In fact, you could say that this project was mostly a test-bed for that. Once I had that in place, I wrote the model. I figured that would give me a good shot at figuring out what kinds of data and operations I needed. I tested the model using curl to send HTTP messages to the HTTP-OSC bridge and simpleOSC and the Python shell to send OSC messages directly. With the model and the HTTP-OSC bridge in place, I started working on the controller. Once I had a first pass at the controller, I moved on to the view. Then I went back and refined the controller.
Repetition
One problem with this approach was that I had to implement the circle and marker classes in JavaScript, Chuck, and Processing (which is really just Java). I know that there are systems out there that let you specify a class in one language and then export to others, but I'm not sure any of them support Chuck. And I wasn't really interested in exploring that for this project. So I went ahead and repeated myself a few times. Not a big deal with a fairly simple set of classes, but a few more layers and a little more complexity might have made it very painful. But by the time I got to the view I more or less knew what I needed, so I was able to code it up quite quickly.
Separation of Concerns
In retrospect it might have made sense to move the code that handled playing the sounds into another system. Basically it would have been another "view" only in this case it would have been for hearing rather than seeing. I probably would have just sent OSC messages from the model to this view whenever I needed to play a sound. Who knows, maybe I would have played around with SuperCollider or something like that.
As an alternative I could have written the model and the view (and even the HTTP-OSC bridge) in the same language and run them as one big executable. This consolidation could have saved me some of the time that I spent re-implementing classes and functions. On the other hand, I would have had to find a language that was expressive in the necessary domains, and I'm not sure that one exists. Chuck handles audio and time very well (and OSC slightly less well), but has no way to handle HTTP or video. Processing does a decent job of drawing pictures, but it lacks Chuck's audio prowress. Ultimately if this was a commercial product I might take what I've learned and rewrite the whole thing in Java or C, but that would be down the road a bit.
The Right Tool for the Job(?)
JavaScript (and JQuery)
If you're writing something to run in a mobile web browser, JavaScript is more or less the ONLY tool for the job. And I don't have a lot of experience with JavaScript, so I don't know that I'm qualified to say much about it. What I can say is that JQuery provides powerful wrapper around the XMLHTTPRequest object. If you're starting out with JavaScript, go ahead and take a look at JQuery.
Python
I've used Python enough that its idioms creep into my general programming. As a result, I didn't have any problem using it to write the HTTP-OSC bridge. The "batteries included" philosophy means that if you want to do something there's probably a pretty easy way to do it using the standard modules. I'll admit that at this point I'm probably too biased to offer an objective opinion.
Chuck
As I've said before, Chuck gives you lots of powerful features. It's fairly easy to send and receive OSC messages. The concept of time is built into the language. And it gives you lots of ways to create and manipulate audio. But there are some things that could be better.
The OSC message receiving system insists on knowing exactly how many arguments each message will have, and the type of each argument. This can make it difficult to use with languages that convert easily between floats and ints and allow that conversion to impact the way they generate the OSC messages that you send to Chuck. I had problems when Python would send an int instead of a float because it got a number from the HTTP-OSC bridge that didn't have a decimal point. You could argue that this is actually a problem on the Python side, but most other OSC systems that I have used allow you to set up a handler on an address and then read out the message arguments one at a time. This at least lets you set up handlers that show you when the program has received a message, rather than just ignoring them.
It would be nice if Chuck let you specify associative arrays using a syntax like this:
{"key1": value1, "key2": value2 . . .}
Other languages let you do this, and it makes it much easier to use them. Also, why doesn't array.size() return the number of elements in an associative array? I had to create a list of array keys and then loop over that to retrieve the values from an associative array. This seems ugly. Maybe there's a good implementation reason.
Processing
Processing gives you some easy ways to draw things in a window. But once you get past the syntatic sugar, you're working with Java. This gives you all the power of Java, but also all the verbosity. As with Chuck, there is no easy way to create things like associative arrays. At least Java gives you lots of powerful datatypes.
I actually tried to get Jython working with Processing. I figured that would give me the best of both worlds, the flexibility of a scripting language with the UI goodies of Processing. Unfortunately I had some problems getting all the classes to load, so I gave up. Maybe I should have tried a little harder.
OSC
OSC is lightweight, fairly flexible, and supported on a number of different platforms. I really can't complain too much.
I wrote a little iPhone game based on some of the same principles that I used in the TUIO system I blogged about earlier. In this system, users go to a URL to download a controller, which they then use to control spaceships (little circles on the computer screen). The game itself is hosted on a computer that is running a special web server and the game software. You can download the source for everything here.
If you were at Maker Faire and played the game at my booth, thanks for being a beta tester. Sorry if I crashed your browser.
UPDATE: If you’re coming from the MacWorld slideshow, thanks for stopping by. If you tried to play the game at Maker Faire and had trouble using the system, I apologize. Thanks to everybody who had the patience to get on the network and try out the game.
Every once in a while I run across a blog post or a comment somewhere about how somebody wants to send a MIDI message from one computer to another. Maybe they have a keyboard that they have hooked up to one computer and they want to use it to control somebody else’s Live session. Or maybe they have some windows software that spits out MIDI, and they want to use it with some Mac software that understands MIDI.
So, let’s say you want a cross-platform solution (since if you’re only using Macs then this ships with the operating system), and you want it for free, and maybe you want it to be fairly easy to change the way the system works in case you want to try to do something weird later on down the road. To that end, I’ve written a MIDI-OSC-MIDI bridge, which you can download here. It is actually two programs, a sender and a receiver, and they use Open Sound Control to communicate. To use it, you have to install a programming language/environment called ChucK. You will probably also want to have some MIDI loopback devices set up. In OS X, you can use the Audio MIDI Setup utility to add virtual MIDI ports by selecting “IAC Driver” and adding ports. In Windows you will have to use a program like MIDI Yoke.
Go ahead and install Chuck and set up some MIDI loopback devices if you haven’t already. I’ll wait.
Now that you have ChucK installed, create two files. Let’s call the first one miditoosc.ck. It will look like this:
Let’s call the second one osctomidi.ck. It will look like this:
if (me.args() != 2) {
<<<"usage: chuck osctomidi.ck:PORT:MIDIDEVICE">>>;
me.exit();
}
me.arg(0) => Std.atoi => int port;
me.arg(1) => Std.atoi => int dev;
OscRecv recv;
port => recv.port;
recv.event("/midi/raw, iii") @=> OscEvent oe;
recv.listen();
MidiOut mout;
if (!mout.open(dev)) {
<<<"Could not open MIDI device " + dev>>>;
me.exit();
}
MidiMsg msg;
while (oe => now) {
while(oe.nextMsg() != 0) {
oe.getInt() => msg.data1;
oe.getInt() => msg.data2;
oe.getInt() => msg.data3;
mout.send(msg);
}
}
The first program waits for a MIDI message. When it receives a MIDI message, it will take the data from the message, put it into an OSC message, and send the message. The second program reverses this process, waiting for an OSC message, putting the data into a MIDI message, and sending that message out to a MIDI port.
To use these programs, place the files on two computers that both have ChucK installed. Then, run the following command on both computers:
chuck --probe
This will display a list of the computer’s audio and MIDI interfaces. On my computer I see something like this:
[chuck]: ------( chuck -- 2 MIDI inputs )------
[chuck]: [0] : "IAC Driver Bus 1"
[chuck]: [1] : "IAC Driver IAC Bus 2"
[chuck]:
[chuck]: ------( chuck -- 2 MIDI outputs )-----
[chuck]: [0] : "IAC Driver Bus 1"
[chuck]: [1] : "IAC Driver IAC Bus 2"
[chuck]:
This tells me that there are two MIDI inputs (numbered 0 and 1) and two MIDI outputs (numbered 0 and 1). If I wanted to run osctomidi.ck and have it listen for OSC messages on port 6969 and send MIDI messages to the output called “IAC Driver Bus1″, I would run the following command from the directory that contained the file osctomidi.ck:
chuck osctomidi.ck:0:6969
If you didn’t already know, arguments are passed to Chuck processes by separating them with colons. The first argument is the device number (0) and the second argument is the port (6969). To run miditoosc.ck, I would use a similar command:
chuck miditoosc.ck:127.0.0.1:1:6969
In this case the first argument is the host to which I want to send the OSC messages (I’m just sending to the local host here), the next argument is the device number, and the final argument is the port. If both of these programs are running and the host named in the second command is the host where the first command is run, then you should have a link between the two computers (or, in this case, one computer) that will connect the MIDI input from the second host to the MIDI output on the first host.
Just a note: If you are doing this all on one computer you may see a message that says something like “cannot bind to tcp port 8888″. You can safely ignore that message.
I’ve defined an OSC message here that looks like this:
/midi/raw [data1] [data2] [data3]
where data1, data2, and data3 are integers. These integers are just the data that would normally make up a MIDI triplet. I’ve used the message address “/midi/raw” because it might make sense at some later point to have different OSC messages for different types of MIDI messages. For example, I might use something like this for MIDI CC messages:
/midi/cc [control] [value] [channel]
But for now I want to remain message-type agnostic.
Since these programs use OSC they can provide a bridge between MIDI and environments that may support OSC more easily than MIDI. For example, there are a number of OSC packages for Python that are easy to use, but MIDI libraries seem to be fairly platform-specific. Or there is the case of Java on the Mac, which does not support the standard Java MIDI libraries.
So there you have it. ChucK, OSC, and MIDI, working together.
This is a little PD hack that lets you set a value using an OSC message. The OSC message comes in on port 5678, and has the following form:
/cmd [receiver] [value]
where “[receiver]” is the name of the receiver, and “[value]” is the new value. The unpack and repack are necessary to convert the first argument into a symbol.
If you have Python and the simpleosc library, you can set the value connected to “jesus” to 10 like this:
>>> import osc
>>> osc.init()
>>> osc.sendMsg("/cmd", ["jesus", 10], "127.0.0.1", 5678)
A little while back Apple rejected an iPhone app that would have let you use your iPhone as a TUIO controller. There was a big discussion about the whole thing over at CreateDigitalMusic, with some people supporting Apple’s decision and others decrying it. Most of those who supported Apple seemed to feel that it was their store and they could damned well sell whatever they wanted. I tend to agree with this sentiment, but I still think that it would be nice to have a way to get an app onto the iPhone without having to get Apple’s permission.
At about the same time the discussion was taking place over at CDM, I was playing with a program called OSCemote on my brand new iPhone. OSCemote lets you create interfaces for your iPhone that can send and receive OSC messages. Interfaces are created as HTML documents, with snippets of Javascript doing the work of monitoring elements on the page and sending the messages. I began to wonder how hard it would be to use web app running in Safari as a multitouch interface.
I’ve been playing around with these ideas for the last month or so, and tonight I decided to go back to the issue that got me going on this train of thought. I took a look at the TUIO protocol and took a swing at creating a web app that would talk to a web server that would in turn send TUIO messages to other programs. I’m not sure I got all the details of TUIO right, but I think I have a workable first pass at something.
I’ve packaged the system up, and you can download it here. You’ll need Python to run the web server. If you have a Mac then you already have Python. Users of other operating systems may have to install it if it isn’t already there. You will also need some sort of program that can do something with the data. I’ve just been dumping it to a Pure Data patch. And you’ll need an iPhone or an iPhone simulator.
UPDATE (4/20/2009): The link now links to the new version which uses the 2Dcur profile instead of the 2Dobj profile. Thanks to Martin Kaltenbrunner for pointing this out.
Last year Tim, Mike and I built some big LED panels for Tim’s band Microfiche. Tim is a graphic/industrial designer, Mike is an electrical engineer, and I’m a programmer, and the project was a great way for us to bring all of our skills together to create something cool. Here’s some pictures of us (and some dear friends) during the build phase:
And in this video you can see one of the panels during a show:
While we were working on the project we started to call ourselves Tricerabot, and the name stuck. This year we aim to have a Tricerabot booth at Maker Faire. Mike will have some new versions of his shifters with microcontrolled LEDs putting on light shows. Tim will have some of his new illustrations and designs (he does the Tricks of the Trade comic strip which appears monthly in Make Magazine). I might have some new interactive goodies (maybe some games or something).
And we’re working on a collaborative project to highlight the way our strengths complement each other. One plan is to use some wireless microcontroller boards to set up a proximity detection system. The system will be able to detect how far we are from the booth, and this information will be used to control a projected image. We talked about the image representing each of us as a different colored square that moves along an path, with the paths intersecting in the the middle. As the squares get closer and overlap, new images will emerge. Here’s a mock up of what we’re talking about:
In the novel Dirk Gently’s Holistic Detective Agency, one of the characters has created a spreadsheet that sets data to music. Thanks to Songsmith, we’re more or less there now.
I suppose it’s only a matter of time before Microsoft realizes that the best use for their new toy is as a chart type in Excel.
Microsoft recently released Songsmith, software that takes a vocal performance and generates a musical score.
The original intention, as I understand it, was to give singers (or people who fancied themselves as such) an easy way to put some music around their songs. Ah, but the law of unintended consequences had other ideas.
What Microsoft has done is opened a rift between our universe and some sort of parallel universe where all the songs we know were actually a collaboration between a songwriter and a bunch of mediocre musicians with Casio synths. A part of me is afraid that this rift will eventually annihilate our own universe. The only way to stop the onslaught is by embracing a new, un-Songsmithable aesthetic.
But I don’t know if I want it to end. I’m really enjoying going to YouTube every day and finding new songs. And as someone with at least a passing interest in the intersection of computers and the arts, I’m at least sort of impressed with what Microsoft has done technologically.
As a final thought, I hypothesize that there is some sort of identity relationship between MIA and Songsmith, whereby feeding Songsmith an MIA song will actually produce the same song.
Or maybe, just maybe, the awesomeness is amplified into this:
