Javascript Sound Experiments - Filtering [ProcessingJS + WebAudio]
I am working on a sound installation project where we are building our custom devices to measure distance (using wireless radiofrequency technology) & create sounds (using DAC integrated circuits with SD card storage, connected to a custom built analogue filter modul) through simple electronic devices. The building of the hardware with Marton Andras Juhasz is really exciting and inspirative in itself, not to mention the conceptual framework of the whole project. More hardware related details are coming soon, the current state is indicated on the video below:
The scope of this entry is about to represent filtered audio online, natively in the browser. I have to build an online representation of the system where the analog hardware modules can be seen and tried out within a simplified visualization (that is mainly to indicate the distances between the nodes in the system). Apart from the visualization, sound is playing a key role in the online representation, so I started playing with the Web Audio API , where a simple sound playback can be extended with filtering, effecting using audio graphs. The state of web audio might seem a bit confusing at the moment (support, browsers compatibility etc). To minimize confusion, you can read more about the Web Audio API here.One is for sure: the context of web audio is controlled via JavaScript, so it can fit well into any project that is using high level JavaScript drawing libraries, such as ProcessingJS.
The basic structure that we do here is
1. Load & playback soundfiles
2. Display information, create visual interface
3. Use control data from the interface and drive the parameters of the sounds
Load & Playback soundfiles
I found a really interesting collection of web audio resources here that are dealing with asynchronous loading of audio, preparing their content for further processing by storing them in buffers. Sound loading and storing in buffers is all handled with the attached Javascript files. If you need to switch sounds later, you can do it asynchronuously by calling the loadBufferForSource(source to load, filename) anytime anywhere on your page.
Display Information, create visual interface
Connecting your processingjs sketch to the website can be done by following this excellent tutorial. You should make functions in your Javascript that are “listening” to processingjs events. You can call any function that can accept any parameters. One important thing is that you have to actually connect the two systems by instantiating a processingjs object as the following snippet shows:
var bound = false;
function bindJavascript()
{
var pjs = Processing.getInstanceById('c');
if(pjs!=null)
{
pjs.bindJavascript(this);
bound = true;
}
if(!bound) setTimeout(bindJavascript, 250);
}
bindJavascript();
We have some work to do in the processingjs side, also:
// create javascript interface for communication with functions that are "outside"
// of processingjs
interface JavaScript
{
void getPointerValues(float x, float y);
void getFilterValues(float filter1, float filter2);
void getGainValues(float gain1, float gain2);
}
// make the connection (passing as the parameter) with javascript
void bindJavascript(JavaScript js)
{
javascript = js;
}
// instantiate javascript interface
JavaScript javascript;
Use control data from the interface and drive the parameters of the sounds
After connecting the html page with our sketch it is possible to use any action in the sketch to control our previously loaded sounds. This way we separate the whole audio engine (handling, storing, etc) from the visual interface (the processingjs canvas with rich interactive contents) by linking the two with a few functions to pass some parameters. These binding functions might remind someone to the OSC way of connecting of two softwares where functionalities are staying clearly isolated within the different systems, only the parameters are traveling between them represented as numbers, labels or other types of symbols.
This example at the moment works only with Chrome (new stable releases). According to the developer docs it should work with the new Safari (WebKit) browser but I couldn’t make it. Hope this will be fixed soon…
Try out the LIVE DEMO (you need latest stable Google Chrome
)
Download full source code (zip) from here