TI RBI
10. Software and Hardware Frameworks
10.2. SETO
10. Software and Hardware Frameworks
(a) of an object may be written as:
1 SynthDef(\testTUIO, {|freq = 400, out = 0, amp = 0, vol = 0.25, famp=1|
2 Out.ar([out, out+1], SinOsc.ar(freq, 0, (amp.lag(0.01)*vol*famp)))
3 }).send(s);
4
5 q = ();
6 (
7 q.synths = IdentityDictionary.new; // a storage for synths
8
9 JITseto.action = {|me|
10 s.bind{
11 // make sure there is a synth
12 q.synths[me].isNil.if{
13 q.synths[me] = Synth(\testTUIO, [\vol, 0.2, \amp, 0])
14 };
15 s.sync;
16 me.visible.if({
17 q.synths[me].set(
18 \freq, me.rotEuler[0].wrap(0, 2pi)
19 .linexp(0, 2pi, 400, 800),
20 \amp, 1
21 )
22 }, {
23 q.synths[me].set(
24 \amp, 0
25 )
26 })}}
27 )
28
29 // instantiate SETOServer
30 t = SETO_OSCServer(’_ixya’, setoClass: JITseto);
31 t.gui;
32 t.start;
33 t.stop;
This approach does what is expected. However, it does not make use of the whole potential the TUIO interface definition provides for example to manage object representations and interactions. For this, SETO can be used, which de-couples functionality from implementa-tion details by providing a higher abstracimplementa-tion level, such that the developer can concentrate more on the intended functionality than on the low-level features. SETO therefore takes care of object changes, including their visibility, and provides implementation interfaces to both actions and interactions of tangible objects.
10.2.1. Implementation
As shown in Figure 10.3(a), SETO mainly consists of three types of classes. Each tracked object is represented by an instance of a class derived by SETObject. Added algorithmic behaviour, e.g. the modulation of a corresponding sound depending on the physical object’s motion can be implemented either as a fixed method of a custom class derived from SETObject, or by implementing the action of JITseto, a class providing just-in-time
10.2. SETO
SETO_OSCServer (establishes OSC connection)
SETOIDistance JITseto
1 .. n
1 .. m
< format;
< id;
<>classID;
<>pos
…
abstract update() // called at update SETObject
<knownObjects
<interactions tuioClass interactionClass
… set() alive() gui()
…
SETOServer
< parts;
…
interaction() // called at update SETOInteraction
Sensors / tuioGrabber sends set() / alive() / fseq()
(a) SETO classes
SETO_OSCServer (establishes OSC connection)
SETOIDistance JITseto
1 .. n
1 .. m
< format;
< id;
<>classID;
<>pos
…
abstract update() // called at update SETObject
<knownObjects
<interactions tuioClass interactionClass
… set() alive() gui()
…
SETOServer
< parts;
…
interaction() // called at update SETOInteraction
Sensors / tuioGrabber sends set() / alive() / fseq()
(b) JugglingSounds classes
Figure 10.3.: UML dependency diagrams related to SETO.
functionality for tangible objects. Interactions between physical objects may be defined alike by deriving fromSETOInteraction. With SETO, the above shown example can be written as
1 (
2 q.synths = IdentityDictionary.new; // a storage for synths
3
4 JITseto.action = {|me|
5 s.bind{
6 // make sure there is a synth
7 q.synths[me].isNil.if{
8 q.synths[me] = Synth(\testTUIO, [\vol, 0.2, \amp, 0])
9 };
10 s.sync;
11 me.visible.if({
12 q.synths[me].set(
13 \freq, me.rotEuler[0].wrap(0, 2pi).linexp(0, 2pi, 400, 800),
14 \amp, 1
15 )
16 }, {
17 q.synths[me].set(
18 \amp, 0
19 )
20 })}}
21 )
Then, instantiate a SETOServer:
1 t = SETO_OSCServer(’_ixya’, setoClass: JITseto);
2 t.gui;
3 t.start;
4 t.stop;
10. Software and Hardware Frameworks
In this listing, a SETO_OSCServer is instantiated that holds all representations of tangible objects for the programmer. All objects tracked by the tracking system are represented as JITseto objects, a child of SETObject that evaluates a global class function whenever they are updated. This is the place where the actual audio controlling takes place. At first sight this example might look as complex as the non-SETO one, however it has some benefits compared to it: The simple approach is limited to an explicit number of objects of which the identification numbers have to be successors of each other.
The SETO example allows to use an arbitrary number of objects. While the mapping of object-behaviour to sound parameters is fixed once the system is set up, in the SETO example it is interchangeable at runtime.
10.2.2. Application
Together with colleagues, I implemented several TAIs with help of SETO: AudioDB (see Section 9.4), JugglingSounds (see Section 9.6), Ambit – a tangible environment for ambient data representation [BHR06b], and Tangible Interface for interactive Sonification of multivariate data [HBRR07]. To give an insight into the possibilities of SETO, I describe SETOs use by means of the implementation of JugglingSounds. For detailed information on its use and especially on the Sonification process, please refer to Section9.6.
JugglingSounds consists of two main parts, data acquisition via motion tracking and Sonification via SETO as shown in Figure 9.43(a). For proper Sonification results, the used tracking system had to support capturing of Objects in 6DOF at an update rate of at least 40Hz. Fort his, we used an optical motion capturing system developed by Vicon Inc. . It consists of at least 6 cameras, a data station and a personal computer running the ViconiQ software as well as the server application (Tarsus), which is connected to the data station via Ethernet. This setup is able to compute the 6DOF position of rigid bodies (here three juggling clubs and the juggler’s head) about 120 times per second. This pseudo-continuous multidimensional stream of tracking data is translated into OSC messages by QVicon2OSC, and sends to the Sonification part, running on a separate computer. The non-TUIO-conforming OSC interface of QVicon2OSC required to subclassSETOServer. Its definition (SETOTarsusServer) may be used as a guideline for implementing new tracking interfaces for SETO. Attaching the non-TUIO-conforming OSC interface of QVicon2OSC to SETO required the implementation of a dedicated SETOServerclass. Its definition – SETOTarsusServer– may be used as a guideline for implementing new tracking interfaces in SETO.
SETO internally uses homogeneous transformations, consisting of a translation and a rotation part represented in a 4×4 matrix. The objects’ 6DOF position, however, was provided by QVicon2OSC in axis notation. To save computational time, SETO only computes the transformation into a homogeneous form when it is needed.9
The complexity of the JugglingSounds system forced us to subclass from SETObjectin order to represent juggling clubs and the juggler’s head. The resulting class dependencies are shown in Figure9.43(b). The classClub represents one tracked juggling club, while JugglersHeadcorresponds to the juggler’s head. Each club’s or head’s motion can trigger the computation of Sonification-relevant features. Examples are the state of the club’s
9For more information on the mathematical background on rotational representations, please refer e.g. to http://en.wikipedia.org/wiki/Rotation_representation_(mathematics))
10.2. SETO
symmetric axis (symAxis), or zeroCrossing: an indicator that returns true only if the symmetric axis has crossed thez-plane in the last update. The action methods of Club and JugglersHead were used to update the values of all incorporated NodeProxies.
Examples are shown in the listing below.
1 Club.action = {|me|
2 me.flipAngleVel.isNaN.not.if({
3 ~rotVel.set(me.id, me.flipAngleVel);
4 }, {
5 ~rotVel.set(me.id, 0);
6 });
7 ~absX.set(me.id, me.pos[0]);
8 ~absY.set(me.id, me.pos[1]);
9 ~height.set(me.id, me.pos.last);
10
11 ~relX.set(me.id, me.posRelHead[0]);
12 ~relY.set(me.id, me.posRelHead[1]);
13
14 ~zeroCrossing.set(me.id, me.zeroCrossing.binaryValue);
15 ~catched.set(me.id, me.catched.binaryValue);
16 ~posRelGPointX.set(me.id, me.posRelGroundPoint[0]);
17 ~posRelGPointY.set(me.id, me.posRelGroundPoint[1]);
18 ~posRelGPointZ.set(me.id, me.posRelGroundPoint[2]);
19 ~posRelHeadX.set(me.id, me.posRelHead[0]);
20 ~posRelHeadY.set(me.id, me.posRelHead[1]);
21 ~posRelHeadZ.set(me.id, me.posRelHead[2]);
22 };
23
24 JugglersHead.action = {|me|
25 ~regionChanged.set(0, me.regionChanged.binaryValue);
26 ~region.set(0, me.region);
27 me.regionChanged.if{
28 q.regionChange(me.region);
29 };
30 ~headAbsX.set(0, me.pos[0]);
31 ~headAbsY.set(0, me.pos[1]);
32 ~headHeight.set(0, me.pos[2]);
33 34 };
35 JugglingInteraction.headClubAction = {|distance, isValid, head, club|
36 ~dist.set(club.id, distance);
37 };
A basic implementation of a JugglingSounds setup can be found on the accompanying DVD.
10.2.3. Conclusion
In this section, I presented SETO, the SuperCollider Environment for Tangible Objects. It can be used to set up complex environments for tangible computing. It is especially suitable for environments where interaction with the code is needed. In this situation, it provides easy access to higher-level functionality. Based on an initial design, SETO was continuously
10. Software and Hardware Frameworks
developed over the implementations of the various Tangible Auditory Interfaces it was used for. Its software design proofed to be flexible, yet efficient for the various applications. In the future, we want to use it for a TAI that incorporates cube-shaped objects with an edge length of60cm. It is developed in co-operation with Animax, Bonn.10
10More information, photos and a video of a proof of concept implementation can be found on http://lfsaw.de/art/installations/SoundBlox.shtml.