Processing

Functionality such as data queuing, thread instantiation, parallel processing, and resource allocation is provided transparently by that abstract class without developer’s assistance.

Furthermore, if a user drops a compiled node in the application classpath it will be added to the interaction library automatically (see Section 3.1.7 – Expandability / extensibil-ity). This modular layout allows well suited Squidy runtimes as only needed filters are deployed with the runtime environment, e.g. for artistic installations. The base program-ming language of the Squidy Core is Java that provides a platform-independent runtime (see Section 3.1.6 – Multi-platform support).

4.9.2 Processing

Distinct interaction techniques can be implemented by the developer as the dataflow can be intercepted when overriding and implementing individual processing methods (see Sec-tion 3.1.1). These processing methods are subject to a preset sequence (see Figure 4.31).

In the “preProcess” stub (see Listing 4.9.2), the collections of data types grouped within a data container are passed to the method’s implementation. This is an easy way to access all data at a glance or iterate through the data collection manually, e.g. to search for interaction patterns consisting of a diverse set of data types concerning multimodal interaction (see Section 3.1.11 –Multimodal interaction).

/*** Diverse collection of data accessible by this

* method stub before each data object is routed

* to individual data type processing . public*/ IDataContainer

preProcess ( IDataContainer dataContainer );

Whenever it is sufficient to process one particular data instance at a time, the “process”

method stub is appropriate. The following code fragment is a generic representation of such a process method stub. In the case of the “process” stub (see Listing 4.9.2), the Squidy Core iterates through the collection automatically and, therefore, it does not have to be done programmatically as in the “preProcess” stub. Here, DATA TYPE is the placeholder for a generic data type (see Section 4.3.1), offering a simple data-type filter for the dataflow. The Squidy Core only passes instances of that generic type (one at a time) to that method implementation.

Figure 4.31: An flow chart diagram illustrating the processing chain of each node.

/*** Processes data of particular generic data type

* ( DATA_TYPE is a placeholder for those generic

* data types ).

public*/ IData process ( DATA_TYPE data );

Before the data collection is published to the next filter of the processing chain or bridged back to any device or application, the data collection can be accessed through the “post-Process” stub (see Listing 4.9.2). For instance, the “post“post-Process” method is applicable to remove redundant data from the dataflow (e.g. perform an action only once) and to reduce data-processing overhead (see CARE properties [12]).

/*** Diverse collection of data accessible by this

* method stub after each data object has been

* routed to individual data type processing . public*/ IDataContainer

postProcess ( IDataContainer dataContainer );

The Squidy Core uses the Java Reflection mechanism to determine if a filter has imple-mented such a data interception and passes inquired data to the implementation auto-matically. Therefore, no additional effort is required for interface declaration, generation and compilation as is needed for the CIDL used by OIDE [13] or SKEMMI [14,15]. This flexibility of the Squidy Core aims for a rapid integration or modification of filter tech-niques and provides the capability often needed for rapid and iterative prototyping of interactive and natural user interfaces. Heterogeneous devices and toolkits can be easily tied to the Squidy Interaction Library using existing Squidy Bridges [66] (OSC Bridge, Native Interface Bridge) or custom bridge implementations (e.g. to integrate devices or toolkits communicating via special protocols). The Squidy Core provides a multi-threaded environment to perform concurrent data processing and thus increases data throughput, minimizes lag and enhances user’s experience while using customized interaction.

Scott MacKenzie and Colin Ware already determined that delayed system response follow-ing human interaction has an impact on a human’s performance. The delay between input action and output response has been measured in tests using the Fitts’ law paradigm¹⁴. They identified an easy measurable degradation of human performance at 75 ms of delay.

Increasing the lag to 225 ms led to a substantially degradation of human’s performance and thus lead to an unusable system [50]. When taking the 75 ms as an upper limit then at least a minimum of fourteen fps¹⁵ are needed for data throughput. A benchmark of the interaction library has been performed on several architectures that are available on

14A paradigm to test capacity of the human motor system.

15Frames per second indicating the value of frame rates in interactive systems.

today’s consumer market (see Figure 4.32). The benchmark measured the data through-put of a single pipeline by circulating a data object and incrementing a counter each time the object reaches the first node. Every second the counter is read out and reset to zero afterwards. Thus the measured value counts fps. Each architecture passed 98 cycles of 40 seconds each, starting from 3 nodes up to 100 nodes. A single test lasted about 65 min-utes on each architecture. Furthermore, the median was taken to avoid extreme outliers.

Indeed, the diagram contains several outliers that arise from the unbalanced threading strategy of the JVM¹⁶ [28].

Figure 4.32: A benchmark of the Squidy interaction library testing dataflow throughput in fps.

It has been performed from 3 to 100 nodes and takes 40 seconds for each cycle to measure the median fps.

In addition, the diagram illustrates an increased lower boundary of 120 fps (red line) that is reached at a pipeline of 99 adjacent Kalman filters. At the moment the Kalman filter is the most resource consuming filter and in fact is highlighting the feasibility of Squidy’s dataflow paradigm on von Neumann architectures since because most pipelines contain much less than 99 nodes.

The loc¹⁷ statistic complements the performance benchmark of the interaction library Squidy. These loc metrics have been erected with the open source tool CLOC – Count Lines of Code Tool¹⁸ and a Squidy version as of August 23^th 2010 at 2:34pm. Hence,

16Java Virtual Machine

17The acronym loc stands for “lines of code”

18http://cloc.sourceforge.net/

the Squidy framework is modeled in a highly object-oriented manner preventing duplicate code instructions and thus consists of more than 19295 loc. This code provides dataflow and signal processing skeletons and furthermore facilitates a generic user interface facade.

Another 31969 loc integrate diverse input devices and output devices and implementing several filter algorithms (e.g. iPhone, Kalman, AdaptivePointing, and Powerpointer). In addition to the benchmark, the interaction library Squidy was rated among other related frameworks and toolkits as the best applicable design environment for the realization of the Curveproject [87]. TheCurveproject is an interactive desk featuring a curved multi-touch display or more precisely offers a seamless transition between the horizontal multi-touch enabled desktop and the vertical multi-touch wall [85].

The interaction library Squidy is publicly available since September 2009 and can be accessed at http://www.squidy-lib.de. Furthermore, the Figure 4.33 illustrates the down-loads of the Squidy binaries since being open-source, which is approximately 20 downdown-loads per week and a total of 872 downloads (as of August 23^rd 2010).

!

+,-.!* /01.!* 234.!* 5,6.!* 789."! :,;."! <=6."! >-=."! <8?."! 7@9."! 7@A."! >@B."!

!"#$%%"&'(

)"$*+(

<391CAD

>EE@F@A81?4,

Figure 4.33: The download statistics of the Squidy interaction library since September 2009 arising from the statistical analysis of SourceForge.net.

It is published under the LGPLv3¹⁹and used in various scientific, artistic, and commercial installations introduced in Chapter 5 – Use Cases.

19GNU Lesser General Public License - Version 3 – http://www.gnu.org/licenses/lgpl.html