The evaluation of the visualization transfer techniques is based on rapid prototypes, which are described in Section 2.2.1. The results are summarized in Table 4.1 with an overall grading in the right-most column. Two of the techniques resulted in a similar grading and where further evaluation in Section 4.1.2.
4.1.1 Prototype observations and evaluation
Prototype 1: HTML5 video steaming. HTML 5 video is an increasingly often used method to stream video and audio content to pure web browsers and will likely be the future direction of streaming video to web browsers. However, any test to load a real-time generated HTML 5 video stream on a pure web browser resulted in a delayed client side presentation of more than one second. The server side test system (VLC) was extensively adjusted by modifying most parameters in order to optimize the encoding and transfer, which, however, did not result in a faster presentation. It was therefore assumed that the delay is caused by a buffering delay added by the web browser.
This observation was later verified by the work from Metzger et al., who state that "the time scale on which streaming applications buffer content lies in the range of seconds" [254]. They conducted detailed tests about HTML 5 video streaming and show that the delay of a client side video playback increases after network delays occurred. This behavior can be explained by the HTML 5 video specification [85], which recommends a video playback algorithm for web browsers that only starts the playback "when it can be ensured that the video can be played without interruption" [254]. The HTML 5 specification currently has no option to control this behavior.
This means that a short transmission delay that might even occur on a very fast connection may
4.1 Visualization transfer
already result in a buffer increase. Such a buffering behavior is very useful for the continuous presentation of video content without the need of a user intervention. But it follows that it can not be assumed that a web browser continuously presents video frames with low delay and, thus, it currently prevents the usage for a real-time interactive visualization system.
Table 4.1. Comparison of visualization transfer techniques (rows) by the criteria defined in Table 2.1 (columns). Each cell shows the summarized main arguments and a grading of 1 (bad support) to 3 (good support), and "/" (excluded), which is summed up to a total score in the left-most column.
Another disadvantage of HTML 5 streaming is the codec support of the web browsers. No single supported video codec exists that is supported on any browser [221]. Thus, it would be necessary to provide specific streaming methods for different web browsers.
Prototype 2: Pulling JPEG (pJPEG). Pulling single images from the client side is a very straightforward method to transfer the visualization, because it is the default technique to load images of web sites. It results in a fluent frame rate with a subjective low delay on the client side. On the contrary, it requires a communication overhead between the client and the server, because any new image needs to be request separately via a new connection.
Prototype 3: Pushing Base64 images. The continuous pushing of Base64-encoded images resulted in a subjective fluent playback. Testing it on a single computer did not result in a noticeable longer delay than the pJPEG image approach. But Base64 image files are on average 33% larger than its binary version [255], which results theoretically in either a lower frame rate or a higher network load factor. Transferring Base64-encoded images seems therefore less performant than the continuous pulling of single JPEG files.
Prototype 4: Pushing motion JPEG (mJPEG). Pushing mJPEG provided a fluent sequence of images on the client with a subjective very fluent playback. In contrast to the other single image transfer techniques, it utilizes a single persistent connection to transfer all images binary. Each new image is just pushed to the client via the opened connection. Therefore, it does not have the additional communication overhead of prototype 2 and 3 of continuously establishing new connections and/or larger file sizes.
4.1.2 Motion JPEG vs. pulling JPEG
Whereas pulling JPEG (pJPEG) requires a separate establishment of a network connection for each image, motion JPEG (mJPEG) only requires a single connection establishment in the begin-ning of a visualization session. Due to this architectural difference, each technique’s performance capabilities can be easily compared analytically. In a setup with
tinit=time (in milliseconds) required to initialize a new HTTP connection
and the time (in milliseconds) to transfer an image with a file sizes (in MB) on a network with a bandwidthb (in Mbps) of
timg(s, b) = s1000
b 8
,
4.1 Visualization transfer
mJPEG would have a theoretical maximum frame rate of f psmJ P EG= 1000
timg(s, b) and pJPEG a frame rate of
f pspJ P EG= 1000
timg(s, b) +tinit .
The comparison of control event transfer techniques in Section 4.2 provides a good reference value fortinitwith 20 ms (the overhead of REST compared to WebSockets). This is true, because both of REST and pJPEG utilize HTTP GET to load data and both of WebSockets and mJPEG utilize a persistent connection for any transfer. Thus, with the additional tinit = 20ms, pJPEG always results in a lower frame rate than mJPEG. Figure 4.1 illustrates the frame rates of pJPEG (red) and mJPEG (blue) based on these equations with a file size between 1 and 100 KByte on a 3 Mbps (dashed line) and 10 Mbps (solid line) connection. It shows that the difference between mJPEG and pJPEG gets less important with larger files, because of the steady decrease of the tinit/timg ratio. But it also shows that there is a large performance difference when using low image file sizes.
4.1.3 Image format choice
Transferring single images to pure web browsers requires the web browsers to support the image format. Currently, the most widely supported image formats on web browsers are PNG [256] and the JPEG file interchange format (JIF) [257]. JPEG is a lossy image standard (which also exists
motion JPEG on 10 Mbps pulling JPEG on 10 Mbps motion JPEG on 3 Mbps pulling JPEG on 3 Mbps 10 fps
50
40
0 30
20
10
Frame rate in fps
File size in Kilobyte 20
0 40 60 80 100
Figure 4.1. Frame rate comparison of mJPEG and pJPEG.
in a lossless compression version), which means that an image differs on the pixel level after a single compression and decompression cycle, even with the highest possible image quality of 100.
In return, JPEG provides a good overall image quality with a compression ratio of more than 10:1 and is optimized for smooth transitions, as they usually appear in medical visualizations.
In contrast, PNG is a lossless image compression standard, optimized for hard transitions as e.g. line drawings [258]. A more efficient image compression format is JPEG2000 [259], which uses wavelet compression in contrast to the discrete cosine transformation of JPEG, resulting in a better compression ratio [260]. Unfortunately there is almost no web browser available that supports JPEG2000 by default. The same is valid for other newer formats as WebP [261] or H.264 intra frame compression [262].