For recording and playback, iPhone OS provides two types of audio codecs, i.e., hardware-assisted codecs and software codecs.
The hardware-assisted codecs include the audio formats listed in table4. The advantage
!"#
$%&'()*+,,-&
.(/-0
!"#
12,3-0'-0
!"#
0-.2'-45
!
6%7#8#9#'2#7;-+=-0! !
6%7#8#9#:02.#+;;<"
6%7#>#9#:02.#.()<
6%7#>#:20#32()-6%7#8#:20#.%7() 1+&&6+)=
6%7#>#9#'2#+;;<
1+&&6+)=#?-'7#(,;%'#+%@(2#@+'+#:02.#
*+0@#@(7=#20#:02.#,-'A20=#7'0-+. B-,@#'2#'*-#,-'A20=
6%7#8 6%7#8 6%7#8
Figure 47: Audio processing chain using Audio Unit Processing Graph interfaces of using the hardware-assisted codecs is to save the processing resource on the iPhone (e.g.
battery). Since all the hardware-assisted codecs share a single hardware path, there is only one single instance these three supported formats that can be played at a time. According to [57],"Offline rendering with an audio queue output allows applications to render audio to a buffer instead of directly to an output device". The main interface used for offline rendering (i.e., using the hardware-assisted codecs) isAudioQueueOfflineRender.
AAC
Apple Lossless MP3
Table 4: iPhone OS audio formats supported by hardware-assisted codecs (source: [53])
The supported software codecs are listed in table5. By using the software codecs, one can concurrently play more than one instance of each of these formats.
AMR (Adaptive Multi-Rate, a speech codec)
iLBC (internet Low Bitrate Codec, also a speech codec) IMA/ADPCM (also known as IMA-4)
Linear PCM µLaw and aLaw
Table 5: iPhone OS audio formats supported by software codecs (source: [53])
3 AUDIO PROCESSING ON THE IPHONE AND IPOD TOUCH 68
For voice recording23, iPhone OS contains the supported recording codecs listed in table 6.
Apple Lossless
iLBC (internet Low Bitrate Codec, also a speech codec) IMA/ADPCM (also known as IMA-4)
Linear PCM µLaw and aLaw
Table 6: iPhone OS: recording audio formats (source: [53])
23If using Audio Unit Services to implement a voice recording function, the audio data got from the microphone is Linear PCM. By using the Audio Converter Services, the PCM recorded voice data can be converted into other different compressed formats
4 PAN4i Concepts and Design
4.1 User requirements
The idea to develop the PAN4i application is to create a P2P networking entertainment application in the form of karaoke singing and in combination with group communication on the iPhone and iPod Touch devices. Any iPhone or iPod Touch (called peer or node in the overlay network) having the PAN4i installed and network (W-LAN) connectivity can use this application. Figure48illustrates the use cases of this application. Generally, the application would allow users to self organize (i.e., create or join) an overlay P2P network without any server interaction. Any user can receive (i.e., joining a streaming group) and contribute (i.e., sending multicast) audio streams to a shared, multi-channel application that we coined for karaoke. Users may request music from a distributed jukebox or a local music file and sing karaoke music on an iPod Touch microphone. Simultaneously, any user may stream the live karaoke music to his or her own, personally created channel on the overlay network.
All other users, when subscribing to this channel, will be able to receive and listen to live stream karaoke music. According to the above description, the following user requirements are defined:
!"#$
%&'()*+,*-.,.+/"
'0,".%
!"1")0*.*)2.##"1
!(#3*-.,.+/"
+#*02"*)2.##"1*%&'() !(#3*-.,.+/"*+#
1+).1*%&'()*4(1"
!"1")0*1+).1*%&'()*
4(1"*4+,*'"#$(#3 5&(0
&'",
66"70"#$88 66"70"#$88 66"70"#$88 9('0"#*0+*02"*'"1")0"$*
)2.##"1*%&'()
66"70"#$88 66"70"#$88
Figure 48: Usecase diagram of the PAN4i application
Requirement 1: Get informed about bootstrapping information Precondition: no Postcon-dition: the local peer has information about a nearby peer on an existing overlay or gets informed that there is no existing overlay network Basic actions (dependent on the bootstrap-ping strategy of the system): 1. If using IP Multicast for providing bootstrapbootstrap-ping information
4 PAN4I CONCEPTS AND DESIGN 70
in a local network, then when a node is connected to the local network, it can get this information via the IP Multicast in the local network. The bootstrapping information can also be retrieved from some permanent active peers on the overlay (the software can store a list of these peers). Or it can be retrieved from a server not belonging to the overlay. This server will need to be informed about peers in the group. The bootstrapping peers can be updated with the newest requesting peers.
In the rest of these requirements, the bootstrapping information is considered to be available for any peer that may need in order to carry out other actions in each user requirement.
Requirement 2: Start up a new overlay network Precondition: no
Postcondition: A new overlay network is created with only one peer Basic actions:
1. The user creates a new overlay network
3. The user is acknowledged on success or failure Requirement 3: Join in an existing overlay network
Precondition: There is an existing overlay network, a new peer has to know any existing peer on this overlay network
Postcondition: This new peer is connected to this overlay network Basic actions:
1. The user gets informed of any existing peer on the overlay network 2. The user sends an overlay JOIN request to this existing peer 3. The user request is acknowledged on success or failure Requirement 4: Create a new streaming group
Precondition: The user peer is on the overlay network
Postcondition: A new streaming group is created and available on the overlay Basic actions:
1. The user gives a new group name, and sends a CREATE request to the overlay network 2. The user request is acknowledged on success or failure
Requirement 5: Multicast a music stream from a local file to a group
Precondition: The user peer is on the overlay network, the streaming group exists on the overlay network
Postcondition: no Basic actions:
1. The user gives a group name, and sends MULTICAST messages of the music stream from a local music file to this group
2. On failure, the user is informed that the multicast attempt was fail
Requirement 6: Multicast mixed Karaoke stream of the live singing voice and the back-ground music stream the a local file
Precondition: The user peer is on the overlay network, the streaming group exists on the overlay network (if the iPod Touch is used, an external microphone has to be connected) Postcondition: no
Basic actions:
1. The user gives a group name, and sends MULTICAST messages of the mixed Karaoke stream to this group
2. On failure, the user is informed that the multicast attempt was fail Requirement 7: Join a group to listen to a music or Karaoke stream
Precondition: The user peer is on the overlay network, the streaming group exists on the overlay network
Postcondition: no Basic actions:
1. The user gives a group name, and sends a group JOIN request to the overlay 2. The user listen to the receiving music or Karaoke stream
3. The user is informed if after some time no stream is available or end of the streaming session
Requirement 8: Join a group for receiving a background music stream and sing Karaoke on this music stream
Precondition: The user peer is on the overlay network, both the streaming groups for re-ceiving the background music stream and for sending the Karaoke stream to another group exist on the overlay network (if the iPod Touch is used, an external micro phone has to be connected)
Postcondition: no Basic actions:
1. The user gives a group name, and sends a group JOIN request to the overlay (for receiving the background music stream)
2. The user sings on the receiving background music, and listen to his or her produced Karaoke music and send MULTICAST messages of this Karaoke stream to another group (his or her own Karaoke group)
3. The user is informed if after some time no receiving stream is available or end of the streaming session
Requirement 9: Leave a group
Precondition: The user peer is on the overlay network, the streaming group exists on the
4 PAN4I CONCEPTS AND DESIGN 72
overlay network, the user is a subscriber of this group Postcondition: music playback of the receiving stream stops Basic actions:
1. The user gives a group name, and sends a group LEAVE request to the overlay 2. The user hear no music playback anymore
3. The user request is acknowledged on success or failure
Requirement 10: Quit the PAN4i application gracefully while being as a subscriber of a streaming group
Precondition: The user peer is on the overlay network, the streaming group exists on the overlay network, the user is a subscriber of this group
Postcondition: music playback of the receiving stream stops, PAN4i stops running Basic actions:
1. The user press on the quit button of the iPhone or iPod Touch device
Requirement 11: Quit the PAN4i application unexpectedly while being as a subscriber of a streaming group
Precondition: The user peer is on the overlay network, the streaming group exists on the overlay network, the user is a subscriber of this group
Postcondition: music playback of the receiving stream stops, PAN4i stops running Basic actions:
1. The user does not press the quit button on the device, but the PAN4i application turned off, e.g., device crashes