Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Suppose a user wants to transmit both audio and video to another user, whose formats are AMR for audio and H.264 for video. Does the user have to transmit audio and video packets always separately? Meaning that it is not possible to mix audio and video within the same RTP packed, is that correct?

If this is true I guess the RTP protocol will need to know the SSRC of both audio and video to be able to check the sync of the two streams.

share|improve this question

1 Answer 1

up vote 2 down vote accepted

In order to ensure proper playback, the audio and video data MUST always be multiplexed together along with the timing information. This is done by various container formats like mp4 or 3gpp or mov.

In the file formats the audio and video data are partitioned in the form of chunks and their time of playout is marked. This allows players to understand when to display video to screen and put the audio samples to speaker - irrespective of how they arrive. Usually there is sufficient buffer to ensure that sufficient time still exists after all network delays to reach the data to rendering even if audio and video has a different amount of delay while transmission.

If you use container formats such as ones mentioned above, RTP doesn't need to know whether particular packet is Audio or Video.

One more thing - SSRC doesn't really provide any crucial timing information by itself. It is only a label. For example, if i a DVR is receiving data from 16 cameras (and 16 microphones for audio), it needs reference for each such source. This is only an address or identifier not a source of timing information. So if logically, audio and video comes from same source, it can have same tag.

share|improve this answer
    
if they are multiplexed within the same packet, why the need of RTCP for AV sync problems? –  Banana Nov 21 '11 at 2:48
    
there are so many ways to achieve av sync. Indeed, if you are streaming using MP4 - you DO NOT need RTCP for AV sync to be achieved. –  Dipan Mehta Nov 21 '11 at 3:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.