[Delta] Codec negotiation
stpeter at stpeter.im
Fri Nov 7 13:38:11 PST 2008
Justin Karneges wrote:
> On Friday 07 November 2008 08:44:49 Peter Saint-Andre wrote:
>> The SHOULD is there for consistency with SDP/SIP, but I'll check on that
>> in the SDP and SIP specs. I'd prefer a MUST here, which would mean "Bob
>> needs to choose from the payload-types that Alice offers and is not
>> allowed to counter-offer with other payload-types".
> Without knowing what to do with these unoffered payload types, XMPP clients
> are just going to throw them out (that's what I would do..). We may as well
> make this a MUST then, right? Unoffered payload types coming from SIP can
> just be thrown out by the gateway.
> If we want the SHOULD, then the XEP should explain what special case this is
> meant for.
Right. So I'd prefer to make this a MUST, but I would like to
double-check SIP/SDP usage first just to be 100% sure.
>> As far as I know, the codec negotiation is kind of black magic. For
>> example Alice might offer Codec X and Codec Y at 320x240 but the exact
>> resolution might be negotiated via the signalling channel (SIP or XMPP)
>> for Codec X whereas it might be negotiated via the data channel (RTP)
>> for Codec Y. And so on. This is an area that I don't know much about
>> because it's part of the black art of media negotiation. However I could
>> contact folks who know more than I do for clarification.
> I don't know if 'negotiation' is the right word here, since RTP is supposed to
> be one-way (unless somehow you can negotiate codecs with RTCP?). But I could
> see a scenario where not all parameters are known at the time of signaling,
> and so even something as important as resolution wouldn't be known until you
> started receiving RTP packets.
> In practice, I'm not sure when resolution wouldn't be known. I believe Theora
> requires width and height parameters to be signaled. Are there any video
> codecs that can be negotiated via SDP with an undefined resolution?
I don't know. All I do know from people like Rob McQueen at Collabora is
that codec negotiation is all over the map -- sometimes it happens in
the signalling channel, sometimes there are these funky things in RTCP, etc.
> I suppose it may also be possible to lie about resolution, by reporting one
> value in the signaling and doing something completely different in the RTP.
> Or perhaps initially the resolution in the RTP matches the signaling, but is
> later changed to something else, but only in the RTP. Then it's not quite
> a "lie" but more like a change of mind..
Right. I'd prefer to force explicit negotiation up front and stick to
that, but that may not be doable in the real world.
>>> It may be possible for each party to use a different resolution, if they
>>> both agree on it. Like, if Alice offers both 320x240 and 352x288, I
>>> think it might be possible that Bob could accept both types. Then either
>>> side could choose what to transmit with.
>> My understanding is that in SIP/SDP Bob can use anything that Alice
>> offers (e.g., they could switch to a new codec halfway through the
>> session if desired), but that's more of the black magic in play. It
>> seems clearer to me if the parties agree explicitly on the codecs and
>> their parameters in the signalling channel and then if they want to
>> change things they would modify the content definition. This stuff about
>> implicitly allowing changes mid-stream feels dangerous to me. But as I
>> said I need to check on this with people who know more than I do about
>> voice and video.
> Yes, my understanding is that Alice and Bob agree on the same payload-info
> set, and if they agree to more than one payload-info of the same media type
> (e.g. three video payload-infos), then either party can transmit using any of
> the agreed upon payload-infos. If there is a desire to transmit anything
> else, then they'd need to use content-add.
> This does mean that Alice could transmit using one video payload for awhile
> and then later switch to another, without having to do any signaling. I can
> see changing the payload being useful if, say, the network gets worse and you
> want to downgrade quality. It seems like you could do just as well by
> agreeing on only one payload per media type at a time, and then use signaling
> to change the one payload to something else, but the nature of SIP/SDP
> systems seems to be that you should be able to handle multiple at once.
Right, that's my understanding too -- you don't want to wait for
signalling to complete before dynamically downgrading or upgrading
codecs to respond to changing network conditions.
More information about the Delta