[Delta] Codec negotiation

[Justin Karneges]

On Friday 07 November 2008 13:38:11 Peter Saint-Andre wrote:
> Justin Karneges wrote:
> > On Friday 07 November 2008 08:44:49 Peter Saint-Andre wrote:
> >> The SHOULD is there for consistency with SDP/SIP, but I'll check on that
> >> in the SDP and SIP specs. I'd prefer a MUST here, which would mean "Bob
> >> needs to choose from the payload-types that Alice offers and is not
> >> allowed to counter-offer with other payload-types".
> >
> > Without knowing what to do with these unoffered payload types, XMPP
> > clients are just going to throw them out (that's what I would do..).  We
> > may as well make this a MUST then, right?  Unoffered payload types coming
> > from SIP can just be thrown out by the gateway.
> >
> > If we want the SHOULD, then the XEP should explain what special case this
> > is meant for.
>
> Right. So I'd prefer to make this a MUST, but I would like to
> double-check SIP/SDP usage first just to be 100% sure.

According to Robert, the initiator would always ignore unoffered payload 
types.  The SHOULD is there because SIP clients might respond with a fixed 
codec list regardless of what was offered.  He didn't want to change this 
into a MUST because then an XMPP<->SIP gateway would need to track more state 
in order to filter out the bogus answers from SIP.  He'd prefer to keep the 
gateway dumb, passing the SIP responses back along as-is, and have the XMPP 
client do the filtering (which it would need to do anyway, even if we made 
this a MUST). 

I'm fine with this.  What we need to do now is explain it in the XEP.  Leave 
the bit about how the responder SHOULD return a subset, and add text saying 
that the initiator MUST ignore any in the response that it didn't offer.

> >> As far as I know, the codec negotiation is kind of black magic. For
> >> example Alice might offer Codec X and Codec Y at 320x240 but the exact
> >> resolution might be negotiated via the signalling channel (SIP or XMPP)
> >> for Codec X whereas it might be negotiated via the data channel (RTP)
> >> for Codec Y. And so on. This is an area that I don't know much about
> >> because it's part of the black art of media negotiation. However I could
> >> contact folks who know more than I do for clarification.
> >
> > I don't know if 'negotiation' is the right word here, since RTP is
> > supposed to be one-way (unless somehow you can negotiate codecs with
> > RTCP?).  But I could see a scenario where not all parameters are known at
> > the time of signaling, and so even something as important as resolution
> > wouldn't be known until you started receiving RTP packets.
> >
> > In practice, I'm not sure when resolution wouldn't be known.  I believe
> > Theora requires width and height parameters to be signaled.  Are there
> > any video codecs that can be negotiated via SDP with an undefined
> > resolution?
>
> I don't know. All I do know from people like Rob McQueen at Collabora is
> that codec negotiation is all over the map -- sometimes it happens in
> the signalling channel, sometimes there are these funky things in RTCP,
> etc.

Discussed this also.  It turns out that:
  1) Most (if not all) codecs specify the video resolution in the RTP.
  2) It is perfectly acceptable (probably as a result of #1) to not specify 
the resolution in the signaling.

More generally, any parameters not agreed on in the signaling could turn out 
to be anything in the RTP.  This makes sense.  For example, vorbis has a 
quality level, but you wouldn't necessarily need to agree on this in the 
signaling.  In that case, the sender could push out vorbis RTP at any 
quality.  Strictly-speaking, this is not a negotiation.  The receiver will 
eventually learn the parameters once it starts receiving RTP, but it has no 
say in deciding what they should be.

-Justin