[Delta] Codec negotiation

[Peter Saint-Andre]

Justin Karneges wrote:
> On Friday 07 November 2008 01:47:43 Pau Garcia i Quiles wrote:
>> Quoting Justin Karneges <justin-psi2 at affinix.com>:
>>> Since
>>> Alice did not offer 352x288, it is an unusable mode I think.  At least,
>>> this is the case in Jingle.
>> Does Jingle require that both participants use the same camera
>> resolution? That does not make sense to me!
> 
> Just to be clear: they wouldn't have to use the same camera resolution, but 
> the same transmission resolution.
> 
> Alice captures at 320x240, transmits 320x240.
> Bob captures 352x288, downscales to 320x240, transmits 320x240.
> 
> I believe Jingle requires that they agree on the same resolution.  Bob must 
> pick from the <payload-info> elements Alice offers.  If she doesn't offer 
> 352x288, then he can't pick it.  Well actually it says he SHOULD only return 
> elements she offered, but I'm not sure what that means..

The SHOULD is there for consistency with SDP/SIP, but I'll check on that
in the SDP and SIP specs. I'd prefer a MUST here, which would mean "Bob
needs to choose from the payload-types that Alice offers and is not
allowed to counter-offer with other payload-types".

As far as I know, the codec negotiation is kind of black magic. For
example Alice might offer Codec X and Codec Y at 320x240 but the exact
resolution might be negotiated via the signalling channel (SIP or XMPP)
for Codec X whereas it might be negotiated via the data channel (RTP)
for Codec Y. And so on. This is an area that I don't know much about
because it's part of the black art of media negotiation. However I could
contact folks who know more than I do for clarification.

> It may be possible for each party to use a different resolution, if they both 
> agree on it.  Like, if Alice offers both 320x240 and 352x288, I think it 
> might be possible that Bob could accept both types.  Then either side could 
> choose what to transmit with.

My understanding is that in SIP/SDP Bob can use anything that Alice
offers (e.g., they could switch to a new codec halfway through the
session if desired), but that's more of the black magic in play. It
seems clearer to me if the parties agree explicitly on the codecs and
their parameters in the signalling channel and then if they want to
change things they would modify the content definition. This stuff about
implicitly allowing changes mid-stream feels dangerous to me. But as I
said I need to check on this with people who know more than I do about
voice and video.

> Hmmm, maybe Peter can step in here and clarify the Jingly bits.

Gosh I get all Jingly just thinking about voice and video support in
Psi. ;-)

/psa