[Delta] Codec negotiation

Justin Karneges justin-psi2 at affinix.com
Thu Nov 6 14:01:36 PST 2008


I've rethought some of the codec negotiation.  The way it's all supposed to 
work is both parties set abstract modes of what they want to support (e.g., I 
might say I want to support 320x240 Theora and 160x120 Theora), but it's easy 
to run into trouble if the other party didn't pick the exact same thing.  So, 
I'm going to allow for some fuzzy logic.  You can leave some parameters 
unspecified, and if you supply at least two high-low points, then PsiMedia 
will still negotiate something in the middle.  It really needs to work this 
way if we want the process to stay sane and simple.

It could be argued that the user shouldn't have to set anything about 
parameters.  I would agree.  Still, these parameters still have to come from 
somewhere.  I was talking with Maiku of Pidgin the other day, and he wondered 
if leaving the GStreamer resolution unspecified was safe when his camera 
captures at 1600x1200 by default.  Ummm, probably not.. :)  He should do 
something about that.

So, for reasons of bandwith and resource usage, it should be possible to 
provide an upper limit.  I can see it also being useful to provide a lower 
limit as well, if some "no worse than this" quality level is desired.  I 
figure the app could ship with some sane upper/lower limits, and the user 
could muck with them if she cares.

What better way to explain all this than with a nice example?

Alice:
  Camera modes: 320x240 at 30, 640x480 at 30
  Client modes: 160x120 at 15, 320x240 at 15, 640x480 at 15, 800x600 at 30 (insane!)

Bob:
  Camera modes: 1600x1200 at 30
  Client modes: 176x144 at 15, 352x288 at 15

First, both parties need to decide what modes they support on their own.  
Because Alice's camera maxes out at 640x480, there's no sense in supporting 
800x600.  That would be upscaling.  However, even though her camera doesn't 
support 160x120, this can still be a supported transmission mode because we 
can downscale from 320x240.  Thus:

Alice modes: 160x120 at 15, 320x240 at 15, 640x480 at 15
Bob modes: 176x144 at 15, 352x288 at 15

So, over Jingle, Alice sends her modes.  Now, Bob doesn't actually support any 
of those modes explicitly, but if Bob is using a smart client, I'd say he 
should end up selecting 320x240, because that's within his range of desired 
modes:

Alice -> 160x120 at 15, 320x240 at 15, 640x480 at 15
320x240 at 15 <- Bob

It may not always be so easy though.  I could imagine a case like this:

Alice modes: 320x240 at 15, 640x480 at 15
Bob modes: 352x288 at 15

Fail.  The quality is either too good or too bad.

Ultimately the negotiation process depends on a liberal "target".  If Bob only 
supports one mode, then it is hopeless.  There might also be close calls:

Alice modes: 320x240 at 15, 704x576 at 15
Bob modes: 352x288 at 15, 640x480 at 15

Fail!

Interestingly, if Bob initiates to Alice then it succeeds:

Bob modes: 320x240 at 15, 704x576 at 15
Alice modes: 352x288 at 15, 640x480 at 15

(Alice picks 640x480.)

Does anyone have any comment on this selection logic?  I think it all makes 
sense to me, if a little weird at times..

-Justin


More information about the Delta mailing list