[Delta] Codec negotiation
Justin Karneges
justin-psi2 at affinix.com
Thu Nov 6 14:01:36 PST 2008
I've rethought some of the codec negotiation. The way it's all supposed to
work is both parties set abstract modes of what they want to support (e.g., I
might say I want to support 320x240 Theora and 160x120 Theora), but it's easy
to run into trouble if the other party didn't pick the exact same thing. So,
I'm going to allow for some fuzzy logic. You can leave some parameters
unspecified, and if you supply at least two high-low points, then PsiMedia
will still negotiate something in the middle. It really needs to work this
way if we want the process to stay sane and simple.
It could be argued that the user shouldn't have to set anything about
parameters. I would agree. Still, these parameters still have to come from
somewhere. I was talking with Maiku of Pidgin the other day, and he wondered
if leaving the GStreamer resolution unspecified was safe when his camera
captures at 1600x1200 by default. Ummm, probably not.. :) He should do
something about that.
So, for reasons of bandwith and resource usage, it should be possible to
provide an upper limit. I can see it also being useful to provide a lower
limit as well, if some "no worse than this" quality level is desired. I
figure the app could ship with some sane upper/lower limits, and the user
could muck with them if she cares.
What better way to explain all this than with a nice example?
Alice:
Camera modes: 320x240 at 30, 640x480 at 30
Client modes: 160x120 at 15, 320x240 at 15, 640x480 at 15, 800x600 at 30 (insane!)
Bob:
Camera modes: 1600x1200 at 30
Client modes: 176x144 at 15, 352x288 at 15
First, both parties need to decide what modes they support on their own.
Because Alice's camera maxes out at 640x480, there's no sense in supporting
800x600. That would be upscaling. However, even though her camera doesn't
support 160x120, this can still be a supported transmission mode because we
can downscale from 320x240. Thus:
Alice modes: 160x120 at 15, 320x240 at 15, 640x480 at 15
Bob modes: 176x144 at 15, 352x288 at 15
So, over Jingle, Alice sends her modes. Now, Bob doesn't actually support any
of those modes explicitly, but if Bob is using a smart client, I'd say he
should end up selecting 320x240, because that's within his range of desired
modes:
Alice -> 160x120 at 15, 320x240 at 15, 640x480 at 15
320x240 at 15 <- Bob
It may not always be so easy though. I could imagine a case like this:
Alice modes: 320x240 at 15, 640x480 at 15
Bob modes: 352x288 at 15
Fail. The quality is either too good or too bad.
Ultimately the negotiation process depends on a liberal "target". If Bob only
supports one mode, then it is hopeless. There might also be close calls:
Alice modes: 320x240 at 15, 704x576 at 15
Bob modes: 352x288 at 15, 640x480 at 15
Fail!
Interestingly, if Bob initiates to Alice then it succeeds:
Bob modes: 320x240 at 15, 704x576 at 15
Alice modes: 352x288 at 15, 640x480 at 15
(Alice picks 640x480.)
Does anyone have any comment on this selection logic? I think it all makes
sense to me, if a little weird at times..
-Justin
More information about the Delta
mailing list