[Delta] Codec negotiation

[Pau Garcia i Quiles]

Quoting Justin Karneges <justin-psi2 at affinix.com>:

> I've rethought some of the codec negotiation.  The way it's all supposed to
> work is both parties set abstract modes of what they want to support (e.g., I
> might say I want to support 320x240 Theora and 160x120 Theora), but it's easy
> to run into trouble if the other party didn't pick the exact same thing.  So,
> I'm going to allow for some fuzzy logic.  You can leave some parameters
> unspecified, and if you supply at least two high-low points, then PsiMedia
> will still negotiate something in the middle.  It really needs to work this
> way if we want the process to stay sane and simple.
>
> It could be argued that the user shouldn't have to set anything about
> parameters.  I would agree.  Still, these parameters still have to come from
> somewhere.  I was talking with Maiku of Pidgin the other day, and he wondered
> if leaving the GStreamer resolution unspecified was safe when his camera
> captures at 1600x1200 by default.  Ummm, probably not.. :)  He should do
> something about that.
>
> So, for reasons of bandwith and resource usage, it should be possible to
> provide an upper limit.  I can see it also being useful to provide a lower
> limit as well, if some "no worse than this" quality level is desired.  I
> figure the app could ship with some sane upper/lower limits, and the user
> could muck with them if she cares.
>
> What better way to explain all this than with a nice example?
>
> Alice:
>   Camera modes: 320x240 at 30, 640x480 at 30
>   Client modes: 160x120 at 15, 320x240 at 15, 640x480 at 15, 800x600 at 30 (insane!)
>
> Bob:
>   Camera modes: 1600x1200 at 30
>   Client modes: 176x144 at 15, 352x288 at 15
>
> First, both parties need to decide what modes they support on their own.
> Because Alice's camera maxes out at 640x480, there's no sense in supporting
> 800x600.  That would be upscaling.  However, even though her camera doesn't
> support 160x120, this can still be a supported transmission mode because we
> can downscale from 320x240.  Thus:
>
> Alice modes: 160x120 at 15, 320x240 at 15, 640x480 at 15
> Bob modes: 176x144 at 15, 352x288 at 15
>
> So, over Jingle, Alice sends her modes.  Now, Bob doesn't actually   
> support any
> of those modes explicitly, but if Bob is using a smart client, I'd say he
> should end up selecting 320x240, because that's within his range of desired
> modes:
>
> Alice -> 160x120 at 15, 320x240 at 15, 640x480 at 15
> 320x240 at 15 <- Bob
>
> It may not always be so easy though.  I could imagine a case like this:
>
> Alice modes: 320x240 at 15, 640x480 at 15
> Bob modes: 352x288 at 15
>
> Fail.  The quality is either too good or too bad.
>
> Ultimately the negotiation process depends on a liberal "target".    
> If Bob only
> supports one mode, then it is hopeless.  There might also be close calls:
>
> Alice modes: 320x240 at 15, 704x576 at 15
> Bob modes: 352x288 at 15, 640x480 at 15
>
> Fail!
>
> Interestingly, if Bob initiates to Alice then it succeeds:
>
> Bob modes: 320x240 at 15, 704x576 at 15
> Alice modes: 352x288 at 15, 640x480 at 15
>
> (Alice picks 640x480.)
>
> Does anyone have any comment on this selection logic?  I think it all makes
> sense to me, if a little weird at times..

I am not sure if Iam going to propose a codec-choosing strategy which  
replaces yours or one which complements yours (I think the latter).  
Here it comes:

1. Take the resolutions Alice supports (A1, A2, A3) and the  
resolutions Bob supports (B1, B2)

2. Calculate a very simple "cross-correlation" between each pair of  
resolutions: A1 and B1, A1 and B2, A2 and B1, A2 and B2, and so on.  
The "cross-correlation" algorithm could be as simple as the percentual  
deviation:

  0.5 * ( B1_x / A1_x + B1_y / A1_y ) = 0.5 * 1.742 = 0.871
  0.5 * ( B2_x / A1_x + B2_y / A1_y ) = 0.5 * 3.484 = 1.742
  0.5 * ( B3_x / A1_x + B3_y / A1_y ) = 0.5 * 6.969 = 3.485
  0.5 * ( B1_x / A2_x + B1_y / A2_y ) = 0.5 * 0.87 = 0.43
  0.5 * ( B2_x / A2_x + B2_y / A2_y ) = 0.5 * 1.742 = 0.871
  0.5 * ( B3_x / A2_x + B3_y / A2_y ) = 0.5 * 3.532 = 1.766

3. Take the value closer to 1, i. e. the resolutions which are closer  
in size. In this case, both 160x120 for Alice and 176x144 for Bob, and  
320x240 for Alice and 352x288 for Bob would be equally preferred and  
deciding between those two would be a case of "I prefer a bigger  
camera window" or "I prefer a smaller camera window".

Trivial simplification: discard the product by 0.5 and take the value  
closer to 2 instead of the value closer to 1.

-- 
Pau Garcia i Quiles
http://www.elpauer.org
(Due to my workload, I may need 10 days to answer)