Here is an excerpt from an exchange I had with Ken Butler (quoted with
permission), inspired by the posts from Bob Stagat and Dick Tuthill
last night.  Double indents are me and single indents are Ken; the
punch line is that there is an equivalent definition of KRACH which is
to my mind a little simpler, and that the uniqueness of the ratings is
indeed ensured by the addition of a fictitious reference team against
which each other team has recorded one tie.
 
> > Okay, guys, am I missing something here, or is KRACH a whole lot
> > simpler than Bob's explanation makes it out to be?  My understanding
> > this: for any set of KRACH ratings, there is a predicted winning
> > percentage for each team, given by the weighted average of
> > KRACH[team]/(KRACH[team]+KRACH[opponent]) where the weighting function
> > is proportional to the number of games a team plays against each
> > opponent.  The correct KRACH is defined to be that which makes the
> > predicted winning percentage equal to the actual one for each team.
>
> To my mind, that's perfectly good as a definition of KRACH.
> There are two assumptions underlying it: (a) that it makes sense
> to have ratings on an "odds scale" so that the per-opponent
> predicted winning percentage is as above, (b) you believe it is
> a sensible idea to choose the ratings so as to match the
> observed and expected winning percentages. If you're happy with
> those, you can take them as being the definition of KRACH (apart
> from any discussion about fictitious games).
>
> > I never had to write down a
> > product of probabilities, and was never hampered by the fact that my
> > background in statistics is less deep than either of yours.  I got the
> > impression from Ken that the Maximum Likelihood business was just used
> > to find the solution.
>
> Or to put it another way, if you write down the equations for
> the maximum likelihood ratings, they turn out to be the same
> ones that John solves. The statistical model that leads to the
> likelihood is called the Bradley-Terry model (it has a
> respectable history going back to the 50s), and is based on the
> assumption (a) above: that it makes sense to have a rating scale
> on which odds multiply (for example, if A has odds 2-1 (prob.
> 2/3) of defeating B and B has odds 3-1 (prob. 3/4) of defeating
> C, the model implies that A has odds 6-1 (prob. 6/7) of
> defeating C). This is quite a strong transitivity condition, but
> then since the goal is to produce a ranking of the teams, some
> sort of transitivity condition is needed.
>
> Just to sketch how the likelihood goes together: if teams i and
> j meet n_ij times with team i winning w_ij of the games (count a
> tie as a half-win for each team), then write
>
> p_ij = KRACH[i]/(KRACH[i]+KRACH[j])
> l_ij = (p_ij)^(w_ij) * (1-p_ij)^(n_ij-w_ij)
>
> and then the likelihood is product(i,j) l_ij, with a condition
> like i<j to ensure that each pair of teams gets counted once.
> The likelihood is then maximized in the usual way: take its log,
> then differentiate wrt KRACH[k] for k=1..#teams. This gives the
> set of equations described above. (As a point of technique, it's
> easier to differentiate KRACH[i] and KRACH[j] wrt p_ij rather
> than the other way around, so the tidiest expression of the
> derivatives has a lot of p_ij in it.)
>
> It's a sort-of coincidence that the likelihood equations come
> out this way; if you assume some other way of relating ratings
> to probabilities, you get quite different equations to solve.
> Though you often end up with very similar fitted probabilities,
> especially when they're not far from 1/2.
>
> As a statistician, I believe in maximum likelihood as a general
> principle, and I believe that the Bradley-Terry model underlying
> KRACH is at least not demonstrably wrong, so that's my basis for
> believing the results I get. As I said, though, if you find the
> equation-solving justification convincing, there's no need to
> look any further. (At least unless you want confidence intervals
> for the ratings.)
>
> > This also ties in with Dick's question about the uniqueness of the
> > solution.  According to my definition, there could also be an
> > existance problem.  Ken, have you studied this?  Intuition tells me a
> > unique solution ought to exist as long as no teams have winning
> > percentages of 1.000 or .000 (which the fictitious tie each team is
> > given vs the reference team will ensure), since each team's winning
> > percentage is a monotonic function of its KRACH rating.
>
> There are some conditions related to 1.000 or .000-ness that
> cause the maximum likelihood ratings to shoot up to +infinity or
> down to 0. The most general condition goes like this: if you can
> split the teams into two groups A and B such that every
> game between an A team and a B team was won by the A team, then
> there will be teams with ratings +infinity and/or 0. If there is
> no way to make this split, all the ratings will be finite.
>
> Note that this doesn't necessarily require individual teams to
> be 1.000 or .000. Consider the following results between teams
> U,V,W,X:
>
> U 1 V 1       U 1-0-1 .750
> W 2 X 2       V 1-0-1 .750
> U 5 W 2       W 0-1-1 .250
> V 4 X 1       X 0-1-1 .250
>
> All *looks* well, but if you put A = {U,V} and B = {W,X}, the
> two A-B games both went A's way. The maximum likelihood ratings
> will then be +infinity for U and V, 0 for W and X (assuming
> stuff like 0/(0+0)=1/2). This kind of nasty subtlety is what
> made me put the reference team in; that way, the ratings are
> guaranteed finite for any set of games.
>
> In any case, when the estimated ratings are all finite (either
> by the structure of the data or by adding the fictitious games),
> the maximum likelihood estimates are uniquely determined. John's
> monotonicity argument will prove this, or you can wheel out some
> statistical heavy machinery that asserts that the likelihood has
> only one stationary point, namely its maximum, for any
> exponential-family model, of which this is one. In practice, for
> estimating the ratings in KRACH, any method that keeps going
> uphill will eventually find the maximum likelihood ratings.
 
                                          John Whelan, Cornell '91
                                                  [log in to unmask]
                                     http://www.amurgsval.org/joe/
 
HOCKEY-L is for discussion of college ice hockey;  send information to
[log in to unmask], The College Hockey Information List.