Greetings!
 
This is an old argument.  Can a computer ranking be trusted and are its
estimates valid.  I think that it's clear that RPI won't work if there
is a subset of teams that have played almost no games against everyone
else. I am not sure that KRACH is robust either, but I do think that
KRACH should be considered ahead of RPI for several reasons.
 
1)  KRACH has an established mathematical/statistical history.  The
ranking method is actually due to Zermelo(1929), who was trying rate
chess players in an incomplete tournament.  It was later invented
independently by Bradley and Terry(1952) and is still known in the
biostatistics literature as the Bradly-Terry method.
 
2)  KRACH has a natural property that I think are desirable:   If the
teams play a complete round robin schedule, then the KRACH ranking of
the teams will be the same as ranking by wins.
 
3)  I also recall that the Bradley-Terry method satisifies some
important optimality conditions (of the type statisticians like).
Unfortunatley, it's been long enough since I looked at this stuff that I
don't remember how to state it precisely.  I believe that if one assumes
a linear model of pairwise comparisons that will have property 2), then
the Bradley-Terry method is the maximally efficient consistent estimator
of the probability that A will defeat B. NOTE: A linear model means that
each team has a rating and that the probability that A will defeat B can
be derived from the difference of their rating (For KRACH, you would
take the LOG of the ratings before subtracting).  I will look up some of
my references to see if I mistated this.
 
In any event, I have long asserted that one cannot have the perfect
computer ranking. See for example my article published in the UMAP
Jounral (1998) (OK, so I plugged my own work, but I did make several
references to College Hockey).
 
 
Go GOPHERS!
 
Gary Hatfield
SKI-U-MAH!
 
Citations:
 
1929: Zermelo,E. Die Berechnung der Turnier-Ergebnisse als ein
Maximumproblem der
Wahrscheinlichkeitsrechung.  Mathematische Zeitschrift Vol 29: 436-463.
 
1952:  Bradley, R.A. and Terry, M.E. Rank Analysis of Imcomplete Block
Designs I: the method of paired comparisons.  Biometrika, Vol 39:
324-345.
 
1998:  Hatfield, G.A.  Sports Ranking Functions.  The UMAP Journal Vol
19 no. 2. :117-126.
 
HOCKEY-L is for discussion of college ice hockey;  send information to
[log in to unmask], The College Hockey Information List.