As the "K" of KRACH, I feel inclined to weigh in here...
First off, KRACH, like any other statistical method, is only as good as
its assumptions. One of those assumptions is that the probability of your
team winning increases steadily as you pick its opponents from further
down the list (the particular kind of increase being specified). This
could fail for a game that is (on paper) a mismatch: you could imagine the
stronger team taking it easy (or the weaker team getting "up" for the
game), changing the probability of winning from what KRACH says.
For each team, KRACH works with the winning percentage and the list of
opponents, but doesn't depend on which wins came against which teams. So a
win against Fairfield and a loss against Wisconsin counts the same as a
loss against Fairfield and a win against Wisconsin. This is definitely
*not* the case with HEAL: there, the fact that the win was against
Wisconsin means that the second scenario is much better than the first.
Is this good or bad? It depends on what you're trying to do.
Wayne makes a fair point:
> So, HEAL may yield results different from win% with a balanced schedule.
> IMHO, this attribute of HEAL causes better selection, because higher
> ranked teams generally have had better success against better teams.
> They may have fallen against poor teams, but who cares about that? We
> aren't crowning a league or national champion with the ranking, but
> ordering teams for the best tournament, using results that matter.
This seems like a fair goal for tournament selection: you want the best
teams and the teams that have performed well against the best teams (which
is sort of swallowing its own tail, but there you go). You want teams that
can come to the tournament and compete, not ones that have amassed a good
record by beating up on patsies and losing all their games against quality
opposition.
It may be true that KRACH isn't the right tool for this, in that you may
feel it punishes teams too severely for losing to teams out of tournament
consideration (I'm not sure I agree, but you could say this). I would
argue that HEAL isn't necessarily the right tool either:
> HEAL has measured "how successful were you over teams with good win%",
> more or less.
This may or may not be a measure of "how successful you were over strong
teams", because winning percentage may or may not be a good measure of
team strength. It seems to me that we are using a rating system because we
*don't* want to use winning % as a measure of strength, and I find it
contradictory that this is a rating system that *does* use winning % as a
measure of team strength, albeit at a kind of second level.
This is the cause of the "Quinnipiac effect": all of the teams in the
MAAC have winning percentages that overstate their strengths (on the basis
that the out-of-conference record of these teams is terrible), so that any
schedule that consists mostly of MAAC teams is going to be treated by HEAL
(and by RPI, which uses the same principle) as stronger than it is.
That said, if you stick to one conference, where winning % *is* a
reasonable measure, HEAL and RPI don't have these problems, and HEAL is a
way of up-weighting performance against strong teams.
So how would I pick the teams for a tournament, given that I didn't want
simply to pick the top however-many teams by KRACH (or winning %, if
appropriate)? My suggestion (which I have to admit I haven't tried on any
real numbers) is a good bit more computer-intensive than HEAL (but hey,
computing is cheap, right?):
1. Use KRACH (or your favourite rating system) to rank all the teams.
2. Eliminate the lowest-rated team (and thus all games involving that
team).
3, Repeat 1 and 2 for the remaining teams until there are as many
remaining as you have places in the tournament.
Here's a little conference with 6 teams. Since it's a complete round-robin
schedule, winning % and KRACH give the same answers.
A B C D E F Record
A - W L W W W 4-1
B L - L W W W 3-2
C W W - L L L 2-3
D L L W - W W 3-2
E L L W L - W 2-3
F L L W L L - 1-4
Which two teams do you want in your "tournament"? Looking at the
standings, you'd pick A and either B or D -- probably B, since they won
the head-to-head. But C defeated both A and B. The "eliminate one at a
time" strategy gives this:
Teams under consideration
ABCDEF ABCDE ABCD ABC
A 4-1 3-1 2-1 1-1
B 3-2 2-2 1-2 0-2
C 2-3 2-2 2-1 2-0
D 3-2 2-2 1-2
E 2-3 1-3
F 1-4
eliminating in order F, E, D (because of losing to B), B and leaving a
championship game of A vs. C. Weird? Maybe, but C's losses became less
important as the teams C lost to got eliminated (because we had other
evidence that D, E and F were weak).
I haven't tried this idea on this season's Div I, but I can guess what
will happen: the first few teams eliminated will be from the MAAC and CHA,
which means that the likes of Quinnipiac (and, to a lesser extent,
Niagara) will lose most of their wins, and soon drop out themselves. In
fact, I think this would be true even if you used HEAL or RPI to find the
worst team at each step.
One final point:
> Perhaps a statistician (I'm on thin ice here, folks) would say that a
> loss to a good team is more probable than to a poor team, so the ranking
> should take this into account. I recall explaining this effect on a
> local radio station a couple of years ago. Team A and Team B were close
> in the standings. Team A played a top team and lost. Team B played a
> poor team and won. Team A charged into the (RPI) ranking lead, just
> because they scheduled a top team. The RPI result was poor, IMHO; the
> win% result was poor, IMHO; HEAL more correctly dealt with it.
As would KRACH -- any win is better than a loss, but a win against a poor
team or a loss against a top one is "predictable" and has very little
effect on the ratings. The RPI result was poor IMHO also.
--
Ken Butler, Dept. of Mathematics & Statistics, Dalhousie University
[log in to unmask] / http://www.mscs.dal.ca/~butler
Tants caps, tants barrets.
HOCKEY-L is for discussion of college ice hockey; send information to
[log in to unmask], The College Hockey Information List.
|
