Okay, it probably needs editing, but I've just finished a piece on the
web describing in detail the Bradley-Terry modified pairwise
comparison system (~=KPWR) and the effects it would have had on this
year's selections.  Of interest to the present discussion is the third
part, about impact of the proposed system, which includes a breakdown
of some of the key comparisons that turn out differently using the
actual and proposed criteria, and a heuristic explanation of what
we're saying when we call a particular comparison "flawed".  That can
be found at
 
http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?kpairwise.3
 
The key point with respect to Niagara is the analysis of their
pairwise comparisons with Michigan State and with Minnesota
State-Mankato.  In both cases, there are no head-to-head games, and
Niagara has a higher RPI and a better record vs teams under
consideration, in the last 16 games, and vs common opponents.  In both
cases, the first three criteria are reversed in the modified system;
Niagara's KRACH is lower, and their performace against TUCs and in the
last 16 games is worse, when strength of schedule is taken into
account, but their performance against common opponents is still
better.  Since RPI (KRACH in the modified system) is used as a
tiebreaker when two teams win the same number of criteria, each
comparison would go the other way if Niagara lost the RPI criterion
and one of the other two.
 
If the committee had been following the PWCs strictly, they would have
given Niagara at at-large bid and seeded them above Michigan State.
Since they seeded them below the Spartans (and there was no need to do
so to avoid intraconference games in the regionals), they must have
used the relative strengths of College Hockey America and the CCHA to
overrule the comparison.  On the other hand, choosing Niagara over
Mankato in the first place means that they chose to pay attention to
Niagara's comparison win in spite of CHA's weakness when compared to
the WCHA.
 
Looking at the criteria, rejection of the RPI result alone is not
enough to overturn either comparison; the lack of strength of schedule
considerations in the other criteria must be taken into consideration.
The most obvious problem is with the record in the last 16 games.
Niagara went 12-2-2 in these games, while Michigan State was 10-3-3
and Mankato 10-4-2.  But Niagara's last 16 games contained no
opponents who made the NCAA tournament, and half of them were against
members of the MAAC and other CHA members.  Meanwhile, both MSUs
played schedules loaded with above-average teams in their last 16
games.  Details can be found in the article, but Niagara's win over
both Michigan State and Mankato in this criterion is equally bogus.
 
This brings us to RPI.  RPI is a combination of a team's winning
percentage and a measure of their strength of schedule.
Unfortunately, that strength of schedule is defined by a combination
of the winning percentages of a team's opponents and their opponents'
opponents.  By this measure, Niagara played a stronger schedule than
Michigan State and an equally strong one to Mankato.  But it also
judges Quinnipiac to be an extremely strong opponent, stronger than
Michigan or North Dakota; Air Force's contribution to Niagara's
strength of schedule makes them look tougher than St. Cloud and almost
as tough as Northern Michigan; Canisius is also seen as a decent
opponent, stronger than Minnesota or Colorado College.  So what RPI is
trying to tell us is that Niagara accumulated more or less the same
record as Michigan State against a stronger schedule, and a better
record than Mankato against a similar schedule.
 
But most would disagree with those statements about strength of
schedule.  Take away Niagara's five games against Air Force and you're
left with a schedule with opponents more or less spread out from Holy
Cross to BU.  On the other hand, Mankato played all but four of their
35 games against above-average opponents (those four were against
Michigan Tech, who were one of only two major conference teams to
finish below Quinnipiac in the KRACH) ranging from Minnesota-Duluth
and Denver to North Dakota and Wisconsin.  This sounds like a
significantly tougher schedule by any reasonable measure.  Michigan
State's distribution of opponents is also above-average and thus
stronger than Niagara's; the spread is from Michigan Tech to
Wisconsin, but 27 of the 40 games were against teams with
above-average KRACH ratings.  So under this analysis Michigan State
compiled a comparable record to Niagara's under a tougher schedule,
the opposite of what RPI is telling us, and it's no surprise that they
have a higher KRACH.
 
So we can declare Niagara's wins over Michigan State in both RPI and
last 16 "mistakes" and overturn the comparison.  As for Mankato, they
had a lower winning percentage than Niagara, but contrary to the
flawed RPI calculation played a tougher schedule.  So does their
strength of schedule make up for their worse overall record?
According to KRACH it does; not as easily as in Michigan State's case
of course, but still by a comfortable margin.  So what we have in the
Mankato-Niagara case is a last 16 advantage for Niagara which is
totally bogus (if this is true against Michigan State, it must be
against Mankato), and an RPI advantage which is based in part on the
equally bogus determination that Niagara's schedule was as strong as
Mankato's.  Since Mankato has a lower winning percentage against a
tougher schedule, we can't qualitatively say they must be better (as
in the case of Michigan State), but at least one way of combining
performance and schedule, namely KRACH, says they are.
 
                                          John Whelan, Cornell '91
                                                 [log in to unmask]
                                     http://www.amurgsval.org/joe/
 
HOCKEY-L is for discussion of college ice hockey;  send information to
[log in to unmask], The College Hockey Information List.