My attention was recently drawn to Paula Weston's CCHA preview this
week on USCHO at
<http://www.uscollegehockey.com/news/ccha/preview.html>, which seems
to imply that the CCHA received unfair treatment in the NCAA seeding
process. Since there are a number of specious or downright misleading
statements both in the article and the quotes by the head coaches of a
couple of CCHA teams, I'm quoting the relevant section here and
challenging the questionable statements as they come up:
> [Last Sunday] the NCAA showed the CCHA what cold is all about.
>
> Seed #5, East and West.
>
> No one around the league is arguing that more than two CCHA teams
> should go to the NCAA's Big Dance. But to have both teams seeded fifth
> seems a bit of a slap in the face.
>
> Not once in the history of the NCAA Tournament has a regular-season or
> conference tournament champion with 26 wins been seeded lower than
> fourth, nor one with 27 wins seeded lower than third. In fact, the
> nearest stat is Bowling Green's third seeding in 1988, when the
> Falcons went 27-11-2.
Total number of wins is a dubious measure even of a team's overall
record, since it doesn't consider the number of games lost and tied.
Presumably it's done here to exclude ECAC and especially Ivy teams
with fewer games on their schedule who were seeded low after winning
the regular season or tournament championship.
> Through Saturday, March 18, Michigan had earned 26 wins and the
> regular-season title; Michigan State had 27 wins and a tournament
> championship to its name.
>
> Maine finished fourth in Hockey East, won the conference championship
> on a questionable goal, earned one win less than did Michigan State,
> and got a first-round bye.
I'm not aware that the NCAA has ever looked at whether a game-winning
goal was "questionable" or not; all that matters is who won the game.
Besides which, Maine's performance over the course of the season would
have warranted the #1 East seed even if they had lost the HE
championship game.
> (For those of you not up to speed, the Spartans posted back-to-back
> shutouts to decisively and without question capture the CCHA
> tournament championship.)
>
> New Hampshire finished second in Hockey East, won neither the
> regular-season championship nor the tourney title, had four fewer wins
> than Michigan State and three fewer than Michigan, and is a third
> seed, facing Niagara.
To tell the full story of each team's won-lost-tied record, Michigan
was 26-9-4 for a winning percentage of .718, Michigan State 26-10-4
(one of their 27 wins was against Division II Alabama-Huntsville,
games against whom are not considered for NCAA selection) or .700,
Maine 26-7-5 or .750, and New Hampshire 23-8-6 or .703. So even
without taking into account the stronger schedules that Maine and UNH
faced as members of Hockey East, Maine had more points in fewer games
than either CCHA team, while UNH's overall winning percentage was
between Michigan's and Michigan State's.
> Even Boston College--with neither a tourney nor regular-season
> title--was seeded higher than both Michigan and Michigan State.
With the exception of the automatic bye for winning both championships
in a conference, the selection committee does not consider where a
team placed in their conference regular season or playoffs in seeding,
only that team's performance based on all their games, conference,
non-conference and playoffs. There's a good reason for this, namely
that the #1 or 2 team in a weaker conference may not be as strong as
the #3 or 4 team in another one, and to focus on what place they each
achieved in their respective conferences punishes teams for playing in
stronger conferences. We in the ECAC have been familiar with this
reality for years.
> This year marks just the fourth year the CCHA tournament champion has
> been seeded lower than third (Bowling Green, 1988, W4; Lake Superior,
> 1995, E5; Michigan, 1999, E5).
>
> Only once before have any conference's regular-season and playoff
> champions both been seeded fifth or lower, in 1998 when Yale was the
> West's fifth seed, and Princeton the West's sixth. Princeton won the
> ECAC tourney championship as the No. 7 seed that year, with a record
> of 18-10-7.
>
> All of this rankles a few people in this here neck of the woods, but
> both Ron Mason and Red Berenson are taking the high road.
>
> "The bottom line is we have to win. It doesn't matter who we play,"
> says Mason. "You know you're going to play good teams. We have won in
> the past. Teams in our league have won on a national level."
>
> Berenson echoes Mason's sentiments, and adds that where the Wolverines
> play isn't an issue.
>
> "It's really not East vs. West when you look at the map," says
> Berenson. "Both venues are equidistant from here, so it's not like
> we're missing a chance to go to Grand Rapids. Geographically, it
> really doesn't matter."
>
> What both coaches will point to as an issue is where the Wolverines
> and Spartans finished in the Pairwise rankings, and how such rankings
> are determined. Neither Mason nor Berenson will point a finger at the
> NCAA per se, but this year the selection criteria are being called
> into question.
>
> "When Niagara and Quinnipiac finish ahead of Michigan and Michigan
> State, something needs to be redefined," says Mason. "There has to be
> something in the numbers that better recognizes strength of schedule,
> that recognizes league strength."
Some of us have been saying for some time that the case of the MAAC
and now CHA points out deeper shortcomings in the selection criteria.
Several criteria do not consider strength of schedule, and others
define it mostly on the basis of the winning percentage of one's
opponents, which can be misleading. The effect is that teams playing
weaker schedules, in particular those playing in weaker conferences,
are overvalued in the NCAA's pairwise comparisons. The NCAA is
adjusting for this ad hoc at the moment by discounting comparisons won
by teams from overly weak conferences, but if the criteria were
adjusted for strength of schedule, it would be fair to return to a
conference-blind analysis.
Here's the catch when applying this to the CCHA: the CCHA is not,
based on their overall performance, a strong conference this year. At
<http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?pairwise$confstr> I
give several tables of measures of conference strength. Head-to-head,
members of the CCHA were 7-10 against Hockey East, 12-15-1 against the
WCHA, 11-10-2 against the ECAC, and 1-2-1 against Division I members
of College Hockey America. The last is deceptive because it includes
only games against Niagara, but by most measures the order of strength
of the conferences is 1.HE 2.WCHA 3.CCHA 4.ECAC 5.CHA 6.MAAC, with the
gaps between 1 and 2 and between 3 and 4 being smaller than the
others. For instance, the KRACH ratings allow us to define the
predicted winning percentage of any team against any other team; if
the Division I members of each conference played one game each against
each of the D1 members of each other conference, the CCHA would
accumulate a winning percentage of .369 against Hockey East, .377
against the WCHA, .526 against the ECAC, .732 against the CHA and .935
against the MAAC.
> Niagara, you may recall, cleaned up in a league with such powerhouses
> as Army and Bemidji State, in a league with just six teams, in a
> league where some conference games are worth four points.
First, Bemidji State is not Division I this year, so Niagara's games
against them were not considered by the selection committee; the same
goes for Alabama-Huntsville and Findlay. The only CHA games which
contributed to Niagara's selection criteria were the four regular
season and one playoff game against Air Force, and the two regular
season games against Army. (These did cause Niagara to have a
somewhat weak schedule, from which they benefitted in the pairwise,
about which more later.) Second, why does it matter that the CHA has
six teams? Only three of them are Division I, which means that in
fact Niagara's schedule was not dominated by conference play the way
for example Quinnipiac's was. And third, the four-point games only
counted double for the CHA standings, which were irrelevant to
tournament selection; for NCAA purposes, they were just ordinary
games, and in fact meant that Niagara's schedule was not padded with
two extra contests against Army. The purpose of this paragraph is
presumably to belittle College Hockey America, but Niagara was not
awarded an at-large berth because they "cleaned up" in the CHA; it was
because of their performance in all their Division I games.
Besides which, neither Michigan nor Michigan State suffered at the
hands of either Quinnipiac or Niagara. While both CCHA teams lost
their pairwise comparisons with QC and NU, the committee recognized
those comparisons as inaccurate by leaving QC out of the tournament
altogether and seeding Niagara sixth in the West despite their
comparison wins over MSU and BC. (The one team with a valid gripe
against Niagara's seeding is the other MSU, Minnesota State-Mankato,
who would have made the tournament in the Purple Eagles' place.
Niagara outperformed Mankato in all of the NCAA's criteria except
head-to-head, but all of those wins, aside from common opponents, are
suspect. For example, Niagara's last 16 games included only two
against teams who ended the season with non-losing records, one of
whom was Canisius, while half of Mankato's last 16 games were against
.500 or better teams. Similarly, Niagara's Ratings Percentage Index
is inflated by five games against Air Force, while Mankato played a
full WCHA schedule.) The point of Mason's statement is presumably to
call into question any and all pairwise comparisons based on the
NCAA's criteria, given that the formula produced anomalous results for
the CHA and MAAC teams. This is a valid point, but again irrelevant
to the seeding of the CCHA teams.
Michigan State were seeded 5th in the West because they lost their
PWCs to New Hampshire and Boston College. Why did they? Well,
looking either at USCHO's file of all the comparisons
<http://www.uscollegehockey.com/rankings/19992000/comparisons.txt> or
the popup breakdowns on my College Hockey Ranking Systems page at
<http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?rankings$pwc> we see
that UNH held the advantage over MSU in the Ratings Percentage Index
.592 to .563 (we've already observed that they accumulated a slightly
lower winning percentage against stronger competition), while
accumulating a better record against Teams Under Consideration (11-6-2
to 8-5-2) and Common Opponents (3-1 to 5-2-2); MSU had a better record
than UNH in their last 16 games (10-3-3 versus 6-5-5), which still
gives UNH the comparison three criteria to one. Looking at the
breakdown of opponents at
<http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?crit.MS> or
<http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?Kcrit.MS>
<http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?crit.NH> or
<http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?Kcrit.NH> we see that
New Hampshire's schedule in the relevant games is at least as strong
as Michigan State's. BC and Michigan State is a little closer, with
the Eagles capturing two criteria (RPI, .584 to .563, and Last 16,
12-4 to 10-3-3) and the Spartans two (vs TUC 8-5-2 to 8-8-1 and Common
Opponents 3-0 to 2-1), but BC taking the comparison due to a higher
RPI. Looking at
<http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?crit.BC> or
<http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?Kcrit.BC> we see that
strength of schedule corrections would if anything tip the balance
further in BC's favor, as the Eagles accumulated their record against
a tougher schedule of TUCs, going 5-6-1 in 12 games against teams who
actually made the tournament and 3-2 in 5 games against other TUCs,
while the Spartans were 1-3-1 in 5 games against other tourney
participants and 7-2-1 in 10 games against other TUCs.
On to Michigan, who received a 5 seed in the East because they lost
comparisons to Boston University and Colgate. The BU-Michigan
comparison sees the Wolverines with the advantage against common
opponents (4-1 to 3-1-1) while the Terriers have a better RPI (.583 to
.568) and record in their last 16 (10-3-3 to 9-3-4) and both teams are
.500 against teams under consideration. Again, looking at strength of
schedule only makes BU's case better. Finally, we come to
Colgate-Michigan, the one case where a CCHA team can argue that the
current criteria unfairly judge them. With Michigan holding the
advantage against Common Opponents (5-1-1 vs 4-3) and Colgate having
the upper hand vs TUCs (11-4 vs 5-5-3) and in the last 16 games
(12-2-2 vs 9-3-4), the comparison comes down to the RPI, in which
Colgate has slight edge, .570 to .568. Since one of Colgate's
opponents was Niagara, who are slightly overvalued as an opponent due
to a high winning percentage and opponents' opponents' winning
percentage, their RPI may be a touch higher than they deserve. At any
rate, the KRACH ratings, which rank Niagara #18 and Quinnipiac #44
without special provisions for their conference but otherwise look
roughly like the RPI, show Michigan slightly better than Colgate,
323.1 to 312.5. So if RPI were replaced with KRACH, Michigan would be
seeded higher than Colgate. However, Michigan is seeded higher than
St. Cloud in the East Regional because they win that pairwise
comparison. Applying the same scrutiny to the Michigan-SCSU
comparison shows that the deciding factor is Michigan's higher RPI;
using KRACH in its place would give the comparison to St. Cloud, so
Michigan would remain a 5 seed, with SCSU and Colgate trading places.
(Intraconference matchups would also come into play, but then if
strength of schedule corrections were made to the pairwise
comparisons, SCSU would likely not have been sent East in the the
first place.)
> "I think our league is pretty strong from top to bottom, and it's big.
> Half the teams in our league--in any league as big as ours--are going
> to have .500 records or below. But those teams are solid."
>
> Berenson says that he's sure the seeding issue "will be debated."
I hope so, but I hope it's done with substantive discussions about the
shortcomings of the system and not just convenient arguments trying to
make a case to improve one's own league's seeds. For example, this
article started off arguing that the top two CCHA teams should not be
seeded below teams finishing third or fourth in Hockey East, and then
went on to blame the Pairwise for ranking teams like Niagara and
Quinnipiac too highly for being from weak conferences. Well, fixing
the flaws in the criteria are only going to improve the standing of
teams from strong conferences, of which the CCHA was not one this
season, which is why MSU and Michigan State were seeded below those
Hockey East teams even using the current system.
> "It's hard to imagine that a team that did not finish in first place
> or took a conference title finished ahead of Michigan and Michigan
> State. The thing that seemed to kill us this year was our Pairwise.
Well, yes, because the pairwise comparisons are based on criteria
which consider performance in all games, not just relative to the rest
of one's conference. Neither the CCHA, nor any other conference, is
entitled to favorable seeds in the tournament if the current season's
results do not warrant it.
> "Our bottom teams were good. We lost games to Bowling Green, Ferris
> State, Alaska-Fairbanks, Omaha--all good teams."
Whoa, hang on a minute. By what measure is Alaska-Fairbanks a "good
team"? They finished 5-25-2, with an RPI of .383 (52nd out of 54
teams) and 43rd in the nation in the KRACH. Michigan Tech was the
only major conference team to finish worse in any of those three
categories.
> Berenson says that he thinks the CCHA itself received little respect
> this season, and he's not sure why. "I don't know if it's a combo of
> the cluster schedule, or how our league did in interleague play, but
> for some reason on paper we came up short.
The CCHA's seeding had nothing to do with respect, and everything to
do with their performance on the ice this season. The numbers on
papers were derived from this season's actual game results. Berenson
is right that interleague play had a lot to do with the CCHA's poor
seeding in the tournament; how else are we to know how good CCHA teams
are relative to teams from the WCHA, ECAC, or any other conference?
> "They [the NCAA] may have to reconsider the formula."
Some of us have already put forth a proposal (before this season
began, in fact): replace RPI with KRACH (which does what RPI tries to
do, only without being tripped up by weak schedules), and modify three
other criteria (record vs Teams Under Consideration, record in the
last 16 and record vs common opponents) to account for strength of
schedule, as judged by the KRACH ratings of the opponents. The
tournament brackets, using these modified pairwise comparisons (which
can be found in their entirety at
<http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?pairwise$kpwc>)
5W Colgate (E) 6E MSU-Mankato (W)
4W St Cloud (W) 3E New Hampshire (H)
1W Wisconsin (W) --+--2E St Lawrence (E)
|
2W North Dakota (W) --+--1E Maine (H)
3W Boston Univ (H) 4E Boston Coll (H)
6W Mich State (C) 5E Michigan (C)
This is a result of giving SLU the automatic bye for winning the ECAC
regular season and tournament championship (UNH would get it based on
the comparisons), but otherwise seeding everything according to the
numbers. If the NCAA made a point of avoiding intraconference games,
the brackets would look like:
5W Mich State (C) 6E MSU-Mankato (W)
4W Boston Univ (H) 3E New Hampshire (H)
1W Wisconsin (W) --+--2E St Lawrence (E)
|
2W North Dakota (W) --+--1E Maine (H)
3W Boston Coll (H) 4E St Cloud (W)
6W Michigan (C) 5E Colgate (E)
Either way, Michigan and Michigan State are seeded even lower than in
the current system, but Niagara and Quinnipiac have been excluded
purely on the basis of the modified criteria. So here is a
redefinition which addresses Mason's concerns: it accounts for
strength of schedule (and thereby also the strength of everyone's
conference schedule), and it does not rank Niagara or Quinnipiac above
Michigan or Michigan State. All it fails to do is give the CCHA teams
better seeds.
> Mason concurs. By way of suggestion, the Spartan head coach says that
> perhaps the NCAA should look at individual league win percentages in
> NCAA postseason play, but in fact, the numbers he seeks for league
> strength vindication are inconclusive.
I've deleted the rest of the article, which goes on to compare these
numbers, because this is such a ludicrous suggestion. First of all,
with an almost complete turnover every four years in college sports,
what do NCAA tournament results from six or eight years ago have to do
with the strength of any team in any conference in the NCAA today?
Second, the makeup of the conferences themselves isn't even the same
as it was in the past, so why should for example Northern Michigan be
judged on the basis of what Minnesota did in the early '90s. Third,
since the tournament went to the single elimination format in 1992,
there have been eleven games in each year's NCAA tournament. That's
88 games in the past eight years. Why should those 88 games be
considered more useful for judging the strength of the various
conferences than the 931 games played between tournament-eligible
programs this year? (If we restrict attention to games involving only
members of established conferences, the number is 744, and even only
looking at interconference games between major conferences, there were
144 played this season.) Finally, given that a higher-seeded team
will typically have some advantage in a tournament game (last line
change, playing closer to home, or even a bye in the previous round),
correlating future seeding with past results could potentially become
a self-fulfilling prophesy. Teams would be seeded poorly because of
their league's past performance, have a disadvantage in the tournament
because of their low seeds, and therefore be more likely to continue
to perform poorly.
I'm very disappointed to see the ill-informed and self-serving
bellyaching contained in the USCHO article coming from a serious
journalist and two Division I head hockey coaches. Typically, this is
the domain of the ignorant fan (for example, those who have
complained, utterly without justification, that the ECAC should have
had a third team in either of the past two tournaments). On the
bright side, we can hope that this draws more attention to the flaws
in the NCAA's current selection criteria and the inadequacy of the
current fix of subjectively overruling some of the comparisons on the
basis of actual or perceived conference strength.
John Whelan, Cornell '91
[log in to unmask]
http://www.amurgsval.org/joe/
Play along at home at http://www.slack.net/~whelan/cgi-bin/tbrw.cgi?tourney
HOCKEY-L is for discussion of college ice hockey; send information to
[log in to unmask], The College Hockey Information List.
|
