This is the kind of thing I continue to subscribe to hockey-l for. Keep it
coming. It is always very interesting to see how the rankings compare, and
where they differ.
I'm not sure that the number of correct 'postdictions' is the proper way to
evaluate a rating, since you would expect that lower rated teams will win
games. KRACH accounts for this quantitatively. The others implicitly. Not
that I have a good alternative.
Imagine my happiness (and some surprise) to see Cornell top rated in every
rating. Huge #1 vs. #2 matchup coming up.
Larry
----- Original Message -----
From: "Ken Butler" <[log in to unmask]>
To: <[log in to unmask]>
Sent: Tuesday, December 17, 2002 8:57 PM
Subject: Rating comparison
> There seem to be about five rating systems going around at the moment,
> so I thought it would be interesting to see how (and whether) they agree
> in their rankings of the teams.
>
> I've looked at:
>
> KRACH (calculated by me, or go digging at http://slack.net/hockey)
> CHODR (http://it.stlawu.edu/~chodr/current.html)
> RPI (http://slack.net/~whelan/tbrw/2003/rpi.shtml)
> HEAL (http://maine.edu/heal/heal.latest)
> RHEAL ("iterated HEAL"; http://maine.edu/heal/rheal.latest)
>
> First, a big table of what ranks who where. These are listed by the
> median of the five ranks (I think the median makes a better "consensus"
> because it won't be thrown off by one crazy ranking):
>
>
> KRACH CHODR RPI HEAL RHEAL Median
> Cornell 1 1 1 1 1 1
> Maine 2 3 2 2 7 2
> Colorado College 4 2 8 3 5 4
> Boston College 5 4 3 4 4 4
> Ferris State 8 5 9 6 2 6
> North Dakota 3 17 4 7 6 6
> New Hampshire 6 6 6 5 3 6
> Ohio State 7 7 5 8 9 7
> Michigan 10 9 14 14 10 10
> Boston University 9 18 7 10 12 10
> Minnesota 12 11 12 15 13 12
> Harvard 11 14 11 12 19 12
> St. Cloud State 14 13 10 13 8 13
> Dartmouth 13 26 13 9 11 13
> Miami 16 10 15 11 17 15
> Notre Dame 17 16 16 17 15 16
> Merrimack 20 29 17 18 16 18
> Northern Michigan 22 15 24 16 18 18
> Denver 18 8 22 19 23 19
> Yale 19 21 18 25 31 21
> Alaska-Fairbanks 21 24 20 24 20 21
> MSU-Mankato 24 22 21 21 14 21
> Massachusetts 15 23 19 22 24 22
> Providence 23 20 29 23 21 23
> Western Michigan 25 25 25 39 32 25
> Mass.-Lowell 26 19 23 33 28 26
> Nebraska-Omaha 27 31 27 32 27 27
> Michigan State 29 27 28 29 26 28
> Minnesota-Duluth 28 12 32 30 22 28
> Northeastern 30 33 26 28 33 30
> Brown 31 28 31 31 25 31
> Clarkson 35 32 34 34 30 34
> Michigan Tech 34 30 38 44 34 34
> Wisconsin 32 34 30 43 39 34
> Union 39 35 41 38 36 38
> Rensselaer 36 36 43 41 38 38
> Bowling Green 37 38 39 51 43 39
> Findlay 46 39 37 26 47 39
> Alaska-Anchorage 38 37 40 55 40 40
> Bemidji State 40 46 42 27 29 40
> Colgate 42 47 36 35 41 41
> Quinnipiac 48 41 33 20 49 41
> Vermont 33 45 35 49 42 42
> St. Lawrence 43 40 54 47 37 43
> Wayne State 41 43 48 46 35 43
> Niagara 45 42 44 45 44 44
> Alabama-Huntsville 44 44 46 40 45 44
> Holy Cross 52 51 49 37 48 49
> Air Force 50 50 47 42 50 50
> Princeton 49 48 50 60 54 50
> Sacred Heart 51 52 52 36 46 51
> Lake Superior 47 53 51 57 56 53
> Canisius 53 57 45 50 53 53
> Mercyhurst 54 49 53 54 57 54
> Connecticut 56 55 56 48 51 55
> American Int'l 55 59 55 52 52 55
> Bentley 57 56 57 53 55 56
> Army 58 54 58 56 58 58
> Iona 59 58 59 59 59 59
> Fairfield 60 60 60 58 60 60
>
>
> Where are the big discrepancies? One idea is to see which rankings
> differ wildly from their median. Veterans of ratings discussions on
> Hockey-L will not be surprised by the team on the top of the next list
> (though at least it's not RPI to blame this time :-) ):
>
>
> Team System Rank Median Diff
> Quinnipiac HEAL 20 41 21
> Minnesota-Duluth CHODR 12 28 16
> Sacred Heart HEAL 36 51 15
> Alaska-Anchorage HEAL 55 40 15
> Western Michigan HEAL 39 25 14
> Dartmouth CHODR 26 13 13
> Bemidji State HEAL 27 40 13
> Findlay HEAL 26 39 13
> Bowling Green HEAL 51 39 12
> Holy Cross HEAL 37 49 12
> Merrimack CHODR 29 18 11
> St. Lawrence RPI 54 43 11
> Bemidji State RHEAL 29 40 11
> Denver CHODR 8 19 11
> North Dakota CHODR 17 6 11
> Princeton HEAL 60 50 10
> Michigan Tech HEAL 44 34 10
> Yale RHEAL 31 21 10
> Wisconsin HEAL 43 34 9
> Vermont KRACH 33 42 9
> Boston University CHODR 18 10 8
> Wayne State RHEAL 35 43 8
> Air Force HEAL 42 50 8
> Findlay RHEAL 47 39 8
> Quinnipiac RHEAL 49 41 8
> Quinnipiac RPI 33 41 8
> Canisius RPI 45 53 8
> Findlay KRACH 46 39 7
> Vermont RPI 35 42 7
> Western Michigan RHEAL 32 25 7
>
>
> It's mainly HEAL and CHODR that differ greatly from the consensus; it
> looks as if HEAL likes the MAAC and dislikes the Western conference,
> while CHODR is just different... interestingly, RPI agrees pretty well
> with the consensus.
>
> In the final list, "predicted" is really the wrong word, since we're
> looking at the past. I've taken all the games between
> tournament-eligible teams and thrown out the ties, leaving 411 games.
> Then, for each rating system, I've counted up how many times the
> higher-rated team won. The bigger the better here -- after all, you'd
> want a rating system to have some consistency with the game results!
> I've separated things out by conference:
>
>
> Observed results "predicted" by system and conference
>
> n KRACH CHODR RPI HEAL RHEAL
> cc 66 51 53 51 53 53
> ch 13 9 6 7 8 8
> ec 43 35 33 34 32 30
> he 34 28 24 27 28 28
> ma 46 37 30 37 31 33
> nc 166 140 130 134 127 136
> wc 43 36 33 36 36 36
> all 411 336 309 326 315 324
>
>
> Any half-decent rating system should do a good job within a conference,
> because there are so many conference games (perhaps not yet, but by the
> end of the season). The real test is with the non-conference games. RPI
> is actually doing quite well here; also, RHEAL's improvement over HEAL
> is entirely down to the non-conference games.
>
> (It's perhaps unfair to include CHODR here because this is not what it
> does. CHODR's aim is to predict goalscoring, which it does as accurately
> as possible.)
>
> Anyhow, a little late-night food for thought.
>
> Cheers,
> Ken.
>
>
> --
> Ken Butler
> At home in Canterbury, England
>
|
