In my last post I presented data about the effects of the MAAC's
closed schedule on their ratings percentage index and pairwise
comparisons, which I think make it clear that the committee would be
justified in overruling those comparisons to exclude MAAC teams from
this year's tournament.  I wanted here to make a few points about how
the criteria could be modified to possibly make such ad hoc exclusions
unneccessary.
 
I think the MAAC, playing very few games out of conference and none
against teams from other conferences, has provided a good stress test
for the selection criteria.  In the extreme limiting case, where a
group of teams played a schedule that *only* included each other, no
statistical measure could compare their level of play to that of the
rest of Division I.  And the MAAC example may be close enough to that
limit that stepping outside the rules is required.  However, even in
that case it can illustrate extreme examples of subtle defects in the
criteria which can have a significant but unnoticed effect in less
pathological situations.
 
The MAAC teams do anomolously well in pairwise comparisons due to
three criteria: record in last 16 games, record vs Teams Under
Consideration and RPI, and that effect is amplified by the fact that
they have almost no head-to-head games with other teams under
consideration and very few common opponents.  I would say that RPI
falls into a different category than the other two: it is more robust,
and only fails because the example is so extreme, and is thus fair
under normal circumstances.  (The average RPI of the MAAC teams is
still significantly below that of the four major conferences.)
 
Record in the last 16 games and vs TUC, however, since they use
straight winning percentage, can be seen to be unfair even in the
abstract.  If you play a weak schedule, you may close the season
11-3-2 against weak opponents and be judged better than a team which
was 10-2-4 against a tougher schedule.  Similarly, if your schedule
includes only borderline TUCs, you may be judged better against
tournament-caliber competition than a team which had the misfortune to
face the top teams more frequently.  (Here the MAAC case is also
extreme, since Quinnipiac for example plays no TUCs outside its own
conference.)  I think it would be a good idea for the NCAA to take a
close look at these criteria over the off-season, consider candidate
modifications to them, and see how this season's numbers would be
different if those were made.
 
A possible replacement for the record in the last 16 games would be to
define an RPI-like rating of recent performance which is made up of
35% winning percentage in the last 16 games, 50% opponents' opponents'
winning percentage (in all games, not just the last 16) weighted by
the number of times those opponents were faced in the last 16 games,
and 15% opponents' opponents' winning percentage, similarly defined.
Perhaps the record vs TUC could be replaced with an RPI calculated
entirely with games between TUCs, although the smaller sample size
might cause a problem (in fact, it is conceivable that a team would
play only one other TUC and thus make it impossible to define such an
RPI at all).
 
At any rate, I think the best idea would be to consider a number of
proposals and apply them to this year's test case.  The MAAC's
insularity is likely to improve over the years, but that change may be
slow, and the sooner they can be judged accurately without invoking
conference-specific treatment, the better for the credibility of the
selection process.
                                          John Whelan, Cornell '91
                                                  [log in to unmask]
                                     http://www.amurgsval.org/joe/
 
HOCKEY-L is for discussion of college ice hockey;  send information to
[log in to unmask], The College Hockey Information List.