Making Sense Of The NCAA Women’s Hockey Tournament Picture

Boston College Women’s Hockey is out of the running for the Hockey East championship, so that means we’ve got some scoreboard watching to do this weekend with the Eagles squarely on the NCAA tournament bubble. Unfortunately, we don’t have any Pairwise Rankings to look at to see exactly where BC stands, so it’s going to be a lot more stressful than usual.

This year, the committee was given a ton of freedom in how to set the tournament field, since the Pairwise is of limited use with virtually zero non-conference games due to COVID.

There isn’t really any good answer to how to set the tournament field objectively this season, so the NCAA largely just said to the committee “I don’t know; you figure it out.” Which, honestly, isn’t the dumbest idea! The committee does, however, still have the various components of the Pairwise in play as part of their criteria, so that suggests that it will at least come into play a little bit.

One tidbit that’s been suggested from those “in the know” has been that the committee may use the Pairwise to rank teams within conferences, since there’s plenty of intraconference data there where that would be useful — for example, the Pairwise may not be able to objectively determine the difference between BC and Clarkson, but it can tell that BC is ahead of Providence, so it stands to reason that the committee wouldn’t put in Team B over Team A if the two teams are in the same conference and Team A has a higher RPI.

Taking a look at the not-entirely-useful-Pairwise going into the weekend (so, with the conference semifinals remaining for all leagues except Hockey East, which only has its finals left), you can get a good sense at least of where teams within conferences stand:

There honestly isn’t too much confusion about where the tournament contenders stand relative to their conference mates. Hockey East has very clearly been Northeastern, Boston College, Providence, and then the rest for a while now. The WCHA is Wisconsin, Ohio State, Minnesota, Minnesota-Duluth, and then some very bad teams. The four team mini-ECAC, missing all the Ivies and a couple others, Has been a duel between Colgate (the clear #1), and Clarkson, with two other good but not great teams bringing up the rear.

The only debate I’ve heard among these contending teams is actually from some who think Duluth might be more deserving of a spot than Minnesota. But as you can see in the “Pairwise,” it’s not particularly close, and if you want to dive into the WCHA-only KRACH numbers, the conference stratification is even clearer:

Duluth does have a couple very good wins, sure. But they are hampered by an absolutely comical strength of schedule outside of those wins and are the only team in the discussion with an unforgivably bad loss (to Bemidji, one of the worst teams in the country). Also, if you were still on the fence, hello, Minnesota went 2-0-0 against Duluth. The numbers just don’t support Duluth being ahead of Minnesota, and it’s not particularly close.

So if we’ve got teams within conferences pretty neatly sorted out, how to we try and slot the teams into an eight-team bracket? While the committee has the freedom to do what they think is best, there’s a question of scope — how can the you compare teams from conferences that didn’t play each other? The only objective option is to look at results from past seasons to determine the strength of each conference and kind of shimmy the teams around from there. Whether results from past seasons should be a factor is a philosophical discussion for another day, but right now it’s all we’ve got.

And so... exactly that’s what we did. We went back through six seasons of RPI rankings and tried to determine how strong each conference is relative to each other. We then used that data to assign a “Conference Multiplier” which we applied to each team’s RPI rating as it stands right now to see if we could get a somewhat useable Pairwise Rankings using the data from this season.

There’s a whole lot of explanatory information here, so buckle up. Sorry in advance. And please keep in mind that while the numbers are completely objective, this is absolutely not an exact science — averaging six years of conference strength to try and make sense of the current season is like putting a band-aid on an amputee, but it’s better than nothing.

If numbers scare you, you can skip ahead a few paragraphs to the next block quote that says “WELCOME BACK!” if you’re just interested in the good stuff.

How did we do it? We (okay, I, being Grant Salzano) have Women’s Hockey Pairwise Calculators dating back to 2015, which have the composite schedules loaded in for each season. We first took the actual RPI numbers as the conference tournaments ended, and then we re-ran the numbers from that same season using only games between conference mates. This has the end result of creating a Pairwise situation that’s similar to what we have this season. From there, we took the difference in each team’s actual vs. conference-only RPI, and averaged them out between conferences.

To simplify things, this is the goal we’re trying to achieve:

Conference-Only RPI x Conference Multiplier = National RPI

The whole spreadsheet is just ridiculous, and I present it here just so you can see how ridiculous it is:

Anyway, running those numbers gave us some interesting Strength of Conference numbers, which I’m happy to say pass the eye test pretty darn well:

Now, we haven’t gotten to the multiplier part yet, but before we move on, I just want to point out that these numbers are pretty fascinating. First off, as expected, there’s a clear #1 in the WCHA and a clear #4 in the CHA, each of which has held pretty steady in their relative strength over the years.

What’s interesting, though, is that the ECAC and WHEA actually swapped positions as the claim to being the #2 conference after the 2016 season with Hockey East showing clear year-over-year improvement from 2015 to 2019 and the ECAC falling further behind almost every season in that same time frame. That is, of course, until last season, which looks to be quite an anomaly all around (Boston College’s transfers and huge roster turnover say hello).

If we’re going to apply multipliers to this year, don’t we need to make some adjustments for the ECAC only having 4 of its strongest teams playing? We most certainly do. In order to get an apples to apples comparison to give us a multiplier for this season’s RPI, we have to re-run the numbers while treating every season as though only Colgate, Clarkson, St. Lawrence, and Quinnipiac were playing in the ECAC in order to see how strong this “mini-ECAC” is. While we’re at it, we need to remove North Dakota from the mix from the earlier WCHA seasons (since they don’t exist anymore and therefore aren’t playing this year) when we remove the idle ECAC teams.

After re-running those numbers to account for the teams actually playing this year, we can use some fancy math to assign our multipliers with the goal of making every team’s cumulative differences over these six seasons add up to zero. I’ll skip the fancy math, but here’s the final result:

This makes quite a lot of sense. With no bottom-feeders dragging the ECAC down, the mini-ECAC is on average pretty much on par with the WCHA. Keep in mind, there are some brutally bad teams at the bottom out west. Meanwhile, Hockey East still looks pretty average, and the CHA is looking lonely as the perennial weak conference.

Boom. So, based on historical conference strength (using only the four teams playing this year as the historical mini-ECAC), we’ve got our multipliers. The CHA’s RPI on average is about 8% lower than an average conference, the mini-ECAC and WCHA are about 10% and 9% higher, respectively, than an average conference, and Hockey East is... about an average conference.

But before we apply these multipliers to the current RPI, there are a couple adjustments we need to make. Mainly, we need to get rid of the few non-conference games that actually happened this year. Remember, the goal here is to do this:

Conference-Only RPI x Conference Multiplier = National RPI

So if some teams have some non-conference games this year, we have to remove them to get a conference-only RPI. We’ll rectify that by doing what we did with prior year numbers and treat any non-conference games as exhibitions.

Come to find out, the only teams with non-conference games this year are the ECAC teams! The ECAC had several games against very weak teams from the NEWHA conference (a new conference of former D-I and D-II independents), plus a couple games with Syracuse of the relatively weak CHA. Those games, it turns out, did a hell of a job of inflating the entire conference’s RPI since the the rankings treat the CHA and NEWHA as just as good as any other conference. Sneaky, sneaky!

WELCOME BACK!

And now, finally, at long last, we can reveal our finished product, the current 2021 Women’s Pairwise Rankings, if you adjust each conference based on historical strength:

There are a few things of note in here:

1. The ECAC gets simultaneously boosted for being a strong group of teams, and dinged heavily for inflating its RPI with bottom-feeders. It’s a net-negative for the conference, and Clarkson in particular takes a massive hit. They’re below .500 as it is, and while the Golden Knights did play ten games against Colgate, who was quite good this year, removing the conference’s wins over Sacred Heart and LIU hurts Clarkson’s strength of schedule and brings their record to just 6-9-1. The conference boost brings much of that back, but these numbers make it pretty clear that Clarkson should really be the contender that’s on the outside looking in.
2. The weak CHA’s Penn State, who is no doubt having a great year, sit (very satisfyingly!) in 8th place. That definitely feels right. The CHA gets an autobid, but based on how weak it is every year, it rarely has a team in the top 8. Penn State having an extra-good year putting it squarely as the last team deserving to get in is very *chef’s kiss*.
3. Hello WCHA, and hello Duluth. The big, big winner in this exercise is Minnesota-Duluth. Of course, the whole WCHA has to be happy with four of the top five teams in these rankings, including the new #1, but Duluth rockets up from 11th and well outside the bubble to 5th.

Now, that does seem to be a bit too strong for a couple reasons — first off, we’ve already seen what an artificially inflated strength of schedule can do to a team’s ranking this year with the ECAC teams. But, while the WCHA conference schedule was mega-unbalanced for the Bulldogs this year, they were still conference games, so we have to consider them.

But the main reason this seems too strong is that never in the 15 years of the 8-team women’s hockey tournament has the WCHA gotten four teams into the field. The math just makes it very, very difficult to keep four teams above the cut line, no matter how strong the conference is. Nonetheless, this is an objective method of ranking the teams, and using this objective method, Duluth finds itself in the field.

Fortunately, the teams in the running have kind of made it clear who the top eight should be, and it’s hard to quibble with Duluth getting in. The only team with a realistic case would be Clarkson, on the back of it’s solid record against Colgate, but with its overall record well below .500, it’s hard to make that argument.

So with no surprise auto-bids, I would expect the tournament field to consist of Northeastern/Wisconsin at the top, followed by Ohio State in 3rd, and then followed by a huge mess from 4th to 7th involving Colgate, Minnesota, Boston College, and Minnesota-Duluth, followed by the CHA champion (Penn State, with no upsets) in the 8 slot.

That would make the committee’s job pretty easy and just a matter of seeding and setting up a compelling tournament. But if there are any surprise conference champions, expect a big, big fight over which team(s) get left out.