Quidditch by the Numbers

Now that the dust has settled from World Cup, my Anonymous Associate and I thought it would be worthwhile to look through the numbers on game length and scoring data from throughout the season and see how impactful the snitch was.

Our initial feeling was that the snitch was too impactful relative total quaffle points scored. We recently scraped a complete set of games played last season, so we thought we'd put our idea the test.

The Source

Our dataset originally consisted of all official games played in the 2015-16 USQ season (Nationals included). Right off the bat, we threw out games that were invalid (too short, lacking data, forfeits, and obviously mistyped ¹). Since we were mainly concerned with how the snitch affected gameplay, we decided to further reduce our dataset by removing games that exceeded the standard quaffle point differential (> 120 points) because the snitch has basically no impact in those games. Lastly, we wanted to focus on regulation, so we ignored the overtime and double-overtime portions of games. Multiple catches in a single game skewed the data we were really concentrating on: the impact of the snitch on regulation play.

1618 total games
- 22 bad data (clearly incorrect data, forfeits)
- 233 blowouts (> 120 point differential)
- 4 double overtime games
= 1359 games in our sample size

You can see our full source spreadsheet here.

The Problem

There's a sense that the better team doesn't always win in quidditch. That's fine (and exciting), but an ideal game should allow for the better team to use their skill to win. If the snitch was worth 150 points and the game only lasted 2 minutes, it would be the only important mechanic. On the other extreme, you have a 5 point snitch in conjunction with a 50 minute seeker floor, where it wouldn't matter at all. The current iteration of quidditch is somewhere in the middle, but definitely on the "too valuable" side.

The Data & Reasoning

Let's look at some averages: ²

• average game length: 00:22:01
• average quaffle points pre-catch (catching team): 93.1
• % increase in points for a catch: (30 / 93.1): 32.2% increase
• % of catching team's points generated by snitch (30 / 93.1 + 30): 24.4% of final points
• average quaffle points / minute: 7.2 points (a team scores a goal ~every 82.2 seconds)
• average change in differential / minute: 2.6 points ³

The first two are straight averages of final regulation game time and final quaffle points for the catching team, simple. The next two stats are the crux of our issue. When a team catches, they have an average of 93.1 points. A snitch catch then instantly increases their score by a staggering 32.2%, which means the catch is worth 24.4% of the average team's final score. Furthermore, that huge points swing comes instantly.

In modern quidditch, our data shows there's a goal scored every 82 seconds. Theoretically, a team would need to play just over 4 minutes to score the 30 points awarded by the snitch. There's a caveat though- the other team gets the ball after each goal. Even for a dominant team, recovering the quaffle after your goal takes the time to either mount a successful defense or a strip the ball away. In fact, our data shows that quaffle differential changes slowly, at an average of 2.6 points / minute. Given that, most teams need just over 15 minutes to be out of snitch range (if it was going to be anyway). The current 18 minute seeker floor allows the worse team to slowball with a good chance of staying in range.⁴ While this is (and should be!) a valid strategy, the fact that there's such a small margin of error for the leading team makes this too effective a strategy. Teams should be rewarded for masterful execution, not for their ability to play as little quidditch as possible.

There's also a contribution problem. To get the uninterrupted 30 point swing that the snitch grants would take nearly 14 minutes and all 6 members of the team. This falls in stark contrast to the snitch, which is usually caught in about 4 minutes (~18% of gametime), only involves the seeker and beaters, and offers no chance for rebuttal. There's an immediate 30 point swing that happen no matter how close your match was.

The Proposed Solution

The snitch is an exciting and unique part of our sport and we don't want to change that; a snitch is a hell of a lot more exciting than basketball's "who has the ball last". Our main goal is to lessen the impact the snitch has on the game relative to the quaffle (without removing it as an important factor in close games). A good solution should demonstrably decrease the value of the snitch, while also (and more importantly) keeping close games decided by the snitch.

We've got just the right thing.

Longer Seeker Floor

By having more minutes in a game, there's more opportunity for quaffle points to be scored, which lowers the impact of the snitch's 30 points by comparison. With an extra 10 minutes of pure quaffle play, there would be ~7 buckets dropped. If 40 of those 70 points were scored by the catching team (so the average quaffle score of the catching team is now ~120), it would lead to their catch being a 25% increase in points (down from 32.2%) and comprise 20% of their final score (down from 24.4%).

On paper, our solution works great. We know by measuring the "Net Quaffle Points Gained per Minute" (or Net QP / Minute, nqppm). The Net QP / Minute is the amount of points the superior team gains in relation to their opponent, per minute.⁵ Currently, the Net QP / Minute across our entire data set is 2.6nqppm. With an extra 10 minutes of seeker floor, the average game will have its point differential increased by 26 points, or two goals, in favor of the superior team. That may seem like it knocks pretty much every game but the absolute closest out of range, but it doesn’t.

2.6nqppm comes from a dataset including games with differentials up to 120 points, so it includes games that were never in range in the first place; this number doesn’t doesn't represent close games. During a close match, the rate of change for quaffle differential will approach 0 as the teams evenly trade blows. If we narrow our view to games that were in range to begin with (436 games), the Net QP / Minute is 0.8nqppm. Extending the game 10 minutes will only increase the point differential in these close games by 8 points, not even a full goal. Thus, games that were in snitch range will stay in snitch range.

Debunking Counter Arguments

Negative Affect on Tournament Length

This is a logistics issue, not a rules one.

However, it is worth mentioning that once brooms are up, the game keeps a fairly consistent length (seeker floor + ~4 minutes). The largest factor with tournaments getting behind schedule is starting the games late (teams not ready, refs nowhere to be found, etc). By adding 10 minutes to each game and dropping the number of games from 5 to 4, you play more quidditch with less transition time. By having fewer time slots, tournaments may run more on time. Nevertheless, that's not our focus.

Exacerbating Blowouts

While this does have the unfortunate side effect of extending games that are clearly blowouts, that's also not really a rules problem. It's a symptom of poor tournament structures that pit top 20 teams against pot 5 opponents. If there are closer matches scheduled, an extra 10 minutes of game time leads to more minutes of good quidditch. In any case, resisting better game design because there are large skill disparities between teams is a bad approach to game design.

Fewer Upsets & Reduced Variance

Like we said, upsets are fun. If March Madness always went chalk, you'd have a final four with all of the #1 seeds and a bunch of viewers just then tuning in. If a weaker team wants to win, they should do that by playing just as well as the better team, not by slowballing until they can make a play at the single most important part of the game. The extra minutes of game time will encourage team depth and ensure that if the game is in range when the seekers are released, both teams have earned it. Our solution doesn't affect close games; they'll remain close and be decided by the snitch.

Reduced Impact of Seekers

That's the idea.

tl;dr

In its current iteration, the snitch is too powerful of a mechanic based on the number of points its worth and the relatively short length of snitch-less gameplay. The best way to balance the two parts of the game is by increasing the length of the seeker floor by 10 minutes. Based on our data, this brings the value of a snitch catch down significantly, while also keeping the close games close. Good, close quidditch matches will be decided by the snitch, but the snitch will be more in balance with the rest of the game.