WrestleStat Prediction Engine – Results so far

I thought I’d go through the prediction engine to see how it is performing so far this season.

  • Total matches where both participants are on a Division I roster: 2860
  • Matches where the Prediction Engine got the winner correct: 2006 (70%)
  • Matches where the Prediction Engine got the match result correct (regardless of the win/loss prediction): 1419 (49.6%)
  • Matches where the Prediction Engine picked both the winner AND match result correct: 955 (33.4% of all matches, 47.6% of matches where the win prediction was correct)

Obviously after only 2 weekends of wrestling, that’s a very small sample set. But it does give a good idea on the predictability of this process. I envision the results of the prediction engine will only get better as we get further into the season when wrestlers have more matches, thus their rating becomes closer to the reality. This is especially true for true freshman, and wrestlers that haven’t wrestled a college match until this season.

Sometime this season, I’ll try and come up with a more elaborate reporting style for this. I’ll probably add a page to the website where users will be able to look through all of these results.

Update 11/18/2016 @ 8:00AM Central

It was mentioned/suggested from a PSU fan that since wrestlers in their very first season will not have accurate rankings, it would make more sense to exclude those wrestlers from the prediction engine analysis, since we don’t have sufficient data for them.

I re-ran the process to exclude these wrestlers, here are the results:

  • Total matches where both participants are on a Division I roster: 2081
  • Matches where the Prediction Engine got the winner correct: 1581 (76%)
  • Matches where the Prediction Engine got the match result correct (ie picked DEC correctly, or picked FALL correctly): 1076 (51.7%)
  • Matches where the Prediction Engine picked both the winner AND match result correct: 744 (35.8% of all matches, 47.1% of matches where the win prediction was correct)

As expected, when you start removing those wrestlers where we don’t have sufficient data to do the analysis, our accuracy rate goes up. In this case, we removed 779 match results from our analysis, but if we analyze just that set of wrestlers that didn’t have sufficient data, the Prediction Engine only correctly picked the winners at 54.6% (425/779).

Summary

We improved our predictions by 6% after removing those wrestlers, and also improved out match result type prediction by 2%. These statistics will allow me to make adjustments to the Prediction Engine algorithm to hopefully make it more reliable in the future.

5 thoughts on “WrestleStat Prediction Engine – Results so far”

  1. Can you check out the new rankings and the curious case of Caleb Gossett, Northern Illinois at 285
    Last week he was ranked 65 with an Elorank of 1243.3, he loses 2 matches to #47 Bo Spiller and #77 Tyler Kral, as well as a win over an unranked wrestler from Indianapolis (which should not factor into his rank) yet his Elorank increases by 53 points to 1296. My understanding of Elo is that you can’t gain points from any loss. Tyler Kral, conversely, beats 2 higher ranked opponents, Spiller and Gossett and his Elorank only goes up 15 points.

    Spiller goes 1-1 against the other 2 and his ranking goes up 50 points even though he was the highest ranked of the 3 coming in.

    Like

    1. You actually found a different issue. Caleb Gosset’s match against Indianapolis was at 157 pounds. He’s a 285 pounder. He actually didn’t even wrestle in that dual. I’ve updated the dual to have McWhirter in that match instead of Gossett.

      You’re right though, that seems like his rating went up too much. Next weeks’ rankings should reflect the correct change.

      Lastly, WrestleStat DOES factor in matches against non-Division I opponents. For those matches, the non-Division I opponent uses the default ranking when calculating the new ranking.

      Like

    2. Ok, I figured out why he went up so much with only 3 new matches (even though 1 of them was wrong anyway) for this next ranking cycle. All of the results from the Easter Michigan Open were not added until AFTER week 1 rankings. So week 2 rankings had both dual results (actually 3) applied, IN ADDITION TO all of the match results from the Easter Michigan Open.

      Like

  2. Instead of starting all wrestlers with the same default, why not start the top 10 recruits at each weight at a default of say 50 and 6 to 10 at 100 and the rest at the standard default?

    Like

    1. I understand your what you’re saying here, but I’m trying to leave out as much human influence as possible with the rankings process. As soon as I’d make the tweak to apply this, someone else will come down the road saying something like ‘X wrestler was the #1 recruit P4P in X class, so he should start out at X rating’. I don’t want to go down the path of updating individual pre-season rankings to accommodate each wrestler(s).

      It has been calculated, and confirmed, that it takes about 14 matches for a wrestlers ranking to get “evened-out”, so it all works out in the end.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s