2024 Analysis Of Expert Rankings

Fantasy Football is a $15 billion-per-year industry. With so much money at stake, it’s no surprise that hundreds of analysts earn a living predicting who to draft. Platform sites like ESPN and Yahoo aggregate expert rankings to create composite rankings. FantasyPros takes this idea a step further by combining rankings from over 100 experts to create consensus rankings.

Last year, I argued that the average of many analysts is more reliable than any individual. I even pitched this concept to Rotoballer, advocating for a true consensus ranking built from all our analysts’ inputs.

Using a Teams In A Vacuum approach, I gathered the early September 2024 rankings and compared them to the actual outcomes of the season. The goal is to discover who achieved the best rankings in 2024.

Table 1 – 2024 Point-Per-Reception Wide Receiver Rankings

The left and center columns of Table 1 display rankings with the same names, just in a different order. The right columns demonstrate that order matters! Ja’Marr Chase was in a tier of his own, followed by Justin Jefferson and Amon-Ra St. Brown in a second tier. If you didn’t pick one of these three receivers, your receivers were “contributors” instead of “stars.”

Ranking The Experts

Since order matters, the best way to evaluate rankings is by measuring how closely they align with the actual results. We can achieve this by comparing each player’s predicted rank to their finish. The difference between these two positions yields a score — the smaller the difference, the better the ranking. By summing these differences across all players, you calculate a metric for assessing ranking performance.

Absolute Value(player ranking – player finish)

For example, if a ranking places Ja’Marr Chase at number 1 and he finishes number 1, it results in a perfect score – zero difference. However, if Amon-Ra St. Brown is ranked at number 8 and he finishes number 3, that’s a five-point miss. Tally these errors across the board and average them to determine the best rankings.

Table 2 – 2024 Average Player Position Differences For Quarterbacks, Running Backs, and Wide Receivers

In Fantasy Football, running backs and wide receivers are typically drafted first. Although quarterbacks score the most points, they also accumulate a similar number of points. Consequently, each team selects a combination of two running backs, two wide receivers, and a quarterback for their first five picks. In a 12-team league, this accounts for the first 72 players in the draft. After the fifth round, the drafts often become chaotic, with tight ends, defenses, and long-shot sleepers mixed in. This analysis is limited to the first 72 positions in each ranking.

Table 2 reveals that wide receivers are the most challenging position to predict. On average, analysts missed the mark by 13 spots per player. Running backs were the next toughest, while quarterbacks were the easiest to rank accurately.

As you can see from Table 2, most of the rankings are equally “off.” The span of values for quarterbacks, running backs, and wide receivers is 1, 2, and 3, respectively. This implies that you should use the rankings of the expert who predicts the most accurate wide receiver order.

Table 3 – Rank Of The Rankers – combined average of the positions

Table 3 outlines the accuracy of each expert’s rankings for the 2024 season, revealing several notable patterns:

1. Winner: Dave Richard (CBS Sports) – It’s not a coincidence that the most accurate wide receiver ranker was also the most accurate overall.
2. Don’t take the easy way out by relying on the drafting platform rankings — ESPN, NFL, Underdog, and Yahoo rankings didn’t perform well. ESPN and Yahoo account for 22 million out of 30 million participants, but this analysis suggests using a different set of rankings for your first five picks. CBS Consensus Rankings were the best consensus rankings in 2024, so they should be regarded for your first five picks in 2025.
3. Average Draft Position (ADP) was disappointing – ADP reflects the collective behavior of millions of drafters. However, the poor performance of wide receivers dragged down the overall ratings. 
4. Fame does not equal accuracy – the most popular names didn’t deliver. Matthew Berry and Andy Behrens, two of the most recognizable voices in fantasy football, ranked near the bottom. 

Experts who excel at ranking wide receivers tend to perform the best overall—a trend highlighted by 2024’s top performer, Dave Richard. Conversely, default rankings from major platforms (ESPN, NFL, Underdog, Yahoo) and Average Draft Position, which reflect the collective decisions of millions of drafters, have not performed well. Therefore, do not rely on the drafting platform’s rankings for your crucial first five picks.

I consider this analysis a work in progress. If you have 2024 rankings you’d like included in the analysis, feel free to reach out on X (formerly Twitter): @TiavGuy.