Ranking the Rankers: Team Scouts & GMs

    Finding the Best NHL Draft Rankings

    By David Venturi

    “Ranking the Rankers” is a 7-piece series that quantifies past NHL draft rankings from various prognosticators based on player performance to date.

    Part 2: Team Scouts & GMs

    Your favourite hockey team just drafted a teenager. He, presumably, is good at hockey. How can you be sure? Because this set of draft rankings from [insert writer / scout / website here] says so.

    This series attempts to determine which set of draft rankings is best at predicting future NHL success (and which one you should hope your team follows on draft day).

    First up: NHL team scouts and general managers.

    Methodology Review

    Check out Part 1 for a detailed summary of:

    • How player performance is measured (career Point Shares/60 minutes of TOI)
    • How draft rankings are gathered (first round only for 2010-2013)
    • How scout performance is measured (absolute difference between draft ranking and that player’s career PS/60 rank among their draft class peers)
    • Limitations (Point Shares and TOI sample size)

    The actual NHL draft order acts as a proxy for the rankings of NHL team scouts and general managers.

    How to Interpret the Following Charts

    Think of “PS/60 Rank” (read: career Point Shares/60 minutes of playing time rank) as how well the player has performed so far compared to his draft class peers. If the bar is small, they have performed relatively well. If it is large, they have performed relatively poor.

    Think of “Absolute Difference” as that draft rank being a “hit” or a “miss”. If the absolute difference bar is small, that’s a hit. If it’s large, a miss.

    The smaller the total absolute difference for each year, the better that set of draft rankings performed. By comparing each prognosticator’s average performance from 2010-2013 (see Part 7: Conclusion when released), we get an idea of whose NHL draft rankings are best at predicting future NHL success.

    The Results

    Toggle the buttons to switch chart views.

    • Tyler Seguin has lived up to his second overall billing exactly, except he’s second to Vladimir Tarasenko, not Taylor Hall.
    • Erik Gudbranson and Jack Campbell are the notable misses (so far) here. Drafting bruising defensemen and goalies early in the first round is a questionable strategy.
    • John Klingberg, Brendan Gallagher, and Mark Stone are top-10 players on the PS/60 Rank leaderboard for the 2010 NHL Draft.
    • Mark Scheifele is a big win for team scouts and GMs here. Generally regarded as a reach by Jets GM Kevin Chevyldayoff on draft day, Scheifele has performed like a top-10 pick and is on an upward trajectory after torrid second half last year.
    • Duncan Siemens at 11 has not worked out well. The theme for drafting defensive defensemen early continues.
    • The Tyler Biggs pick remains frustrating.
    • Nikita Kucherov, Johnny Gaudreau, and Ondrej Palat, all later round picks, lead the PS/60 Ranks for the 2011 draft class.
    • Alex Galchenyuk, Hampus Lindholm, and Matt Dumba weren’t necessarily off-the-board picks, but they are nice hits here.
    • Everyone missed on Nail Yakupov.
    • The Griffin Reinhart pick appears similar to the Erik Gudbranson situation above, though the Oilers defenseman has more time left to prove his worth.
    • Calder Trophy nominee Shayne Gostisbehere, the underutilized Andreas Athanasiou, and fancy stat darling Colton Parayko top the PS/60 Ranks for the 2012 draft class.
    • Nothing looks terrible here yet, partially because 2013 is where TOI sample size limitations become apparent.
    • I expect the top-six picks to develop into great players if they haven’t already.
    • Oliver Bjorkstrand and Shea Theodore, who lead the PS/60 ranks for this draft crop right now, look like misses for the team scouts and GMs.

    Summary

    Interesting note: the team scouts and GMs failed to win a draft year in terms of lowest “Total Absolute Value of 1st Round Draft Misses”.

    Notice that draft miss scores decrease as we progress to more recent draft years. I suspect this occurs because later-round picks tend to get their chance to establish themselves later than first-round picks. Top-30 talents that don’t live up to the hype then drop in the PS/60 rankings as they are outperformed by the draft steals.

    Part 7 puts the draft miss scores in context so we can compare the talent-identifying abilities of team scouts and GMs to those of Bob McKenzie’s scouts, Corey Pronman, ISS Hockey, and Craig Button.

    Links to each “Ranking the Rankers” piece will be hyperlinked below when complete.