Draft Study: Later Round CHL Players

The NHL Entry Draft is an annual event full of hope and potential — an introduction to the next generation of stars and new faces of the league. However, an event filled with such hope and promise cannot escape the inherent failure that surrounds, and ultimately comes to define, its exercise. At its purest distillation, the entry draft is an exercise in futility and as such, becomes a competition to see which team can avoid the trappings of failure with the most aplomb. Every year, there are seven rounds over two days; with each team allotted one pick in each round, that comes to a total of 210 drafted players a year. There are 22 roster spots on each NHL team, leaving 660 spots to be filled. Quite obviously, the attrition rate for the league is not anywhere near 30%, so it stands to reason the majority of draft picks will never see NHL ice.

Modern scouting involves in-person and video evaluation of players all over the world from Kennewick, Washington to Leksand, Sweden. The methods of “bird dogs” are tried and true; scouts watch hours and hours of 17 and 18-year-old kids play games in cold rinks, drinking bad coffee, and driving on bad roads—as all the mythology and archetypes suggest. They may be looking for a small, speedy, sniper like Martin St. Louis, or a big and feared defender like Brooks Orpik. Scouts look at many different “tools” in players and many different “types” of players, of which a proper and ideal variety would comprise a championship-winning team (the end game for every team in the league, hopefully). Scouting requires being able to know what to look for in a player on the ice, off the ice, what his character is like, how he treats his teammates, how he responds to coaching, how big he is (or even how big his parents are), and most importantly how his game will project to the next level.

However, in a business where a success rate of 30% is considered elite, and 50% is unheard of and untouchable, there is clearly room for improvement. One way we can begin to improve our future performance in scouting is to look back at past performance to see where we failed. The next great solutions, or at least a glimpse into them anyways, may lie in historical shortcomings or oversights. 

Looking back and determining where assets (draftees) may have been overvalued in some areas, or undervalued in others, could provide us a glimpse into how we can exploit those aggregated shortcomings for purposes of individual team improvement.

In this “project,” I have attempted to identify common characteristics in eventually successful draft prospects from rounds three to nine (after 2004, only seven rounds were conducted). I have gathered qualitative information (scouting reports), from various websites and sources (primarily Hockey’s Future) to allow myself to see what type of player every prospect was, regardless of their success.

Background
I looked through rounds three through nine for the five draft years from 2003 to 2007. I thought 2007 to be an appropriate cutoff because five drafts ago is a fair amount of time for a prospect to develop. Some may still be developing, but we should be able to have a fairly clear idea idea of the path prospects are progressing on (for the most part). (Anything prior to 2003 was beginning to become difficult to find information for as well.) I'd also like to think scouting trends have changed and advanced a bit in that time. The game changed noticeably with the rule changes in 2004-05, and drafting departments have certainly adjusted for type — I definitely noticed a downtick in the drafting of big, huge coke machines and enforcers post-2004. For my purposes, I didn't collect reports and comments on goalies (since I know nothing about them really), and neglected to go further than a single word if the prospect was a pure enforcer. It is my belief that an enforcer's value to a team or farm system is all but negligible in the long run.

I gathered the draft lists from hockeydb.com and transferred them into spreadsheets by draft year. I further filtered each draft year to rounds 3-9 and then distilled those picks even further to their respective CHL leagues. This made sense to me because an area scout would likely be creating their list and ranking players by region (or league) and thereby comparing them to players within that league, and not other leagues or jurisdictions. Consolidating the list is likely the responsibility of the head scout, or a joint effort involving meetings. So I decided it would be more practical and applicable to look for trends that way.

I individually sought out a brief scouting report or few sentences on every single skater that described their talents and attributes, typically from the website Hockey's Future. I then attached them to the player's name in each spreadsheet under a "comments" column. I also looked up draft year stats for each player to see if they could tell us anything about the player.

Upon completing that, I highlighted the players from each league that ended up being draft successes under two categories — true NHL player and replacement level. Blue highlight represents a "true NHL player" while purple denotes "replacement level." To me, a true NHL player is a player or defenseman that could play on any team in the league (contender or pretender) and still provide value and positive output to a team's success. A replacement level player is a minor league call-up player that you could sign easily in free agency or could go through waivers. As the name suggests, they are replaceable parts, but can still be valuable to your system in order to foster a winning attitude throughout your organizational structure. As the information shows, to even become a replacement level player by way of being a later round draft pick is to beat the odds (and quite significantly may I add).

An example of this from the WHL picks in the 2006 draft is as follows: 

• The entirety of the collected data in the same format for the years 2003-2007 for the WHL, OHL, and QMJHL can be found here. 
• The collected data for the successful "true NHLers" can be found here.
• The data/calculations for the statistics discussed in the ensuing paragraph (and more) can be found here.

Some Raw Statistics, Void of Context

Before we get into the minutiae of individual scouting reports for each player and league, I sorted through some raw data without context to establish any preliminary numbers or trends.

While 5 years is a fairly small sample size, there were still 356 players drafted from the CHL in that span. Of those 356 players, 39 ended up as true NHL players for a success percentage of 10.96%. 33 more ended up as replacement level players, so the success rate for drafting a replacement level player or greater was 20.22%. Those are pretty startlingly low percentages.

If you draft a player from the CHL in rounds 3 to 7 (or 9), there is only roughly a 1 in 10 chance they will become a valuable everyday contributor, and a 1 in 5 chance they will be able to become at least a legitimate roster fill-in. It strikes me that the terminology and mentality of "busting" after you exit the first two rounds needs to be adjusted to better represent these percentages.

Of the three leagues, the OHL had the most players drafted in that span at 132, followed closely by the WHL at 130 with the QMJHL trailing behind at 94. The success ratio includes anything replacement level and above

Here are the averages for the three leagues from '03-'07:

It appears on a year-to-year basis, the OHL seems to be leading the pack in both number of players drafted and amount of successful draft picks, by a small amount over the WHL. Scouting departments seem to be on the right track there. The QMJHL seems to lag behind a bit in both sheer number of players drafted AND the number of NHL players they are producing. Drawing a conclusion strictly anecdotally, this could be due to a variety of factors including a disproportionate population base. The low success rate from an already smaller number of drafted players could also stem from a justification bias from scouts and directors to view all three leagues as equal and thusly trying to draft an equal number of players from each league, perhaps subconsciously. Again, further work and year-by-year statsitical breakdowns for each league can be found here.

Common Themes: Skating, Compete, Motor, and Battle Level

Of course, as always, statistics only tell us so much and are limited in their scope. Scouting is watching games and identifying talent, not looking at spreadsheets and doing calculations. The QMJHL might have the lowest success rates for NHL players and the lowest number of drafted players on a consistent year-to-year basis, but that doesn't mean you should stop scouting there. There could very well be a slightly undersized 17-year-old who loves to shoot and hit playing in Val d'Or on a cold Friday night — and he might might be your next franchise defenseman. The purpose of this exercise is to identify certain characteristics in draft-eligible prospects that can provide you the most value down the road. It's supposed to look at the qualitative, not quantitative. And it's supposed to invite more information into the process, not exclude any possible sources of value.

Upon compiling my results of successes (same spreadsheet as earlier), it seemed to become clear that a lot of late round successes at forward from the CHL share some common attributes. They are typically good skaters with great motors, instant energy, and intense competitors. Typically, these players will also exhibit great work ethic and leadership as well. Prospects that need to have their effort level and energy questioned from night-to-night typically aren't going to make great picks in the later rounds. If you're a top 2 round pick and there's a small chance you can float by on skill, but by the time you reach the later rounds, skill alone isn't enough to overcome the forces working against you.

Players like Cal Clutterbuck, Brad Marchand, Derek Dorsett, Ryan Callahan, Tyler Kennedy, Colin Fraser, Dan Carcillo, Tom Pyatt, Darren Helm, and Brandon Prust are amongst the prospects that comprised the majority of my "successes" at forward. Without spelling it out, if you've watched enough hockey, it becomes pretty apparent when listing those players off what attributes they have in common.

This "type" of player appears to be your best bet when it comes to the later rounds. Prospects exhibiting  said qualities seem to have the character and drive to succeed once they get to the next level - guys that will stop at absolutely nothing to win and never need to be told twice to give 110%. Oftentimes, a later round draft pick will have to battle their way through the lineup in the minor leagues in order to make the professional ranks, and it's not secret coaches love these types of players — likely another factor in their ascension to the NHL. With this type of player having by far the highest success rate, it is tempting to conclude that this type of player may be an undervalued asset in the draft.

15 out of 25 successful forwards from a five year span exhibited these heart-and-soul type qualities combined with good skating and good hockey sense. As mentioned, these players are pretty easy to spot when you see them. Of the remaining ten players, five (Versteeg, MacArthur, D'Agostini, Brent, Brodziak, Perreault) had common buzzwords and phrases that hilghlighted their general battle level and skating abilities. While they didn't necessarily possess the balls-to-the-wall playing style of the other 15, they had the skating and compete qualities to go with a more offensive-oriented style.

Note: Perreault is an interesting case because of his notably deficient size. At a generous 5'9", he was a classic undersized offensive wiz in junior, but the scouting reports still noted he had "terrific/dynamic skills, quick and shifty, great vision and makes decisions quickly." Not too much different than current rookie Brendan Gallagher. Obviously, the size scared some teams off, but if you're able to see these battle and compete attributes in a smaller guy (and he can think the game extremely fast) you may have an undervalued asset and a player on your hands. For a smaller player at the junior level, it's important that they're able to push the pace, think the game at a higher level and avoid contact while not being afraid of it. Smaller players that slow the game down, have deficient skating and are soft perimeter players are a recipe for ECHL careers and Europe.

"Power Forwards" are Iffy
True power forwards seem to be some of the most inefficient picks in the draft. Of all 356 players drafted and 25 successful ones, only two true "power forwards" came out of draft - Troy Brouwer and Dwight King. Brouwer also holds the additional caveat of being an overage pick at 18, a year after his initial year of eligibility. He was only drafted after he demonstrating some offensive upside alongside his power forward attributes. King was also able to demonstrate his offensive upside with 44 points in 62 games in his draft year. He was amongst the only prospects I detailed with consistency questions that ended up making it. However, his scouting report was still favorable in complimenting his physical, gritty style and projectable frame. The rationale is that players like this are extremely difficult to acquire on NHL rosters. Players like Lucic, Simmonds, Downie and Clowe are very valuable in the NHL. There's the additional caveat that these players tends to decelerate even faster than usual as they get older because of their style of play — so how else are you supposed to acquire them?

With that in mind, when looking back, the CHL really doesn't appear the place to be looking for those qualities in the later rounds. If you are to draft "power forwards," it's important they are truly able to demonstrate their puck skills and instincts in their draft year while being able to think the game at speed and bring a consistent effort shift-to-shift. With his draft year numbers, Lucic is an anomaly of anomalies, and likely flashed skills the scouts were able to identify.

The remaining two picks of the 25 were John Mitchell and Matt Beleskey. Mitchell was noted to have good size, skating, possession skills and two-way play of a third line centre, but was dogged by consistency issues and they still follow him to this day. However, he has shown he can provide defensive value for a fourth line in the NHL. Beleskey was hit by the injury bug in his draft year, but was noted to have plus hands and shot with skating being a deficiency. He was the anomaly of all 25 successful players to have their skating noted as a minus and has developed into a competitive and physical forward who will play gritty and bang in rebounds.

Staying Away from Defensive Defensemen

It seemed to strike me in analyzing the results, that when looking for defensemen in later rounds, puck skills, instincts, hockey sense and skating should be valued first and foremost, and picks shouldn't be made purely as "projects." While that sounds like what every team would want to do in theory, it's not what always happens. It does not appear to be worthwhile to gamble on prospects with size and toughness with a lack of offensive upside and questionable skating ability. Valuable defensemen like Kris Letang, Kyle Quincey, Dustin Byfuglien, and Kris Russell were mined for their skills, not their raw physical attributes necessarily. Defensemen exhibiting "toughness" and size without great puckhandling and decision-making abilities rarely, if ever, progress to the highest level.

As you comb through the big spreadsheet, even the replacement-level players have typically demonstrated a certain level of offensive proficiency typically. The players under the "Successes - Dmen" sheet almost all did.

Of the 38 successful picks, 13 were defensemen. May it be a 5'8" offensive dynamo from the back-end like Kris Russell or a 6'2" primarily defensive defenseman like Kyle Quincey, a common theme in almost all of the "successes" for defensemen is skating. Skating is the name of the game (along with hockey sense). Those were two attributes in almost every scouting report for the 13 "successful" defenseman, with the exception of Clayton Stoner. Marc Methot is a bit of an outlier in that he showed good hockey sense, but he is an anomaly amongst hundreds of draft picks as a defensive defenseman in the NHL who demonstrated almost very little offensive ability at any other level.

It strikes me that a lot of "defensive" defenseemen fail once they reach the AHL because if you aren't providing offense as a defenseman, you better be pretty rock solid defensively for your coach to trust you. If the speed isn't there and the ability to process the game at speed isn't there, it's easy to get overwhelmed by the major jump in pace from major junior to pro. It's easy to see the potential in some of these players, but with success rates are so astoundingly low it makes sense to stay away. There is such a small margin of error and emphasis on winning in pro hockey, along with the inherent sense of self preservation that every coach has.

For the most part, the drafting of the huge, physical specimen who can clear the crease and punish forwards physically, but lack offensive upside, saw a reduction in the 2006 and 2007 drafts. However, there were still players selected of that ilk such as Ryan Molle, Maxime Frechette, Dane Crowley and Jordan Bendfeld. Once again, the general thought process of drafting these types of players is understandable — every team wants their next Brooks Orpik, Tim Gleason or Robyn Regehr. With that being said, even if these types of players don't produce a lot offensively at the NHL level, they've typically demonstrated that ability to some extent at a lower level.

Not Ruling Out Overage Defensemen
I also wouldn't hesitate to go for an overage defenseman. Firstly, defensemen take longer to develop in general, and secondly, it's an extremely difficult position to learn. Due to a variety of reasons, some may develop at a different pace than others, but if these prospects are able to show potential, ability and hockey sense, age shouldn't be held against them if they have exhibited accelerated enough development to put them on par with, or above, their age-relative peers (not necessarily draft class, because they will have to transition to the professional ranks sooner). Some examples of this include Chris Campoli, Shane O'Brien, and Andrew MacDonald. Jake Muzzin also re-entered the draft and was signed by the Kings as a free agent after bouncing back from some injury plagued seasons.

Character and Mental Fortitude

"Character" and "leadership" qualities are often words that are tossed around when evaluating prospects or discussing them. This is usually related to being a guy that's "good in the room" and a hard worker without "attitude problems." It brings about images of a guy that stands in front of his teammates down a goal in the second intermission and gives an inspiring speech. This is the kind of guy a lot of people think about when they think of "high character guys."

While all of that is a nice image, I much more prefer to view character as a player's mental strength. Players that have been drafted in any round have often been the best player (or one of them) on their teams for their entire life. From the time they were 6 or 7 until their draft year, they've always been met with success, accolades and praise. To go from the proverbial "king of the castle" to being the "low man on the totem pole" is a major, major adjustment that cannot be understated in the least. It's a blow to anyone's ego - to go from dominating 16 and 17-year-olds to getting knocked around like a rag doll by 25-year-old men who are veterans of the AHL. It's an ugly initial transition that's going to meet the majority of players when they make the jump — it's how a player responds to that and how they approach the situation that will oftentimes determine his fate as a pro.

Of course, as this exercise has shown, a certain baseline of skill is almost absolutely necessarily. And sometimes, a certain complement of skills (read: competitive, versatile) can be more advantageous in ascending the ranks. A player like Daniel Carcillo might be viewed as a bit of a wild card and undisciplined, but his pure competitiveness and stop-at-nothing attitude likely propelled him through the professional ranks. I doubt he took a second to listen to anybody that told him he couldn't make it. A certain cockiness can almost be beneficial at times.

It is to be sure that "character" is an extremely difficult trait to address. Scouts aren't trained psychologists, and even trained psychologists would have a difficult time tapping into the minds of 17-year-old hockey players. It is impossible at times to tell how a young man turning 20 will react to the first true sign of adversity in his young life. Research and going below the surface is paramount and mandatory, but it should be always kept in mind that this entails not only the physical adjustment of playing against men and willingness to train and get better, but also the mental adjustment of occupying a spot on the fourth line or seat in the press box — which can sometimes be far more monumental of a battle than waking up for early morning workouts. You always want nice and respectful kids of course, but there needs to be an intense competitive edge in there somewhere, particularly with later round players. First and second rounders are going to be given opportunity and roster spots by pure virtue of the institutional mechanisms of how any organization is ran. Third, fourth, fifth, sixth, and seventh rounders are the players that will have to claw their way through minor league lineups from the precipices and the margins, and then claw their way through NHL lineups to get their spot, or even just a single chance. You want battlers and competitors — as Mike Babcock says, "everydayers." If you're going to try coast by with your skills or do what you've always done and are unwilling to completely reinvent yourself at every stop, there's a huge world of minor pro hockey or Canadian university hockey to get lost in before the NHL will ever come knocking.

Further Investigation
While I uncovered some interesting information (for myself, anyways), this is only the proverbial "tip of the iceberg." There is a lot more to be mined from historical draft data that can assist the process going forward. I would like to access more in-depth reports for comparisons of failed draft picks. Particularly picks that would have matched my criteria for a draftable player. I've taken the liberty of attempting to read these brief reports and formulate my own "draft lists for each league" and major junior as a whole by year. Obviously, it's much easier to do in hindsight but I tried to stay as true to my findings as possible. I took into consideration their qualitiative traits, and how they lined up with the general trends of my successes, while also factoring in offensive production from draft years. It's pretty rudimentary and spartan in it's process, but it can be found here.

I'd also like to examine the developmental model of the AHL/ECHL and compare it to that of Major League Baseball and perhaps further examine the effects that farm systems can have on player development and what level of resources/attention needs to be paid to those developmental modes.

Similar work and historical draft reviews can and perhaps will be done by me in the future on players drafted from the USNTDP, US high school, US Jr. A, Canadian Jr. A. Through pure inference and speculation, the developmental paths of these prospects could be notably different because of the extra time they spend in college and their relatively increased level of "readiness" when they finally reach pro. It would be interesting to look into this a little more. It's also worth noting that drafting players from longer developmental paths can also have an implication on signings and the 50 contract rule, certainly something to take into consideration.

In staying with that, mining historical draft data and reports from European prospects would also be an extremely interesting project. To see how development of European prospects should be approached as opposed to major junior and college route players is an interesting topic. There is no strict developmental path for prospects; they should each be approached and developed individually as the assets they are in order to ensure their future success, and perhaps more importantly the success of an organization going forward. Investment is necessary, we know that much. Past mistakes and assumptions have proven to be an undoubtedly terrific source of education.

Once again, my raw collection of data and reports from the drafts of 2003-2007 can be found here.