Gut Feeling or Hard Data?

I recently heard a question posed that really resonated with me: “Are sports organizations irresponsible if they don’t use behavioral data and assessments in the pre-screening and player development process if it’s available?” Which made me look closer at, when should we go with our gut, or when should we use hard data to make organizational decisions? I wanted to share.

Statisticians love numbers, but the mark of a good statistician is one who knows which numbers are important, which ones are relatively meaningless and understands the limitations of formulas and numeric data.

We would all like to have a single formula or a single number that tells us everything we want to know, but particularly when dealing with human beings, there are far too many variables for this to be practical (or accurate).

The “secret” to elite player (or employee) evaluation is taking the numeric data, understanding what it means (and also what it doesn’t mean), and putting that together with critical thinking based on knowledge of the sport and the athletes who participate in it.

Objective (numeric) data alone can be beneficial, but it has limitations. Subjective (based on interpretation and judgment) data alone can be beneficial, but it also has limitations.

The key to consistently making correct decisions is to realize that objective and subjective information complement each other; they are far more powerful used together than they are when used individually.

Examples of mistakes made by using only subjective information are easy to identify, they happen every day in all walks of life. Hiring and other types of corporate decisions are often made in error using reasoning like “I just had a good feeling about him/her.”

The 2014 NFL Draft highlighted an example of this from the sports world. The Cleveland Browns were undecided on which player to target with their first-round draft pick…at that point, they hadn’t even decided which position to target.

Along comes flashy Heisman Trophy-winning quarterback Johnny Manziel, who reportedly sent a text to the Browns during the draft stating “Hurry up and draft me, I want to wreck this league together.”

This clever marketing maneuver by Manziel excited and inspired the Browns to trade up a few spots and choose him with the №22 overall selection in the first round.

We all know now that Johnny Manziel’s NFL career was a short-lived disaster. But, the Cleveland Browns made an emotional and somewhat impulsive decision to draft him, and the franchise paid a dear price for this blunder.

Objective data is many times of great value to decision-makers, but it too can be relied on too heavily.

Back in 2001, basketball data analysis was still in its infancy. One team, the Dallas Mavericks, employed a PhD in Mathematics to supply the team with data to help them make personnel decisions.

The team’s math wiz came up with a very complex formula to rate NBA players, and an interesting result came from this research: an unheralded center named Evan Eschmeyer was one of the very best players in the NBA!

This data was passed on to the team’s owner, with a strong recommendation to sign Eschmeyer as a free agent. The Mavericks did indeed sign Evan Eschmeyer before the 2001–02 season, thinking they had a potential star player based on the advanced mathematical analysis they had at their disposal.

The problem was, Dallas’ PhD mathematician wasn’t a basketball expert; he was relying strictly on his formula and the data that it produced. Eschmeyer played two very forgettable seasons with the Mavericks, averaging 1.6 points and 2.6 rebounds per game and was out of the league after his second season in Dallas.

Incidentally, the Mavericks still had to pay Eschmeyer more than $10 million over the next three years, even though he was no longer playing basketball.

Of course, many great decisions have been made by sports franchises that altered the course of their history. One such move was made in 1998 by the Indianapolis Colts.

The Colts were in the market for a quarterback, and they had the №1 pick in the draft. Their choice came down to Washington State star Ryan Leaf, or Tennessee signal caller Peyton Manning.

Ryan Leaf was a very physically gifted player, but there were concerns about his level of maturity, work ethic and drive. Manning, on the other hand, did not have the physical tools Leaf had, but he was superior in terms of his mental and emotional makeup.

Indianapolis looked at all the variables, both objective and subjective, and came to the conclusion that even though Leaf was the player with more natural talent, Peyton Manning was more likely to become a great quarterback due to his “intangibles.”

Hindsite tells us that Ryan Leaf was one of the biggest draft busts in NFL history, while Peyton Manning went on to a stellar career that will land him in the Pro Football Hall of Fame the moment he is eligible.

So, what’s the answer to our key question of when should we go with our gut, or when should we use hard data to make organizational decisions?

We firmly believe that if you are a sports franchise or sports program decision-maker, you should have every possible weapon at your disposal when it comes to making critical decisions, and behavioral data is a key part of this and it’s available today in both team development and recruiting. If your organization doesn’t use them, I wouldn’t necessarily agree it’s “irresponsible” because there are a lot of coaches out there that have had a lot of success with out them. But, I might challenge these organizational decision makers with the question, are you doing EVERYTHING to be as good as you possibly can be?

If you weren’t sure that was a rhetorical question.