Too good to be true? Yes, according to Forbes. Turns out, data scientists spend most of their time (up to 79%!) on the part of their job they hate most.

The Demand for Data Scientists:

Thousands of companies across a myriad of industries are hiring data scientists, likened to the “quants” of Wall Street in the 1980s and 1990s for their exclusive abilities to understand and interpret data, a kind of secret weapon for doing better business, as depicted in The Big Short.

But with a supply of just over 11,000 data scientists and a rapidly growing demand, the competition among employers to secure this role is steep. The U.S. Bureau of Labor Statistics projects that demand will be 50-60% higher than supply by 2018. And McKinsey predicts that by 2018, the United States alone will face a shortage of 1.5 million analysts and managers who know how to use big data to make decisions.

Companies who don’t hire a data scientist now might not be able to find one at all.

The Role of a Data Scientist:

Once an organization has a data scientist, however, what then? How do they cultivate an environment that maximizes that person’s skills and makes them want to stay?

Consider first what an average data scientist does all day:

• Builds training sets (3% of the time)
• Cleans and organizes data (60%)
• Collects data sets (19%)
• Mines for data patterns (9%)
• Refines algorithms (4%)
• Other (5%)

Here’s where we see just how un-sexy the role has become, because an overwhelming majority of data scientists agree that collecting data sets and cleaning and organizing them is their least favorite part of the job. Worse, collecting and organizing data has absolutely nothing to do with insights; it’s simply data preparation. It takes a high level of skill to do, but it’s not data science.

Companies could free their data scientists to spend up to 79% more of their time on analysis by having someone else prepare the data. Not only would companies derive more value from every extra moment spent on insights, but they would enable their data scientists to do what they love.

Data preparation, therefore, should be applied to the correct role—data engineer.

The Role of A Data Engineer:

To read the rest of this article, click here. To read more DSC articles involving data engineers and data science, click on the links.