AI set to replace data scientist jobs now too

Research from Gartner has estimated more than 40% of data science tasks will be automated by 2020, highlighting the ambitions of AI.

While any estimates from analysts should be taken with a pinch of salt, the team at Gartner believes the role of the data scientist will become much more refined as automation becomes more advanced. The more simplistic data analysis models will become automated allowing the data scientists to concentrate on more business critical tasks which are a greater benefit for the outlook of the business.

Currently, there are a vast number of data sets which have to be processed by data scientists, some of which would be considered low value to the business, though still necessary. These are high volume and time-consuming tasks, which will not make best use of an already expensive data scientist. Automating these tasks will free up the data scientists to concentrate on high value tasks, but also create a new role within the organization.

“Most organizations don’t have enough data scientists consistently available throughout the business, but they do have plenty of skilled information analysts that could become citizen data scientists,” said Joao Tapadinhas, Research Director at Gartner. “Equipped with the proper tools, they can perform intricate diagnostic analysis and create models that leverage predictive or prescriptive analytics. This enables them to go beyond the analytics reach of regular business users into analytics processes with greater depth and breadth.”

These citizen data scientists would not be qualified to deal with the high value data scientist tasks, though they would be in a good position to action to insight which is derived from the automated low level analysis tasks. It’s a compromise which is unlikely to raise many objections; a vast amount of analysis produced by citizen data scientists will feed and impact the business, creating a more pervasive analytics-driven environment, while at the same time supporting the data scientists who can shift their focus onto more complex analysis.

“Making data science products easier for citizen data scientists to use will increase vendors’ reach across the enterprise as well as help overcome the skills gap,” said Alexander Linden, Research VP at Gartner. “The key to simplicity is the automation of tasks that are repetitive, manual intensive and don’t require deep data science expertise.”

While the prospect of more tasks becoming automated may be a scary thought for some, in reality it is the only option. The sheer volume of data to be analysed at the low-end of the value chain is beyond the capability of manual processing. This is a prime example of how AI can be of value to society; maybe it’s not all bad news all the time.