Leveraging the value of big data with automated decision making

It is a widely accepted fact that we are living in the era of Big Data. Many traditional companies are looking for ways to improve their business through the virtues of Big Data and Data Science. While matured startups born in this era (Facebook and Twitter) seem to naturally exploit the value of their data, many traditional companies struggle to find new ways of utilizing data to leverage its value.

In this post, I elaborate on the specific domain of decision making where Big Data and Data Science can help improve the efficiency of conventional businesses. Proving the benefits of Big Data in a lighthouse project is of utmost importance in long-established companies with regard to overcoming initial resistance in the digital transformation of business processes. We will see that the automation of operational decisions, i.e. routine decisions related to the day-to-day running of the business, are especially suitable candidates for lighthouse projects to prove the value of Big Data in a company.

The notion of automating data-driven decisions with the help of Data Science is often denoted with the term Prescriptive Analytics, which can be regarded as the conclusive step after Predictive Analytics. In other words, the predictions generated with the help of Predictive Analytics are used to optimize a predefined metric under consideration of side conditions, strategic direction, business processes, etc., to derive excellent business decisions. The predictive analytics diagram from Gartner illustrates the business value compared to the difficulty of different analytical approaches.

In many businesses repetitive operational decisions consume lots of working time. For instance pricing of articles and services, replenishment of stores or stocks, demand forecasts and customer services involve operational decisions which are often conducted in a manual process supported by traditional, rule-based decision support systems. Automating these decisions with the help of data-driven decision systems has several benefits:

  1. Labor costs are reduced and scarce expertise can be leveraged for non-routine, exceptional decisions. Less routine decisions means having more time for decisions in extraordinary circumstances, as well as decisions that are of a more tactical or strategic nature. This encompasses decisions in situations where data is lacking, as well as decisions about creative and visionary solutions. For instance, no machine-learning algorithm could have ever predicted the success of the first iPhone since it was something completely new and its success was a consequence of many other soft factors.
  1. The quality of decisions is improved given that all information sources used in the manual decision process are available as machine-readable data. Modern machine-learning algorithms are able to quickly analyze huge amounts of data that a human being could never even read in a lifetime. This plethora of data allows the inference of patterns that leads to fast, consistent, high-quality decisions which are resistant to the long list of cognitive biases or decision fatigue.
  1. Prescriptive Analytics allows the number of decisions to scale. Too often in traditional businesses, decisions are made by not actually making a decision, which consequently leads to idleness and thus unexploited potential. Being able to scale the number of decisions enables this untapped potential to be fully realized and can also generate new services. Imagine for instance that one marketing tool of a company is to give special offerings and product recommendations based on different market segments, not single customers. Being able to scale the number of decisions due to automation would allow special offerings and recommendations for individual customers, just like Amazon’s recommendation system.

To justify our statement that the automation of routine decisions with Prescriptive Analytics is exceptionally well suited as a pioneer project in a traditional company, it is necessary to elaborate on certain characteristics that many operational decisions hold.

While a single operational decision, e.g. a small change in the price of a single article, may have an insignificantly small but direct impact on the revenue of the whole business, the sum of all decisions quite often has great economic impact. This is due to the fact that the frequency of operational decisions is often huge, meaning that a small overall improvement in decision quality is highly profitable. Obviously, candidates for a Prescriptive Analytics project should have exactly these properties of high and direct economic impact. The ability to measure such an impact requires that a performance metric or key performance indicator (KPI) is already established. This is another important prerequisite for a successful Prescriptive Analytics project.

Since operational decisions are often related to the core of the business, even in traditional companies huge amounts of data are already collected and available. Often, routine decisions that are taken by analyzing spreadsheets and personal experience are based on data with high predictive power. A quote, often attributed to Mark Twain, says that “history doesn’t repeat itself, but it does rhyme,” which captures the essence of what automated decision making is about. Having lots of data about past events allows us to find patterns and relationships which can predict future events to some extent. The goal is to develop a model that describes what happened in the past without being bound to the past, and thus allowing us to apply the model to the future. It’s the same way as our brain learns from experiences and infers future outcomes in similar situations.

Consequently, the high frequency of routine decision with a direct economic impact, combined with an abundance of data and a metric to measure performance, are favorable characteristics of a business process that can be successfully automated. In order to quantify the added value of Prescriptive Analytics, an estimation of the gain in decision quality and its impact on revenue is needed with the help of the predefined metric or KPI. For this complex estimation, traditional companies may want an experienced partner and, optionally, a proof of concept to evaluate the predictive power of the data and the business case as a whole.

Florian Wilhelm CSC Blogs

Dr. Florian Wilhelm is a Senior Consultant Data Scientist within CSC’s unit Big Data & Analytics (BD&A). He is specialized in Predictive & Prescriptive Analytics, Data Science and Big Data within several industries for more than 5 years and has strong experience in consulting, digitalization and project management.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: