What is predictive analytics?
In the business world, the data we collect and the information we work with mostly relate to events that have already occurred. We have post-sale invoices, post-expense bills, and employee files pertaining to the employees we have already hired. But of course we want to know the future. How many sales will we have next year? And how many products should we have in stock to meet this demand?
To answer these questions about the future, we can look for patterns in our existing records of past events and project them into the future. We call this process predictive analytics.
This pioneering form of data analysis has many different uses.
- Manufacturer can analyze past failures and predict when devices need servicing to avoid future failures.
- Marketer can analyze their best accounts and create campaigns targeting similar people in the hopes of attracting new, high-quality prospects.
- E-commerce sites who tell us that the people who bought this bought this too: We’re all tempted, aren’t we?
How does predictive analytics work?
In the best case, predictive analytics can provide deep insight into future events. On the other hand, poorly implemented systems can quickly lose our trust. However, all systems share some common processes and some common keys to success and failure. Here is the basic process:
- We analyze our existing data to learn statistical patterns.
- From these patterns we create a set of rules – a model – that describes how the patterns are applied to new data.
- We then feed new data through the model and the rules make predictions about what might happen in the future.
For example, we may analyze existing customer data and find that younger people like products with more features, but older buyers are willing to pay extra for products made from higher quality materials. Based on these patterns, we can apply rules to new customer responses when they register in our system. When they are younger we can successfully offer them more features and when they are older we can offer them higher quality products. In this way we hope to optimize our sales.
We often repeat this process regularly to keep the model current: It learns new patterns, so you’ll hear this part of the predictive analysis process a lot: machine learning.
Four important keys to successful predictive analytics
- Most importantly, good predictions are based on good data. If your current records are incomplete or inaccurate, you cannot expect good predictions from predictive analytics. For example, do you have demographic information about your customers, and if so, is it thorough and up-to-date?
- Good future results depend on choosing the best predictive modeling techniques when looking for patterns. There is a certain art in this that is part of the expertise of the data scientist. But today predictive modeling uses automated machine learning, which can do quite complex statistical modeling experimentally on its own to get the best practical results.
- Ambiguity is inevitable with predictions and we must learn to work with imperfect results. We cannot predict the future with certainty – especially when it comes to customer behavior. We need to understand the accuracy of our model and how confident we can use its results. All of this may sound challenging, but we do it all the time, for example with the weather forecast, which is generally accurate enough to be useful but seldom perfect.
- The predictions made should be actionable insights. That said, you should be able to do something useful with the prediction and also test in the future whether the prediction turns out to be accurate enough to be helpful.
What’s new in predictive analytics?
Predictive analytics may sound very new. Not really. Some of the statistical techniques – Bayesian analysis and regression – have been around for over 200 years. Nonetheless, contemporary predictive analytics really took off with the development of digital computing in the 1950s, when the development of modern algorithms, including neural networks, began. In the past few years, however, there have been very significant improvements that have resulted in both simpler everyday analysis and more advanced artificial intelligence.
The drivers behind these new developments are simple but powerful.
- We have more data than ever and storing that data is inexpensive, especially in the cloud.
- We also have more complex data than ever before and can not only access structured data sets, but also images, sound files and documents.
- We have more computing power available, often in the cloud, which enables us to cope with this size and complexity.
- Finally, better software design takes advantage of all of these developments to make building, testing, deploying, and using predictive analytics easier and more reliable than ever.
Where is predictive analytics used?
With these new capabilities and capabilities, predictive analytics is finding application in an ever-growing range of use cases and industries. Here are some examples.
Financial services. Predicting stock prices and other financial indicators is an important practice. But banks, mortgage lenders and credit card companies also want to detect fraudulent transactions, offer their best customers the best interest rates and sell new financial products to new customers. In all of these cases, predictive analytics has proven itself.
Retail, other consumer-oriented industries. Other consumer-centric companies such as retail and telecommunications use similar algorithms when dealing with customer relationships. They also want to know beforehand if customers may be dissatisfied or likely to switch to a different provider or service – what they call churn analysis.
Airlines. Airlines predict how many seats they can occupy – not always successfully. Remember what we said about dealing with ambiguity and inaccuracies.
Transport and logistics company. Predictive analytics is used to optimize supply chains – here, too, we know the ambiguities.
Overall, predictive analytics has made modern companies very efficient.
How to start Benefits of predictive analytics
With this in mind, how can we start effectively with predictive analytics?
I like to recommend three simple scenarios because they can be widely used across many different organizations and you probably already have the data you need. In addition, the techniques used are relatively simple. Finally, you can easily implement and test the results.
Time series analysis
A time series is simply any data that records a change in values over time. Think about daily sales, weekly bills, your monthly expenses, or your annual budget. But you can also take into account the operating temperature of some devices or the number of visitors to a website.
How this can benefit your company. Now imagine what you could do if you could start predicting based on your existing data. How high will our sales be in the next month? What will our average invoice value be in the next year? Maybe this device is running hotter than usual – dangerous that way?
Almost every company, somewhere in their business, can benefit from these predictions. Modern algorithms for doing this can be quite sophisticated, but you could choose even simpler methods. Moving averages are very simple – you can work with them in Excel – and are still very popular in stock market and commodity analysis. Statistics, engineering, and marketing students often learn algorithms like ARIMA (AutoRegressive Integrated Moving Average) in college and the techniques available in many business intelligence tools. These algorithms can even adapt to seasonal changes.
With simple time series analysis, you can have well-informed, effective business conversations about current trends and future opportunities.
Remember our example where young people buy products with more features and older customers might pay more for better quality? If we could graph the ages of customers versus the amounts they spent, we could see groups popping up on the page. Older, higher paying customers, younger customers who spend less, maybe the middle customers buy the most. And scattered around the edges, some outliers that do not fall into any group.
How your company can benefit from it. Cluster analysis naturally finds these patterns in a much more sophisticated way than just described. You can then use these groups for targeted marketing or for the cross-group design of products that are as attractive as possible. You can even look at these outliers – why don’t they quite fit? Perhaps their expenses are suspiciously high or disappointingly low. Sellers want to know.
We all know this use case. At best, we may not even notice that an analytical engine is stimulating us to make a new purchase or behavior.
The most basic recommendation is the familiar one Customers who bought X also bought Y. Or they saw A and saw B too, so we recommend you give B a try.
Such systems can be built with very simple data – often just a customer’s ID, the ID of the product or service they selected, and perhaps some data on when these events occurred. A more sophisticated analysis will include all sorts of data on prices, genres and styles, etc. But you can really get started simply.
Challenges to Avoid. These systems go wrong if we are not careful to only use patterns and rules that have strong backing. If only one customer bought A and B, that’s not a great recommendation. We also need to be aware of what our customers have already bought in the past – nothing looks clumsier than algorithms that recommend something to the consumer that they have already bought.
Recommendations need not be related to sales or viewing habits. They can also be applied to the maintenance of equipment and even to driving routes for navigation applications.
One last prediction
If you’ve never done predictive analytics before, these three quick wins may be just the way to get started. But I think I can be sure of one thing: whatever your business interest or problem, the future lies in predictive analysis.