What would you recommend?

Personal recommendations are probably the strongest referral. If your friends tell you to checkout a restaurant they recently discovered, you are highly likely to follow their advice (assuming a general level of trust between you and your friends). For e-commerce companies and online platforms, recommendations are of critical importance. No wonder that plenty of research has been attributed to the development of the “perfect” recommendation engine.

Amazon, Netflix, Coursera, AirBnB – all of the tech giants are using recommendation algorithms to engage user and drive growth. What product will a certain user buy next? (exemplified in Amazon’s famous “people who have bought this product also bought xy”) What movie will a certain user enjoy with high probability? Which online education course should a user delve into after having finished four basic accounting courses? After the trip to Barcelona, for what destination should a user be offered a discount? While every user certainly possesses a decent degree of curiosity and freedom to actively search driven by own desires, tailored recommendations increase conversation chances significantly.

Recommendation engines have become so common place, that e-commerce websites can use off-the-shelf plugins to present their product portfolio in a highly dynamic, customer-centric way. These plugins oftentimes charge per order realized, thus having extremely low entry barriers and high adoption rates. Beyond the high impact that recommendation algorithms continue to have in an e-commerce setting, understanding the underlying principles can also enable a fresh perspective on other fields of application, which we have not necessarily associated with recommendation.

A lot of recommendation algorithms can be categorized as content based recommendation algorithms. Looking at the characteristics of a certain set of items (e.g. those that a user has bought, liked etc.), the most similar items are being computed and consequently recommended. Essentially, even though a “recommendation” is the final outcome, at the core of these algorithms are similarity detection mechanisms.

Calibrating similarity sounds easier than it is, because the more dimensions there are as part of a “description”, the more fine-grained the distinction has to be. For example, having only information about “price” of an item makes it easy to find out similar items. Once additional information such as “weight”, “color”, “description”, “country of origin”, “category” etc. come into place, computing similarity is more difficult. So, even if you don’t want to recommend anything to your users, knowing the similarity between items might be helpful. Take the following examples:

  • Automatic categorization: if you have large categorization tasks in your enterprise (e.g. of new product, etc.) you can use similarity metrics to identify the most likely category, thus automating a task which would otherwise require massive human resources
  • Product strategy: Analyzing the your own and the market’s products might help you identify white spots or niche segments, which you are currently not serving adequately
  • Monitoring current events: Detecting similarity between news articles allows for clustering, allowing for an easy monitoring of current events.

Another large set of recommendation algorithms takes a closer look at user interactions (collaborative filtering). Broadly speaking, if a user likes a certain set of items, the algorithm automatically compares these preferences with those of other users. User A is then more likely to favor a certain random item, if a large set of users with an overlapping set of previous behavior have expressed a positive opinion about this item. Based on this, a user-specific prediction / recommendation can be generated.

In the same way as these recommender systems operate, one can approach challenges of a similar kind, because the underlying principles are applicable to different domains:

  • Pattern recognition: When it comes to analyzing user behaviour (be it for electricity consumption, or fraudulent behaviour), it is essential to structure the behaviour. What do users with bad intent usually do before they engage in fraudulent behavior? What does usually happen after an early morning electricity peak if the user has not used any electronic device the last week? There are issues, which require “collaborative filtering” in some sense, as common behaviour is used to extrapolate on future behaviour.
  • Identifying new sales targets: In B2B sales, when the target audience of your product or solution is large, identifying the right subset of companies to approach is critical. Knowing the purchasing behaviour of current clients in other areas, companies can scan the market for companies with similar behaviour. Simplifying: If you know that some of your clients are also clients at a different store, you might want to target the entire client base of that store.

Powerful recommendation engines have been the fuel for growth in a lot of the major tech companies. Beyond off-the-shelf recommender systems, understanding the underlying concepts is key to expand them to other areas (beyond the well known e-commerce application) within the enterprise to further the agenda of efficiency and automation. It remains unclear, how much room for “improvement” recommendation engines do still have – but the fact that they continue to receive a lot of research attention signals that the end of the road has not been reached.

Big Data