Back to the complete issue
Wednesday, 16 June 2021

From cleaner inboxes to Youtube rabbit holes: how recommendation engines have taken over our lives

If you’ve ever wondered how you lost an evening to surfing through cat videos, you can blame Distinguished Doctor Doug: What started out as a seemingly benign tool designed by a small group of computer scientists to reorganize email by priority has evolved to become the much larger force influencing our tastes, preferences and attention to an alarming extent. One of the foundational architects of recommendation engines and the inventor of the “like” button, a man called Distinguished Doctor Doug (yes, that what he actually goes by) explains how his idea went from a time saver to a time sucker on this episode of NPR’s Planet Money podcast (Listen, runtime 20:45)

A collaborative effort to take back precious time gone awry: Stuck ploughing through hours of email a day in the early nineties, Doug was looking for a way to better sort his inbox. He started by setting basic filters that would prioritize some emails over others based on sender and subject. Later, he added “like-it” and “hate-it” buttons that would better flag the kinds of email he wanted to avoid. Eventually his friends jumped aboard, rating their emails, which made him realize their inboxes could collectively become even better at figuring out what was important to them, cutting down daily time spent on email by half. The discovery marked the beginning of what was called “collaborative filtering” and paved the way for companies like Netflix and Youtube to become the internet giants they are today.

In 2006, Netflix offered USD 1 mn to anyone who could make a 10% improvement on their recommendation engine: An international coalition of machine learning experts and statisticians who called themselves BellKor’s Pragmatic Chaos ended up taking the prize by expanding Doug’s collaborative filtering model to rely on lesser known statistics about viewing habits like hidden themes in movies, and whether or not someone has rated anything in the past.

So what’s the problem with that, you ask? While there’s nothing inherently wrong with getting better recommendations for movies and music, a bias towards so-called recommended content can start to overrule our actual preferences. Some research suggests that people are sometimes even more likely to pay significantly higher prices to purchase songs that have received an artificially higher rating when they were told it was based on their personal preference.

Recommendation engines could threaten our ability to make independent choices: “Recommender systems can encroach on individual users’ autonomy, by providing recommendations that nudge users in a particular direction, by attempting to “addict” them to some types of contents, or by limiting the range of options to which they are exposed,” write the authors of this very recent study. Questions on privacy, the criteria based on which engines make recommendations, and the “fairness” of the entire process are also concerns.

Are we heading towards an AI standardized “taste”? People could also begin to veer away from their actual likes and dislikes and towards what comes up in the recommendations tab, which could ultimately become the same for everyone. Research on the ethical questions surrounding recommendation engines is still in its infancy, but some have raised concerns that they could lead to homogenized preferences among consumers, with everyone buying, watching, and listening to the same thing. Conversely, the “fragmentation effect” is also a concern, whereby ultra-specialized recommendations keep consumers trapped in their niche, and limit the scope of their exposure to new ideas and products.

Our take? Recommendation engines are definitely useful, but don’t let them keep you pigeonholed. Shake up your watch-list every once in a while by venturing beyond the recommended content and seek out things you may like through reviews, or recommendations from friends and family.

Enterprise is a daily publication of Enterprise Ventures LLC, an Egyptian limited liability company (commercial register 83594), and a subsidiary of Inktank Communications. Summaries are intended for guidance only and are provided on an as-is basis; kindly refer to the source article in its original language prior to undertaking any action. Neither Enterprise Ventures nor its staff assume any responsibility or liability for the accuracy of the information contained in this publication, whether in the form of summaries or analysis. © 2022 Enterprise Ventures LLC.

Enterprise is available without charge thanks to the generous support of HSBC Egypt (tax ID: 204-901-715), the leading corporate and retail lender in Egypt; EFG Hermes (tax ID: 200-178-385), the leading financial services corporation in frontier emerging markets; SODIC (tax ID: 212-168-002), a leading Egyptian real estate developer; SomaBay (tax ID: 204-903-300), our Red Sea holiday partner; Infinity (tax ID: 474-939-359), the ultimate way to power cities, industries, and homes directly from nature right here in Egypt; CIRA (tax ID: 200-069-608), the leading providers of K-12 and higher level education in Egypt; Orascom Construction (tax ID: 229-988-806), the leading construction and engineering company building infrastructure in Egypt and abroad; Moharram & Partners (tax ID: 616-112-459), the leading public policy and government affairs partner; Palm Hills Developments (tax ID: 432-737-014), a leading developer of commercial and residential properties; Mashreq (tax ID: 204-898-862), the MENA region’s leading homegrown personal and digital bank; and Industrial Development Group (IDG) (tax ID:266-965-253), the leading builder of industrial parks in Egypt.