The web has grown to become the main source of information and discovery for many people, and they depend on it to help build their perception of the world. However, at the same time the amount of information available has exploded.

Just think of Spotify’s 30m music tracks – there’s enough content just there to fill hundreds of human lifetimes. YouTube receives more video in a minute than you could ever watch and Facebook must choose from an average of 2600 relevant posts when you fire up your feed.

So, to manage the constant stream of potential information from overloading us, we all daily interact with recommender systems now. Some well-known examples include Facebook and Instagram’s Feed, Spotify’s Discover Weekly, movie and book recommendations on Netflix and Amazon – but recommenders are everywhere, assisting you to do everything from booking your travel to dating or ordering food.

However, there is one problem. Diversity.

The public discourse has now accepted terms like filter bubbles, echo chambers and fake news, but we’ve still done preciously little to consider the systems design that caused this new trend.

I heard a panelist in Amsterdam last week say that he genuinely believed Facebook and Google could incidentally cause the next global conflict by virtue of creating a more polarized media landscape. They didn’t plan to, but by rewarding attention-grabbing content that drives engagement, these companies have created a perverse incentive structure for content creators.

Ev Williams, the founder of Medium and Twitter, often uses the car crash analogy. The current systems rewards extremes he says. Say you’re driving down the road and see a car crash. Of course, you look. Everyone looks. The internet interprets behavior like this to mean everyone is asking for car crashes, so it tries to supply more of them.

It interprets what we do, as the person we are.

When popularity and engagement drive the publishing industry, we lose sight of what gave the industry its privileged status in society in the first place – trust and human aspiration. So, while it is tempting for businesses to interpret popularity as a signal that people simply want more of that stuff, that would be a mistake. People also want diversity, because it serves a different purpose in our lives – that of our better, future self. In the recommender systems space, we call this problem exploitation vs exploration.

One of the editors on our blog, Sam Lay, wrote beautifully about why diversity is an important counteract to popularity – and why the reason is human aspiration.

“Every Monday my unambitious and unsophisticated musical choices stare me in the face. I can clearly see why Discover Weekly is choosing the songs it does and that’s slightly embarrassing.…My saves, shares and playlist adds on Spotify indicate my aspirational self, the music I would like to be associated with, whilst what I actually listen to often serves a practical purpose or satisfies a guilty pleasure.”

In short, he argues that there is a difference between what our actual self may do in the moment and what future our aspirational self is trying to steer us towards – and that any product that helped him be more of the latter would be worth more to him. Daniel Kahneman calls this the difference between our experiencing self and our remembering self.

The problem is often that aspiration and long-term product satisfaction cannot be measured as easily and immediately as popularity. However, we know from research that users actually tend to be more satisfied with diverse recommendations, i.e. being exposed to a wider variety of content, which can prompt the experience of serendipity — discovering something new when we were not expecting it. Those sorts of discoveries have disproportionate value.

So, if our ugly actual self stares us in the face every time we open Facebook, Instagram and YouTube, the best business opportunity around might be to cater more for our better selves. We know from countless branding studies that aspirational brands can charge a premium on their products and services (think Apple vs Dell), so I see no reason to assume that this shouldn’t be true in the publishing space too.

When speaking to friends about this article, one thing that kept coming back was how bookstores like Foyle’s in London are offering superior value by having diverse staff pick tables and an aspirational store environment. People buy an experience, not just a book. It might feel counter-intutivie when you’re chasing short-term clicks, but that experience will become ever more valuable if it reaches out to our better selves, rather than just re-circulating our current tastes.

***

The original article is here: https://www.thebookseller.com/futurebook/dont-underestimate-readers-when-it-comes-recommendations-642661