Generative AI Threatens Diversity and Hyperlinks
There is no doubt that generative AI and chatbots can be useful. In some situations they will be transformative. As for the downsides, the fact that they “hallucinate” is troubling and often discussed, if disturbingly also often forgotten. But what is rarely mentioned is how they also represent an assault on diversity of opinion and information. Here we try to explain in some detail why this is the case. We are not taking a position on the net worth of this technology and trend, but rather seeking to explain why it matters a lot, and in a way that has been rarely gone into in any detail.
Why are chatbots and AI assistants particularly appealing? Well for one thing we are all, of course, busy (and often lazy). And so it is too easy to accept the one source of information or “truth” that they frequently provide. When that information comes from the predictive text of one AI model, instead of the writings of multiple sources, we are vulnerable. We are being drawn into a trap, through the allure of convenience.
We’ve seen a similar story, as it played out in the last decade in social media. Our feeds were curated by (AI) recommendation algorithms and we passively accepted the ease of curated amplification. This was designed to both keep us engaged, and on their platform. Now you are being enticed to engage with a new type of AI, one that is always available, always responsive. Just as long as you stay on its platform.
As before, digital applications and platforms will do anything to keep you from leaving them by clicking hyperlinks. In the words of a song, our digital gatekeepers welcome you to the Hotel California, and bring you a user experience where “You can check out any time you like, but you can never leave.”
Let’s not forget, our modern world blossomed and flourished through the personal and business connections, and transactions, made online. What underpinned these new relationships and transactions? The answer is hyperlinks. So ask yourself: is suppressing hyperlinks, through AI and chatbots, going to make things better?
In the 2000s the web giants were a positive force, supporting hyperlinks through their search engines: Yahoo!, Google and Bing. Then social networks, notably Facebook and LinkedIn but also others with their walled gardens, started to suffocate the web and started the process of “hyperlink-rot”. Meanwhile the older monopolist, Microsoft, and the new one, Google, carefully started building moats around their search engines. This was done most significantly through imposed choice architecture built into their browsers and operating systems. Now they are seeking to ramp up these tactics by demoting and suppressing hyperlinks, as they engage in an AI search arms race, most prominently using Bing Chat and Bard.
Meet the Old Boss: Answer Engines
Google and Bing are seeking to be your Answer, not Search, engine. This race to be “The Answer Engine”, and keep you on their platform, is nothing new. Google have been at it since at least 2005, when they started iGoogle. This offered a personalised home page with plugins that provided useful utility functions through an API. Does that sound familiar here in 2023 (hint: AI plugins)?
Although it was killed by Google some of the features of iGoogle lived on, migrating into Google Search as “Direct Answers” from 2009. Direct Answers, like chatbots, can be useful and convenient. Both however increase hyperlink-rot. Google didn’t stop there, with these initial answer engine features. Next it was “People Also Ask” in April 2015, rapidly followed by Featured Snippets, Knowledge Cards and Live Results in July 2015. All these features encourage users to stay on the Google search results page, rather than navigating elsewhere on the web. Of course Google is, or at least was, free to do this; it’s simply commercial self-preferencing. But as a monopoly, it becomes a problem.
Microsoft, with no Bing innovations to speak of followed suit, introducing clones of these Google search features. Others don’t go as far but have similar functionality. One such example is the Bing proxy DuckDuckGo, who offer up “Instant Answers”; their own version of Direct Answers.
Meet the New Boss: AI Answer Engines
These direct and instant answers are helpful to the platform and users. Convenience looks and feels like a winner. Who wants to navigate from search results to an independent website to check the weather, or the dictionary meaning of a word? Perhaps only those who value the web, the diversity it provides, and the people and businesses that depend upon it.
Let’s take an example shown below on Mojeek, Google and Bing search with the query: “Great places to visit in Rome”.
- Mojeek is all organic hyperlinks
- Google is 100% Ads
- Bing is 100% Ads
Now, what happens when you try the new generative AI chatbots? As you can see in the second figure:
- Bard is all consensus information on the page, with no hyperlinks
- Bing Chat is also consensus information with two named hyperlinks; one of them is from TripAdvisor – who perhaps not coincidentally have placed Ads below the map.
How long do you give it before Ads start appearing in Bard and Bing Chat? Either way, diversity of information through organic hyperlinks is under further attack by Google and Microsoft.
The Joy and Power of the Web
Do you remember the joy of serendipity as you discovered a new source of information or perspective on the web? This joy was once a common experience on Google, but they whittled it away. And now they are following the charge of Microsoft who, with OpenAI, seek to strangle the Web with AI. It’s still there if you turn to Mojeek.
We are not suggesting that generative AI and chatbots are not useful, when used appropriately. We recognise the productivity gains they can bring. They can be a great assistant to thinking, information processing and programming. But we need to be aware of what we lose when we succumb to the convenience they provide.
Below we’ve tried to illustrate what is happening in graphic form. The irony is that chatbots are based on Large Language Models, and these in turn depend on web search databases for their training. As the figure below illustrates answer engines and chatbots extract web data, build a model (“brain”) then want to keep you in their walled garden (“Stay here”). Search Engines suggest instead that you explore the Web (“Go here”). They return you to the source of their intelligence, and so play fair with the website content and creators on whom they depend by linking back.
Do AIs that hoover up web data and don’t link back play fair? It seems to be a case of doing what they can get away with. It’s no wonder that likes of Reddit and Stack Overflow are putting up their walls with paid APIs. Small businesses and publishers are meanwhile collateral damage in the race for AI supremacy, as their hyperlinked traffic on which they rely gradually dries up.
Only by supporting and encouraging a diverse and accessible internet can we continue to reap the benefits of this powerful network of hyperlinks and information. Search engines are a wonderful tool for discovering new information, different suppliers and navigating the web. As Google and Bing go further down the path of being answer engines, the value of a real search engine like Mojeek is becoming more and more obvious.
