From his blog:
Most tech folks I’m connected to seem to think that Google search results are significantly worse than they were ten years ago . . . However, there’s a sizable group of vocal folks who claim that search results are still great . . .
I [Luu] suspect what’s going on here is that some people have gotten so used working around bad software that they don’t even know they’re doing it, reflexively doing the modern equivalent of hitting ctrl+s all the time in editors, or ctrl+a; ctrl+c when composing anything in a text box. Every adept user of the modern web has a bag of tricks they use to get decent results from queries. From having watched quite a few users interact with computers, that doesn’t appear to be normal, even among people who are quite competent in various technical fields, e.g., mechanical engineering. However, it could be that people who are complaining about bad search result quality are just hopping on the “everything sucks” bandwagon and making totally unsubstantiated comments about search quality. . . .
Since it’s fairly easy to try out straightforward, naive, queries, let’s try some queries. We’ll look at three kinds of queries with five search engines plus ChatGPT and we’ll turn off our ad blocker to get the non-expert browsing experience . . .
Luu follows up with the results for six different search sites, applying the following six queries:
– download youtube videos
– ad blocker
– download firefox
– Why do wider tires have better grip?
– Why do they keep making cpu transistors smaller?
– vancouver snow forecast winter 2023 . . .
He gives lots and lots of details and analyses. You really have to read the whole thing—it will take a couple hours because he links to various other posts. But it will be an entertaining couple of hours. [Say what you want about Google, when I searched *the norm of entertainment*, it came up with my blog post as its first link. — ed.]
It’s kind of impossible for me to summarize Luu’a results, mostly because I don’t understand a lot of the technical details. But I’m a big user of search! My take-home message is that it makes sense to use different search engines for different goals, rather than just running everything through Google. As Luu puts it:
On the question of whether or not a collection of small search engines can provide better results than Google for a lot of users, I don’t think this is much of a question because the answer has been a resounding “yes” for years.
The only trouble is, he doesn’t provide direct guidance on what collection we should be using. He does give us this teaser:
On how to get good results for other queries, since this post is already 17000 words, I’ll leave that for a future post on how expert vs. non-expert computer users interact with computers.
OK, the clock is ticking!
Regarding Google specifically, Luu agrees with the statement that “straightforward search queries are inundated with crap” and writes:
Although Google doesn’t publicly provide the ability to see what was historically returned for queries, many people remember when straightforward queries generally returned good results. One of the reasons Google took off so quickly in the 90s, even among expert users of AltaVista, who’d become very adept at adding all sorts of qualifiers to queries to get good results, was that you didn’t have to do that with Google. But we’ve now come full circle and we need to add qualifiers, restrict our search to specific sites, etc., to get good results from Google on what used to be simple queries. If anything, we’ve gone well past full circle since the contortions we need to get good results are a lot more involved than they were in the AltaVista days.
I’m surprised that his post says nothing about https://www.perplexity.ai/, which has long been my go-to search alternative to Google, ChatGPT, and the like, because it consistently yields more accurate, in-depth, source-cited answers than the others.
The seemingly impressive performance of Perplexity is largely due to their willingness to regurgitate the work of others without compensation (either through links or a licensing fee):
https://www.theverge.com/2024/6/27/24187405/perplexity-ai-twitter-lie-plagiarism
Your right. As the thief explained to the judge presiding over his case, though:
“I understand that shoplifting is against the law, your honor, but look at all the great things I got”
This isn’t particularly surprising to me, since Perplexity (from my thirty seconds of using the site) isn’t really a search engine. Looks like it’s generating an article on-demand from the search query.
For what it’s worth, the result for my one query (“inside out 2 box office”) gave me a false answer. Cited answer says it sold $1.35B globally, and the generated article says $1.29B. The domestic sales number is wrong as well.
How does it do with Dan’s evaluation in the article?
Honestly, I have felt like Perplexity’s ability to turn up good search results is generally pretty poor in my experience. I like it for topics for which I expect finding good sources is trivial and instead I want it to do the work of reading and summarizing for me.
I have a friend who used to work for Google, and I think his wife still does. This is apocryphal but can probably be backed up with some good searching (on what engine!?): about a decade ago, they essentially made the decision to go all-in on SEO and ads, rather than providing a service. The reasoning was, I suspect, that it was perceived as a relatively minimal drop in quality but for a significant increase in revenues. I think this basically coincided with the transition to alphabet, more focused on providing a suite of interlinked Google-based services. When most people say that searching has become worse, I think they are still basically just using Google for everything, and it has certainly become worse.
The focus on SEO is likely also why facetious answers from Reddit tend to get picked up by LLMs – people want real humans answering a question that they have posed, rather than a marketing site, so they add ‘reddit’ to the query. The LLM tags this as a high-value source and you end up getting all manner of unvetted silliness.
It has been documented that the executive who tried to defend the search team from infection by the ad team lost that battle and was moved aside.
Ed Zitron, who writes and podcasts on the tech industry (and I believe is primarily a PR guy who has experience dealing with some of these companies), goes into some detail in his Substack, starting with this post: https://www.wheresyoured.at/the-men-who-killed-google/
I had a go with that Marginalia search engine referenced in the article and it was pretty fun. It didn’t get me exactly what I was looking for (song I’d forgotten the name of), but I did get a fun retro website in the process with a bunch of other songs.
The theory that small search engines are viable now but not in 2016 is cool!