Live data from Hacker News

How is search so bad? A case study

svilentodorov.xyz

351–360 of 416 posts

Re: How is search so bad? A case study

#351
When I'm ready to flip the table over about search results, I remember that https://millionshort.com/ exists and I give that a whirl. Then when I'm sick of seeing links to sites that are highly-SEOed but low-signal, Tampermonkey is there to give me a nice little [block] button to remove them.

Fantasy-future: Mozilla could "widen out" their library and hire 20,000 librarians to curate the "New Web" in a non-wiki-format. If you paid each librarian about $150K/annum, that's about $0.60/annum from 5B subscribers, just for their salaries for the advertising-free library.

Re: How is search so bad? A case study

#352
post #222

Earlier quoted context omitted.

> I can possibly not find anything deep enough about any topic by searching on Google anymore. > It kills my curiosity and intent with fake knowledge and bad experience. I need something better. It's hard for me to take this seriously when wikipedia exists, and almost always ranks very highly in search results for searches for "knowledge topics". Between wikipedia and sources cited on wikipedia, I find the depth of a…

The fact that Wikipedia exists, is frequently (though not always) quite good, has citations and references , and ranks highly or is used directly for "instant answers" ... ... still does nothing to answer the point that Web search itself is unambiguously and consistently poorer now than it was 5-10 years ago. Yes, I find myself relying far more on specific domain searches, either in the Internet sense, or by searchin…

This. More and more, I have been finding that good books provide better learning than the internet.

You search for quality books online, mostly through discussion forums as search fails here, or through following references of books and articles. Then spend time digesting them.

Re: How is search so bad? A case study

#353
post #217

Earlier quoted context omitted.

Good to hear your concerns. > The real problem with search engines is the fact that so many websites have hacked SEO that there is no meritocracy left. I intend to announce the alpha test of my search engine here on HN. My search engine is immune to all SEO efforts. > I can possibly not find anything deep enough about any topic by searching on Google anymore. In simple terms my search engine gives users content with…

This sounds extremely implausible; especially claiming "immune to SEO" is like declaring encryption "unbreakable". A lot of human effort would be devoted to it if your engine became popular.

How to be immune to SEO? Easy once see how.

More details are in my now old HN post at

https://news.ycombinator.com/item?id=12404641

For a short answer, SEO has to do with keywords. My startup has nothing to do with keywords or even the English language at all. In particular, I'm not parsing the English language or any natural language; I'm not using natural language understanding techniques. In particular, my Web pages are so just dirt simple to use (user interface and user experience) that a child of 8 or so who knows no English should be able to learn to use the site in about 15 minutes of experimenting and about three minutes of watching someone use the site, e.g., via a YouTube video clip of screen captures. E.g., lots of kids of about that age get good with some video games without much or any use of English.

E.g., how are you and your spouse going to use keywords to look for an art print to hang on the wall over your living room?

Keyword search essentially assumes (1) you know what you want, (2) know that it exists, and (3) have keywords that accurately characterize it. That's the case, and commonly works great, for a lot of search, enough for a Google, Bing, and more in, IIRC, Russia and China. Also it long worked for the subject index of an old library card catalog.

But as in the post I replied to, it doesn't work very well when trying to go "deep".

Really, what people want is content with some meaning they have at least roughly in mind, e.g., a print that fits their artistic taste, sense of style, etc., for over the living room sofa. Well, it turns out there's some advanced pure math, not widely known, and still less widely really understood, for that.

Yes I encountered a LOT of obstacles since I wrote that post. The work is just fine; the obstacles were elsewhere. E.g., most recently I moved. But I'm getting the obstacles out of the way and getting back to the real work.

Re: How is search so bad? A case study

#354
post #7

Earlier quoted context omitted.

My guess is that suppressing spammy pages got too hard. So they applied some kind of big hammer that has a high false positive rate. You're getting the best of what's left. Maybe also some quality decline in their gradual shift to less hand weighted attributes and more ML.

My guess is that Google et al are all hell-bent on not telling you that your search returned zero results. They seem to go to great lengths to make sure that your results page has something on it by any means necessary, including: searching for synonyms for words I searched for instead of the specific words I chose, excluding words to increase the number of results (even though the words they exclude are usually the…

It's surprisingly easy to get zero results returned pasting cryptic error messages. It doesn't mean there is nothing, though. Omit half the string, and there's the dozen stack overflow threads with the error. Maybe it didn't read over the line break on stack overflow or something, but I haven't tested anything.

Re: How is search so bad? A case study

#355

This should probably be a separate submission but why is search so bad everywhere? - Confluence: Native search is horrible IME - Microsoft Help (Applications): .chm files Need I say more. - Microsoft Task Bar: Native search okay and then horrible beyond a few key words and then ... BING :-( - Microsoft File Search: Even with full disk indexing (I turned it on) it still takes 15-20 minutes to find all jpegs with an SS…

My gut feeling is search is bad everywhere because no one provides a pure-text API to the content. Cleaning data is hard and its easier to chuck in all the text blasted off an HTML page than to exclude everything non-signal.

I have no answers for Microsoft File Search, it never returns any results for me, I wonder if they even tested it sometimes.

Re: How is search so bad? A case study

#356
post #172

I've been thinking about this for years[1]. The truth is, what Google solved was parsing the search query, not identifying the best results. In fact, Google is not incentivized to give you the best results, they are designed to maximize their revenue, derived from getting you to view / click ads. Google is not a search company, they are an advertising company. The more searches you make, the more revenue they make. T…

Is it time for paid search engines? Make users vote with their wallets and pay for the eternal arms race. Problem is, whoever is behind something like that would have to start with an already sufficiently superior experience (or massive geek cred) to make people pay from the early days. Maybe going back to manually curated results of a subset of topics would work? Or some Stack Overflow-esque model of user powered me…

Simpler idea:

Paid search engine that ranks sites based on how often the users click results from that site (and didn't bounce, of course). The fact that it's paid prevents sybil attacks (or, at least, turns a sybil attack into a roundabout way of buying ads).

Of course, at this point, you are now the product even though you paid. But it's a tactic that worked for WoW for ages.

Re: How is search so bad? A case study

#357

Earlier quoted context omitted.

If they were ripe for disruption and it was easy to do this disrupting just be returning better search results, and returning better search results was an easily doable thing then I suppose all the other functioning businesses that have a stake in web search would already be doing that disrupting. Search disrupted catalogs. What will disrupt search?

Boutique hand crafted artisanal catalogs? Not joking I have a feeling subject specific topics will be further distributed based on expertise & trust.

Pubmed is an excellent example of a boutique search engine.

Re: How is search so bad? A case study

#358
post #70
post #9

Earlier quoted context omitted.

Did you try putting them in quotes? EDIT: I don't know why this being downvoted. This is a genuine question to understand if the problem is the size of the index or the fuzzing matching that search engines do.

Google blatantly disregard quotes.

I think the behavior is more complex. I do get disregarded quotes from time to time so I typically leave them off. However, for the query 'keyword1 keyword2', if I get a lot of keyword1 results with keyword2 struck through, and I search again with keyword2 in quotes, it works as expected.

Re: How is search so bad? A case study

#359

I've been thinking about this for years[1]. The truth is, what Google solved was parsing the search query, not identifying the best results. In fact, Google is not incentivized to give you the best results, they are designed to maximize their revenue, derived from getting you to view / click ads. Google is not a search company, they are an advertising company. The more searches you make, the more revenue they make. T…

The ‘Google is an advertising company’ is said often on HN. I agree to some extent but, doesn’t that imply that every newspaper company is also just an advertising company? Google solves a real problem and this works well, for them, with an advertising based revenue model. Do they compromise their search to that end? Probably. Do newspapers? Hopefully not, but maybe. To me, that doesn’t make them advertising companie…

Of course they are. Page ads are probably as old as print. Most reviews are ads. Travel sections are ads. There are real estate sections, and classifieds as well. Perhaps the most honest reporting is found in the local sports section.

Re: How is search so bad? A case study

#360
post #11

This article is essentially just complaining that DDG and Google don't have special parsing for reddit pages ("How come it doesn't know that thread didn't get many upvotes?", "How come it thinks some change to the site's layout was an update to the page?") Maybe if you want to search reddit, the best search engine is the search bar on reddit.com.

The search bar on reddit.com only searches through post titles and not comments. Using Google/DDG/ seaches through all content posted on Reddit and not just post titles. Until Reddit implements proper search, people will keep using search engines to search for content on it.

appending site:reddit.com to your search is very useful.
Post reply on HN