Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

To Break Google’s Monopoly on Search, Make Its Index Public

bloomberg.com

Post reply on HN
Print view View original

511–520 of 630 posts

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#511

Post by nothis »

nordsieck wrote: post #342
nothis wrote: post #334

I don't mind google search. Let them have search, they're obviously the best at it. What I have a problem with is literally every other service they provide. Google shouldn't own the most popular browser, the most popular video streaming platform, the most popular map service, the most popular operating system and (one of the?) most popular email services. Why is a single company allowed to have control over essentia…

> What I have a problem with is literally every other service they provide. Google shouldn't own the most popular browser, the most popular video streaming platform, the most popular map service, the most popular operating system and (one of the?) most popular email services. So you're saying they're giving away too much good free stuff and need to be stopped?

I mean, obviously it’s not “free” to society.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#512

Post by bleuarff »

drstewart wrote: post #411
nothis wrote: post #334

I don't mind google search. Let them have search, they're obviously the best at it. What I have a problem with is literally every other service they provide. Google shouldn't own the most popular browser, the most popular video streaming platform, the most popular map service, the most popular operating system and (one of the?) most popular email services. Why is a single company allowed to have control over essentia…

Why is it people care so much about Google being in many different markets but I never see anyone calling for Samsung or Berkshire Hathaway to be broken up? Just goes to show how easy populism is to push. Google and Facebook are in the news a lot and used by the average consumer, making them the perfect scapegoats. Meanwhile, the average consumer just sees Samsung as the TV people (actual industries: apparel, automot…

I get your point, but Samsung is a manufacturer, it sells products. Google and FB can have much more impact leveraging the data they have on everyone, which is why we need as a society to be cautious and keep them in check.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#513

Post by nothis »

vinaypai wrote: post #339
nothis wrote: post #334

I don't mind google search. Let them have search, they're obviously the best at it. What I have a problem with is literally every other service they provide. Google shouldn't own the most popular browser, the most popular video streaming platform, the most popular map service, the most popular operating system and (one of the?) most popular email services. Why is a single company allowed to have control over essentia…

What's stopping you from using Bing Search, Open streetmaps, hotmail, Firefox etc.? Google doesn't "control" anything you don't let it.

I’m using Firefox and DuckDuckGo and it’s okay. But, for example, I naively made gmail my work mail so I’ll be stuck with that for a while. But a few idealists on the Internet going through the effort of de-google-ing their lives doesn’t change their grip on society.

It’s rather obvious and there’s a lot of google employees on hacker news so I don’t really see the point of arguing further. You’re having a giant part of the defining tech of the 21st century controlled by one company. How’s that healthy?

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#514

Post by lostctown »

arwineap wrote: post #507
lostctown wrote: post #447

Earlier quoted context omitted.

I wish interfaces were more straight up about their intentions and made it easier to implement account level partitions. For work I love Google's magic tracking effects, but at 1 am, hell no.

You can have multiple identities in chrome[0], even guest identities. [0] https://support.google.com/chrome/answer/2364824

Right. I'm just saying it should be clearer. Ex: I want to have a list of accounts, netflix style, that I'm presented with on an empty chrome window. If in fact multiple identities don't merge data implicitly in anyway than this is just a UI issue.

But I have a hard time believing google truly partitions everything in a multi account setup.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#515

Post by altcognito »

nostrademons wrote: post #400
mrtksn wrote: post #388

Earlier quoted context omitted.

I think it is possible to make way, way better search engine because Google Search is no longer as good as it used to, at least for me. I can no longer find anything remotely good quality, I discover new and quality stuff from social media like Twitter and HN. The search results seem to be too general and too mainstream. Nothing new to discover, just a shortcut to the few websites like Reddit, StackOverflow for more…

The reason for that is because Google's building for a mainstream audience, because the mainstream (by definition) is much bigger than any niche. They increase aggregate happiness (though not your specific happiness) a lot more by doing so. It's probably possible to build a search engine for a specific vertical that's better than Google. However, you face a few really big problems that make this not worthwhile: 1) Sp…

> It's probably possible to build a search engine for a specific vertical that's better than Google.

Funny, I don't disagree with this, but my perception has been that Google seems to detect when I've switched roles from one type of programmer to another. I don't know if that's organic from the topics I'm looking up or not, but if I'm looking up a generic string search, it seems to return whatever language I've been searching for recently. (very recently in fact)

My point is, it seems like the search engine intuitively understands my "vertical" already. Maybe it's just because developer searches are probably pretty optimized.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#517

Post by [deleted] »

jazoom wrote: post #500
appleshore wrote: post #493

Earlier quoted context omitted.

If there were viable alternatives, people would shift over time. If I type in “ Pentagon” on Google, the first link is LinkedIn. DuckDuckGo doesn’t even list it at all. There’s countless examples where DuckDuckGo just can’t find basic information. DDG is just unreliable beyond it’s silly name.

I try to use and like DDG, but the results just aren't as good. For example, it seems to be completely unaware of Docker Hub. Like, pages from that entire subdomain never show up. I can search "Docker hub" and it doesn't even show up.

[deleted]

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#518

Post by foota »

zubspace wrote: post #468
foota wrote: post #449

Earlier quoted context omitted.

> I don't want Google results! I want different web crawlers ordering the results according to my taste Okay, great, we'll see how you browse and use that to determine... > without tracking each and every page impression of me. Give me that and I'll switch in a heartbeat. Uh? How exactly do you want them to do that?

The search engine basically owns an index of all websites. I assume, that crawling websites and storing an index is a solved problem. Not trivial on that scale and expensive. But conceptually solved. What google does really well are two things: 1) It ranks those pages according to the well-known PageRank algorithm, although the details are kept secret. They also learned how to punish sites, which try to mislead the a…

While you may be able to open up some switches, I don't think you can build a reasonable user experience on something like that, since anything that allows you significant control over how the results are chosen (beyond the simple control you've mentioned, like prefer certain types of results) will be both too complex for anything other than like distributed common sets of options (like a plugin that comes with presets) and/or have so many knobs as to make tuning for good results infeasible.

I would argue Google already supports some of what you want, like allowing you to search just for news, although additional filters based on something like the niche you're looking for or the type of content don't hurt.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#519

Post by pests »

altcognito wrote: post #515
nostrademons wrote: post #400

Earlier quoted context omitted.

The reason for that is because Google's building for a mainstream audience, because the mainstream (by definition) is much bigger than any niche. They increase aggregate happiness (though not your specific happiness) a lot more by doing so. It's probably possible to build a search engine for a specific vertical that's better than Google. However, you face a few really big problems that make this not worthwhile: 1) Sp…

> It's probably possible to build a search engine for a specific vertical that's better than Google. Funny, I don't disagree with this, but my perception has been that Google seems to detect when I've switched roles from one type of programmer to another. I don't know if that's organic from the topics I'm looking up or not, but if I'm looking up a generic string search, it seems to return whatever language I've been…

I think its totally possible, two examples already:

Google Ads (used to?) lets you target by "bahaviour" vs "in-market". They can tell the difference between someone who is passionate about beds, maybe involved in the bed business (behavior) and the people who are making the once-in-a-decade purchase of a bed (in-market).

Google can tell devices apart on the same google account and keep together search threads. I might be programming on my desktop making engineering searches but at the same time I'm googling memes on my phone; both logged into the same account.

Re: To Break Google’s Monopoly on Search, Make Its Index Public

#520

Post by frabcus »

nostrademons wrote: post #491
Eridrus wrote: post #475

Earlier quoted context omitted.

Sure, it costs $50 to grep it, but how much does it cost to host an in-memory index with all the data? This is not a proposal to just share the crawl data, but the actual searchable index, presumably at arms length cost both internally & externally. The same ideas could be extended to the Knowledge Graph, etc. IMO the goal here should not be to kill Google, but to keep Google on their toes by removing barriers to com…

The data was about 55TB of compressed HTML last I looked, so that's about 70 r5a.24xlarge instances, each going for $5.424/hour, so about $350/hour or $250K/month. That's not cheap, and definitely not something you'd put on your personal credit card, but it's well within the range of a seed-funded startup. Sizes may vary a bit depending upon the exact index format, but that should be a rough ballpark. With batch jobs…

"Remember, search touches (nearly) every indexed document on every query" - wait, why does that happen?

Doesn't it only touch ones with at least one of the search terms in, or stemmed/varied words relating to some of the terms? And does that via an index?

Post reply on HN