Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: Tweets no longer seem to be properly indexed by Google

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 65 posts

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#41

Post by voytec »

jahewson wrote: post #35

Next week, Twitter is starting to charge for their API https://arstechnica.com/tech-policy/2023/02/cash-strapped-tw... Could be related? Even if they’re just crawling the public pages I could certainly imagine Elon trying to extract money from Google for the privilege.

Just a few days ago I wrote a comment, that Elon's decisions are pushing Twitter towards becoming a ghost town. But I'm unsure whether he could be so irrational to cut the biggest search engine off.

One thing I find important to remember, is that Jack Dorsey gave Elon Musk one bil for the Twitter purchase so maybe there's some kind of Grand Plan that I'm too stupid to see. But would it start with Twitter's bankruptcy?

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#42

Post by bdcravens »

klyrs wrote: post #37
pifm_guy wrote: post #18

Earlier quoted context omitted.

Twitter indexing was basically real-time - you could see a tweet within a few minutes of it being made Mastodon seems to take many days to appear, if at all. I suspect Google just scrapes the webui occasionally and doesn't use any push-based mechanism for notifications. That probably means it misses some too.

HN is indexed in effectively real time as well. I've googled several phrases I've read here, hoping to find more information, where the HN comment of interest is the sole result.

Googled "HN is indexed in effectively real time as well" and your comment is the top result

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#43

Post by bdcravens »

benevol wrote: post #33

Google has become a major force of censorship. Twitter (under Elon Musk) is now opposed to that. That should explain what you experience.

Censorship seems alive and well on Twitter under Musk.

https://www.motherjones.com/media/2022/12/elon-musk-twitter-...

https://www.nytimes.com/2022/12/16/business/media/elon-musk-...

https://www.msnbc.com/opinion/msnbc-opinion/elon-musk-twitte...

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#44

Post by dpedu »

I am finding that this is not the case when I'm searching.

Yesterday I read this tweet (below) that talks about some military history relevant to current events. I wanted to read more so I googled `"1957" china balloon` and the first hit was the very same tweet. Testing now, searching for a substring from the tweet in quotes works too.

https://twitter.com/NavalInstitute/status/162163973828837376...

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#45

Post by bdcravens »

cma wrote: post #3

They shouldn't index stuff with a login or pay wall anyway, or should put it in a separate tab of the results or something (that is, other than in countries where they were forced to).

Under the same standards, HN (and many other sites) wouldn't be indexed either.

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#46

Post by consumer451 »

bastawhiz wrote: post #40
jahewson wrote: post #35

Next week, Twitter is starting to charge for their API https://arstechnica.com/tech-policy/2023/02/cash-strapped-tw... Could be related? Even if they’re just crawling the public pages I could certainly imagine Elon trying to extract money from Google for the privilege.

My understanding was that Google was paying for its firehose access already. Though I suppose they could be forcing a contract renegotiation.

> Though I suppose they could be forcing a contract renegotiation.

This was my attempt at an explanation. I wonder who will blink first and if the public will ever learn the details.

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#47

Post by waboremo »

29athrowaway wrote: post #26
bthrn wrote: post #20

Earlier quoted context omitted.

It’s heavily dependent on the quality of the search offered. For example — Reddit’s search functionality is terrible. I’ve found that Google is much better for searching Reddit.

Or Amazon. Amazon's search sucks. Results are often completely irrelevant, or dismissive of the search terms. You search for a refrigerator filter from a known refrigerator manufacturer and the first results you get are the toxic offbrand refrigerator filters from companies with randomly generated company names that poison your entire family.

That combined with Amazon's aggressive category filtering is annoying. Search a generic term like rings, it forces you into the clothing category without the ability to switch. You have to then slap on more keywords in hopes it navigates to the right category, because manually changing the category while including generic searches does not work. It does not matter if you first went to the category you believe your search to be in.

Somehow Amazon has managed to make the bloated Aliexpress experience seem more straight forward and easy to navigate.

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#48

Post by ahuify »

bthrn wrote: post #20
LarryMullins wrote: post #19

I never use general-purpose search engines if I already know which website the thing I'm looking for is on (assuming that website has it's own search functionality.) If I am looking for a wikipedia article, I go to wikipedia and search for it there. If I am looking for a tweet, I go to twitter and search for it there. If I'm searching for something on HN, then it's algolia. Search keywords in firefox make this easy.…

It’s heavily dependent on the quality of the search offered. For example — Reddit’s search functionality is terrible. I’ve found that Google is much better for searching Reddit.

You may get better results searching the Pushshift archive, e.g. via https://camas.unddit.com. It will search deleted and removed items too.

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#49

Post by 29athrowaway »

waboremo wrote: post #47
29athrowaway wrote: post #26

Earlier quoted context omitted.

Or Amazon. Amazon's search sucks. Results are often completely irrelevant, or dismissive of the search terms. You search for a refrigerator filter from a known refrigerator manufacturer and the first results you get are the toxic offbrand refrigerator filters from companies with randomly generated company names that poison your entire family.

That combined with Amazon's aggressive category filtering is annoying. Search a generic term like rings, it forces you into the clothing category without the ability to switch. You have to then slap on more keywords in hopes it navigates to the right category, because manually changing the category while including generic searches does not work. It does not matter if you first went to the category you believe your se…

Amazon, despite having a high bar for hiring, has fallen victim to one of the most trivial forms of SEO.

Companies create multiple subsidiaries with randomly generated names, so they can spam search results and faceted results with their products and exclude other vendors. Search for "flying bird cat toy" and you'll see what I mean.

Much of that could be solved using a filter by country of origin. Because the usual suspects are always from the same territory.

Why is this necessary? because shell companies sell products loaded with lead, cadmium, phthalates, formaldehyde, VOCs, and other highly toxic stuff.

Re: Tell HN: Tweets no longer seem to be properly indexed by Google

#50

Post by klyrs »

bdcravens wrote: post #42
klyrs wrote: post #37

Earlier quoted context omitted.

HN is indexed in effectively real time as well. I've googled several phrases I've read here, hoping to find more information, where the HN comment of interest is the sole result.

Googled "HN is indexed in effectively real time as well" and your comment is the top result

Delta of 7 minutes. I thought about putting in a nonsense phrase to test the hypothesis, but apparently that wasn't even necessary. Spooky.
Post reply on HN