Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I rebuilt my blog's cache. Bots are the audience now

hoeijmakers.net

Post reply on HN
Print view View original

51–60 of 64 posts

Re: I rebuilt my blog's cache. Bots are the audience now

#51

Post by tardedmeme »

Barbing wrote: post #20
microtonal wrote: post #15

Earlier quoted context omitted.

I also block all AI crawlers. I am not sure why I should give them my content for them to rip it off and make money from it through training or agents. Sadly, a lot of AI companies are trying to make requests indistinguishable from regular browsers from residential connections, so unfortunately I have to use Cloudflare to block them. Ideally I'd make the content available to crawlers for training open models, but tha…

>so unfortunately I have to use Cloudflare to block them. That can’t block Grok, can it? (You might have a fake iPhone or something visit your site if you ask Grok to retrieve information from it)

What's the IP address of the supposed iPhone? Does it come from T-Mobile or from xAI?

Re: I rebuilt my blog's cache. Bots are the audience now

#53

Post by monkpit »

cultofmetatron wrote: post #43
01284a7e wrote: post #10

Earlier quoted context omitted.

I pulled all the websites I had - some existed for a decade plus and made me hundreds of thousands of dollars. All that is left is bots that theft the value of my work. Until something changes, goodbye.

I have no problem with these AI summaries but someone needs to make a law requiring the AI provider to generate links to the primary source as part of the output. If a human writer ripped off someones work and didn't attribute, it would be a huge scandal. when an AI does it, its expected.

Because of the way training works, is this possible/feasible? Not saying it shouldn’t be done, just wondering _how_ it could be done?

Re: I rebuilt my blog's cache. Bots are the audience now

#54

Post by faangguyindia »

tardedmeme wrote: post #49
robhoeijmakers wrote: post #35

Earlier quoted context omitted.

It is good to make a proper distinction, in the ChatGPT context, between crawlers and agents. The crawlers go for the content to build a new model, the agents serve content to users. The last one can be very useful.

They use different user-agent strings. The crawlers obfuscate themselves and use residential proxies. The agents call themselves ChatGPT-User. Of course Cloudflare wants OpenAI to pay them for not blocking ChatGPT-User by default.

It's true, crawlers used for AI training don't say they are crawlers at all.

Re: I rebuilt my blog's cache. Bots are the audience now

#55

Post by Barbing »

tardedmeme wrote: post #51
Barbing wrote: post #20

Earlier quoted context omitted.

>so unfortunately I have to use Cloudflare to block them. That can’t block Grok, can it? (You might have a fake iPhone or something visit your site if you ask Grok to retrieve information from it)

What's the IP address of the supposed iPhone? Does it come from T-Mobile or from xAI?

Residential I thought? It might’ve been even someone on here who posted about watching their server logs while they messaged Grok themselves.

Curious if xAI has a phone farm. Maybe just running simulators on servers?

Re: I rebuilt my blog's cache. Bots are the audience now

#56

Post by lenkite »

jdw64 wrote: post #33
ryandrake wrote: post #28

Earlier quoted context omitted.

I think in general, "Writing on the Internet with the intent to make money" is effectively dead, or at least soon to be dead. AI+bots mean we now have the "infinite typewriter monkeys" from the thought experiment. With infinite supply, the price goes to zero. We need to stop this treadmill of trying to "build reputation" and stop focusing on "symbolic capital" and "clout" and whatever else bloggers are going after. Y…

I am not trying to make money from writing. What I need is for my writing to spread enough that I can receive opportunities to have my programming ability evaluated. The reason I write about programming is that, in the past, some readers found my programming essays interesting, and that led to chances for me to be tested. I had to leave graduate school because of financial problems, and I did not graduate from a pres…

The world-wide web has been unfortunately been conquered, annexed and is being burned by the Tyrannical AI Empire. Until a Pure-Humans-Only alternative resistance movement and platform arises from its sordid ashes, we will all need to bow our heads and kneel before the God-Emperor.

Re: I rebuilt my blog's cache. Bots are the audience now

#57

Post by gbgarbeb »

tardedmeme wrote: post #47
01284a7e wrote: post #10

Earlier quoted context omitted.

I pulled all the websites I had - some existed for a decade plus and made me hundreds of thousands of dollars. All that is left is bots that theft the value of my work. Until something changes, goodbye.

You might as well serve different content to bots. Incorrect content.

Why not just redirect all requests to your website to the Reddit frontpage?

Re: I rebuilt my blog's cache. Bots are the audience now

#58

Post by mmarian »

jdw64 wrote: post #2

Personally, I think this is a good idea. But the core problem is this: How is a newcomer supposed to build reputation now? Without exaggerated business promises or capital, basic online reputation usually depends on writing. In fact, my own first step into freelancing came because someone found the articles on my Korean blog interesting. So the question is: if the subscribers are bots, what benefit do they actually g…

By syndicating everywhere you can and inviting discussions (Reddit, HM, etc). The reality is that your blog post would've rarely been found anyway due to the SEO blog spam.

I managed to get some great discussions through sharing my posts on Reddit and HN.

Re: I rebuilt my blog's cache. Bots are the audience now

#59

Post by nicbou »

jdw64 wrote: post #2

Personally, I think this is a good idea. But the core problem is this: How is a newcomer supposed to build reputation now? Without exaggerated business promises or capital, basic online reputation usually depends on writing. In fact, my own first step into freelancing came because someone found the articles on my Korean blog interesting. So the question is: if the subscribers are bots, what benefit do they actually g…

It's already happening. My traffic is down 60-70% from its peak, because Google serves answers from my website directly before it. ChatGPT picks up the rest.

I am losing motivation to put out hard-earned information if it's all going to get stolen. I'm just training 3 companies to better capture the value of my hard work. Why even bother?

I live from that website. I have poured 9 years of my life into it. It's known by everyone locally. In a year or two I won't be able to cover my bills anymore. I could go back to my old job, but it's also getting automated.

I'm just so tired.

Re: I rebuilt my blog's cache. Bots are the audience now

#60

Post by 01284a7e »

gbgarbeb wrote: post #19
01284a7e wrote: post #10

Earlier quoted context omitted.

I pulled all the websites I had - some existed for a decade plus and made me hundreds of thousands of dollars. All that is left is bots that theft the value of my work. Until something changes, goodbye.

This is like choosing to be an elementary school teacher and then quitting because it turns out your students for the year aren't your pets in perpetuity.

No, it's not at all. Bizarre analogy.
Post reply on HN