GPT-5 is behind schedule • Hacker News Board

Re: GPT-5 is behind schedule

#841

Post by ammanley » Mon, Dec 23, 2024, 5:36 PM UTC

EVa5I7bHFq9mnYK wrote: post #791

jpablo wrote: post #753

Earlier quoted context omitted.

They aren't blocking anything. They are just asking nicely not to be crawled. Given that AI companies haven't cared a single bit about ripping of other's peoples data I don't see why they would care now.

In their attempt to block OpenAI, they block me. Many sites that were accessible just 2 years ago, require login/captchas/rectal exam now just to read the content.

Im looking forward to the life experience that is content I want to read badly enough to endure a rectal exam.

Re: GPT-5 is behind schedule

#842

Post by sampullman » Mon, Dec 23, 2024, 5:36 PM UTC

nycdatasci wrote: post #723

sampullman wrote: post #701

Earlier quoted context omitted.

That's basically my experience. It's great for learning or getting things done when the subject is related to one you know well (i.e. you understand the fundamentals and can verify responses quickly). It's not so good for a completely new subject, or one you have a lot of experience in.

They're not useful when you're completely new to a subject? On the contrary, I've found that they are excellent when you have limited knowledge in a domain and less useful when you have expertise. They have allowed me to go from zero to functioning MVP on numerous computer vision projects, even though I have zero experience.

Do you have programming experience? I probably didn't explain well, what I meant is that if you have zero relevant experience it can be difficult to verify correctness.

For example, I'm comfortable with frontend development but hadn't used webworkers or websockets. ChatGPT was useful for getting up to speed quickly. I've had less luck with topics that are completely new to me, one example is coming up with a training regimen for long distance running. I have to manually verify every little thing, which ends up taking longer than doing research the old fashioned way.

I'd be surprised if you could go from zero to a useful CV app with LLMs, but it's possible I just haven't given it a fair shake.

Re: GPT-5 is behind schedule

#843

Post by parineum » Mon, Dec 23, 2024, 5:36 PM UTC

gizmo wrote: post #686

netdevphoenix wrote: post #640

Earlier quoted context omitted.

> The ability to “talk to an expert” about any topic I’m curious about and ask very specific questions has been invaluable to me. It is dangerous to assume that LLMs are experts on any topic. With or without quotes. You are getting a super fast journalist intern with a huge memory but inability to reason critically, lacking understanding about anything and huge unreliability when it comes to answering questions (you…

People who are experts (PhD and 20 years of experience) often have very dumb opinions in their field of expertise. Experts make amateur mistakes too. Look at the books written by expert economists, expert psychologists, expert historians, expert philosophers, expert software engineers. Most books are not worth the paper they're written on, despite the authors being experts with decades of experience in their respecti…

> People who are experts (PhD and 20 years of experience) often have very dumb opinions in their field of expertise.

And the training data contains all those dumb opinions.

Re: GPT-5 is behind schedule

#844

Post by kyledrake » Mon, Dec 23, 2024, 5:40 PM UTC

cedws wrote: post #751

jaybna wrote: post #741

25% of the top 1000 websites are blocking OpenAI from crawling: https://originality.ai/ai-bot-blocking I am betting hundreds of thousands, rising to millions more little sites, will start blocking/gating this year. AI companies might license from big sources (you can see the blocking percentage went down), but they will be missing the long tail, where a lot of great novel training data lives. And then the big sites w…

Cloudflare has a toggle for blocking AI scrapers. I don’t think it’s default, but it’s there.

This just feels like mystery meat to me. My guess is that a lot of legitimate users and VPNs are being blocked from viewing sites, which numerous users in this discussion have confirmed.

This seems like a very bad way to approach this, and ironically their model quite possible also uses some sort of machine learning to work.

A few web hosting platforms are using the cloudflare blocker and I think it's incredibly unethical. They're inevitably blocking millions of legitimate users from viewing content on other people's sites and then pretending it's "anti AI". To paraphrase Theo Deraadt, they saw something on the shelf, and it has all sorts of pretty colours, and they bought it.

Re: GPT-5 is behind schedule

#845

Post by zppln » Mon, Dec 23, 2024, 5:40 PM UTC

tiffanyh wrote: post #800

SamPatt wrote: post #415

I'm sure the debate over the definition of AGI is important and will continue for a while, but... I can't care about it anymore. Between Perplexity searching and summarizing, Claude explaining, and qwen (and other tools) coding, I'm already as happy as can be with whatever you want to call this level of intelligence. Just today I used a completely local AI research tool, based on Ollama. It worked great. Maybe it won…

I have the opposite reaction. AI right now feels like that MBA person at work. They don’t know anything. But because they sound like they are speaking with authority & confidence, allows them to get promoted at work. (While all of the experts at work roll their eyes because they know the MBA/AI is just spitting out nonsense & wish the company never had any MBA/AI people)

And the MBA person (at my company this is everyone in middle management) is also the person who go around and suggest we shoehorn AI into everything...

Re: GPT-5 is behind schedule

#846

Post by pncnmnp » Mon, Dec 23, 2024, 5:46 PM UTC

glenstein wrote: post #809

jaybna wrote: post #741

25% of the top 1000 websites are blocking OpenAI from crawling: https://originality.ai/ai-bot-blocking I am betting hundreds of thousands, rising to millions more little sites, will start blocking/gating this year. AI companies might license from big sources (you can see the blocking percentage went down), but they will be missing the long tail, where a lot of great novel training data lives. And then the big sites w…

>Bill Gross correctly calls this phase of AI shoplifting. I call it the Napster-of-Everything (because I am old). I am also betting that the courts won't buy the "fair use" interpretation of scraping, given the revenues AI companies generate. That means a potential stalling of new models until some mechanism is worked out to pay knowledge creators. To your point, I have wondered whatever became of that massive initia…

> I have wondered whatever became of that massive initiative from Google to scan books, and whether that might be looked at as a potential training source, giving that Google has run into legal limitations on other forms of usage.

A few months ago, there was an interesting submission on HN about this - The Tragedy of Google Books (2017) (https://news.ycombinator.com/item?id=41917016).

Re: GPT-5 is behind schedule

#847

Post by Workaccount2 » Mon, Dec 23, 2024, 5:52 PM UTC

EricMausler wrote: post #794

Workaccount2 wrote: post #765

Earlier quoted context omitted.

Doing basic copyright analyses on model outputs is all that is needed. Check if the output contains copyright, block it if it does. Transformers aren't zettabyte sized archives with a smart searching algo, running around the web stuffing everything they can into their datacenter sized storage. They are typically a few dozen GB in size, if that. They don't copy data, they move vectors in a high dimensional space based…

No comment on if output analysis is all that is needed, though it makes sense to me. Just wanted to note that using file size differences as an argument may simply imply transformers could be a form of (either very lossy or very efficient) compression.

You can argue any form of data is an arbitrarily lossy compression of any other form of data.

I get your point, but nobody is archiving their companies 50 years of R&D data with and LLM so they can get it down to 10GB.

They may have traits of data compression, but they are not at all in the class of data compression software.

Re: GPT-5 is behind schedule

#848

Post by RationPhantoms » Mon, Dec 23, 2024, 5:53 PM UTC

pphysch wrote: post #840

cshores wrote: post #815

Earlier quoted context omitted.

It ultimately doesn't matter because a fairly current snapshot of all of the world's information is already housed in their data lakes. The next stage for AI training is to generate synthetic data either by other AI or by simulations to further train on as human generated content can only go so far.

How is synthetic data supposed to work? Broadly speaking, ML is about extracting signal from noisy data and learning the subtle patterns. If there is untapped signal in existing datasets, then learning processes should be improved. It does not follow that there should be a separate economic step where someone produces "synthetic data" from the real data, and then we treat the fake data as real data. From a scientific…

Would you trust a ML self-driving algorithm trained on a "digital twin" of a city? I would. I view synthetic training data like a digital twin in which it can provider further control or specified noise to understand from.

Re: GPT-5 is behind schedule

#849

Post by pixl97 » Mon, Dec 23, 2024, 5:58 PM UTC

5h wrote: post #831

dageshi wrote: post #814

Earlier quoted context omitted.

If it reaches the point where google is no longer a useful driver of traffic then there's probably little point in having a website at all any more.

Strange take ... I seem to remember websites having a lot of point before google.

They had a lot or point because....

1. They were a major site that was an initial starting point for traffic

2. Search engines pointed to them and people could locate them.

---

That was all a long time ago. Now people tend to go to a few 'all in one sites'. Google, reddit, '$big social media'. Other than Google most of those places optimize you to stay on that particular site rather than go to other people's content. The 'web' was a web of interconnectedness. Now it's more like a singularity. Once you pass the event horizon of their domain you can never escape again.

Re: GPT-5 is behind schedule

#850

Post by pixl97 » Mon, Dec 23, 2024, 6:01 PM UTC

kyledrake wrote: post #844

cedws wrote: post #751

Earlier quoted context omitted.

Cloudflare has a toggle for blocking AI scrapers. I don’t think it’s default, but it’s there.

This just feels like mystery meat to me. My guess is that a lot of legitimate users and VPNs are being blocked from viewing sites, which numerous users in this discussion have confirmed. This seems like a very bad way to approach this, and ironically their model quite possible also uses some sort of machine learning to work. A few web hosting platforms are using the cloudflare blocker and I think it's incredibly unet…

> I think it's incredibly unethical.

The internet isn't built on ethical behavior, unfortunately.