Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4

openai.com

Post reply on HN
Print view View original

411–420 of 1001 posts

Re: GPT-4

#411

Post by kranke155 »

Atreiden wrote: post #37

I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…

We're approaching the beggining of the end of the human epoch. Certainly Capitalism won't work or I dont see how it could work under full automation. My view is an economic system is a tool. If an economic system does not allow for utopian outcomes with emerging technology, then it's no longer suitable. It's clear that capitalism was born out of technological and societal changes. Now it seems it's come its time to end.

Re: GPT-4

#412

Post by belter »

Atreiden wrote: post #37

I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…

We can stop being enslaved by these type of AI overlords, by making sure all books, internet pages, and outdoor boards have the same safe, repeated string: "abcdefghjklmnpqrstvxzwy"

That is our emergency override.

Re: GPT-4

#414

Post by AndrewKemendo »

acc_297 wrote: post #229
AndrewKemendo wrote: post #138

Imagine ingesting the contents of the internet as though it's a perfect reflection of humanity, and then building that into a general purpose recommendation system. That's what this is Is the content on the internet what we should be basing our systematic thinking around? No, I think this is the lazy way to do it - by using commoncrawl you've enshrined the biases and values of the people who are commenting and provid…

Yeah looking at the responses they include without using a safety layer it’s pretty clear that the underlying unfiltered model assigns quite a bit of truth to 4chan-esque ideals and values It’s an open question how much of this makes it through the safety layer like if asked to interview job candidates would these undesired biases make it through or are they caught along the way

It means growth is bottlenecked by the terrible data

So the linearly growing safeguards will either stifle the growth of the underlying models

or, more likely

After a certain point people throw their hands up about the guard rails because integrations have obviated people who understand the system and they have no idea how to unwind it

Re: GPT-4

#415

Post by steve76 »

Atreiden wrote: post #37

I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…

[dead]

Re: GPT-4

#416

Post by diimdeep »

ml_basics wrote: post #136

From the paper: > Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar. I'm curious whether they have continued to scale up model size/compute significantly or if they have managed to make significant innovati…

Without paper and architecture, GPT-4 (GPT-3+1) could be just a marketing gimmick to upsell it and in reality it is just microservices of existing A.I models working together as AIaaS (A.I. as a service)

Re: GPT-4

#417

Post by AndrewKemendo »

thomastjeffery wrote: post #322
AndrewKemendo wrote: post #138

Imagine ingesting the contents of the internet as though it's a perfect reflection of humanity, and then building that into a general purpose recommendation system. That's what this is Is the content on the internet what we should be basing our systematic thinking around? No, I think this is the lazy way to do it - by using commoncrawl you've enshrined the biases and values of the people who are commenting and provid…

It's worse: their solution is "guardrails". The problem is that these "guardrails" are laid down between tokens, not subjects. That's simply what the model is made of. You can't distinguish the boundary between words, because the only boundaries GPT works with are between tokens. You can't recognize and sort subjects, because they aren't distinct objects or categories in the model. So what you end up "guarding" is th…

This is very insightful perspective thank you, and it's a very intuitive topological explanation that I hadn't considered!

Re: GPT-4

#418

Post by sva_ »

> gpt-4 has a context length of 8,192 tokens. We are also providing limited access to our 32,768–context (about 50 pages of text) version,

That's a crazy amount of context.

Re: GPT-4

#419

Post by beders »

Atreiden wrote: post #37

I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…

"general thinking" - this algorithm can't "think". It is still a nifty text completion engine with some bells and whistles added.

So many people are falling for this parlor trick. It is sad.

Re: GPT-4

#420

Post by fzliu »

teruakohatu wrote: post #26

Access is invite only for the API, and rate limited for paid GPT+. > gpt-4 has a context length of 8,192 tokens. We are also providing limited access to our 32,768–context (about 50 pages of text) version, gpt-4-32k, which will also be updated automatically over time (current version gpt-4-32k-0314, also supported until June 14). Pricing is $0.06 per 1K prompt tokens and $0.12 per 1k completion tokens. The context le…

One way to get around context length is to perform embedding and retrieval of your entire corpus. Langchain (https://langchain.readthedocs.io/en/latest/) and Milvus (https://milvus.io) is one of the stacks you can use.
Post reply on HN