GPT-4 details leaked? • Hacker News Board

Re: GPT-4 details leaked?

#631

Post by fragmede » Thu, Jul 13, 2023, 4:50 PM UTC

londons_explore wrote: post #384

fragmede wrote: post #186

Earlier quoted context omitted.

Google Classroom, teenager's essays, written by humans, for learning what it means to be human, and graded by humans, is a richer dataset than anything else I can think of that anyone else couldn't get their hands on.

An awful lot of teachers can grade a 10 page essay in about 90 seconds... Skim read it, mark out some grammar errors, assign it a grade based on the quality of the opening and closing paragraphs.

Yup, and they're doing it the whole country over, and putting that data in to Google Classrooms for Bard to know "this is C-grade work" and "this is A-grade work". Knowing what's deemed good and bad writing is where I'm thinking this dataset shines for training LLMs.

Re: GPT-4 details leaked?

#632

Post by ShamelessC » Thu, Jul 13, 2023, 6:14 PM UTC

natpalmer1776 wrote: post #626

ShamelessC wrote: post #621

Earlier quoted context omitted.

I don’t appreciate the many bad-faith assumptions made. in particular the assertion that I’m not from a “strong academic background”. For what it’s worth, I made a mistake in a public forum, admitting to this twice. I’ve sought accurate versions of my response which no one provided. Nevertheless I have continued to educate myself on the subject and only feel more confident that I wasn’t misinformed to the degree you…

I take it all back. The comment I responded to appears to be both correct in assertion and implication.

Okay then. Best of luck with all of that. There’s a forum called LessWrong you would likely be interested in.

Re: GPT-4 details leaked?

#633

Post by natpalmer1776 » Thu, Jul 13, 2023, 6:21 PM UTC

ShamelessC wrote: post #632

natpalmer1776 wrote: post #626

Earlier quoted context omitted.

I take it all back. The comment I responded to appears to be both correct in assertion and implication.

Okay then. Best of luck with all of that. There’s a forum called LessWrong you would likely be interested in.

I'm not the biggest fan of LessWrong, I do however refer new developers who seem interested to the "Rationality: From AI to Zombies" sequences to help refine conscious development of rational thinking.

Also, I absolutely loved the fanfic "Harry Potter and the Methods of Rationality" written by the forum's creator.

** Edit **

And for the record, my comment wasn't intended to be an accurate depiction of you specifically, which honestly wasn't very effectively conveyed.

It was to highlight a common 'type of person' who make authoritative statements on areas they're not 'experts' in, through no malice on their part, rather as a function of their default mode of behavior.

As others who replied to me highlighted, there are at least two of these people in the world and I wanted others to at least be aware of their existence and point of view; the end goal of this being they might offer others the benefit of the doubt and perhaps some constructive feedback instead of unproductive criticism in similar exchanges.

In essence, I thought other comments were being too hard on you and wanted to point out a potential scenario in which their critiques were at at best unproductive.

Re: GPT-4 details leaked?

#634

Post by Exuma » Thu, Jul 13, 2023, 7:31 PM UTC

Shocka1 wrote: post #630

Exuma wrote: post #591

Earlier quoted context omitted.

You have clarified something I have always thought about very intensely and deeply but haven’t really ever read anyone else who understands that so well or rather put it into words so clearly. I’m an inferenced based learner to an extreme and it definitely has many upsides and also downsides. The upsides are being able to learn extremely rapidly by making connections between pieces of information where there’s gaps a…

This was very well said and I don't know if you could have said it any better. FWIW, I'm a person with multiple degrees in CS, but the best programmers I've worked with and who get stuff done have zero degrees. I have eight years of hardcore programming experience to include professional and side project stuff - I've learned more actually doing than in any classroom. Yeah it's cool to know what a bubble sort is and h…

Thank you, I really appreciate it.

Re: GPT-4 details leaked?

#635

Post by Exuma » Thu, Jul 13, 2023, 7:33 PM UTC

natpalmer1776 wrote: post #629

Exuma wrote: post #591

Earlier quoted context omitted.

You have clarified something I have always thought about very intensely and deeply but haven’t really ever read anyone else who understands that so well or rather put it into words so clearly. I’m an inferenced based learner to an extreme and it definitely has many upsides and also downsides. The upsides are being able to learn extremely rapidly by making connections between pieces of information where there’s gaps a…

I really appreciate your response, thank you for sharing your perspective and experiences. When I read message I can't help but picture you as the storybook 'inventor' who is locked away inside his house with strange colored smoke coming out the chimney & weird noises heard from the street, yet when the doors open the whole town would gather to see what you made.

Haha, that would be me! In some ways you are right about strangeness. I actually work lying down... in my bed. It allows my mind to completely dissolve into the code or problem as if I'm weightless. My business partners all joke that "uhhh yes, the next stop on our tour, well.. this is our CTO's office but its actually just a bed in there so we wont go in that room...." This gives me a good laugh every time

Re: GPT-4 details leaked?

#636

Post by drumttocs8 » Fri, Jul 14, 2023, 12:25 AM UTC

natpalmer1776 wrote: post #524

toxik wrote: post #486

Earlier quoted context omitted.

To be clear, you just made up MoE details while MoE is actually well established and hails from decades old research?

This is common behavior for inference based learners who don’t hail from strong academic backgrounds. Many developers who are self taught utilize a similar method of learning, essentially using pattern recognition to make “educated guesses” that are then internalized as potential facts and tested at the earliest opportunity. In this instance the test was to project the incorrect information out onto a public forum co…

Are you saying that this was a form of Socratic questioning- intentionally presenting an incorrect statement in order to obtain the correction?

Re: GPT-4 details leaked?

#637

Post by drumttocs8 » Fri, Jul 14, 2023, 12:27 AM UTC

ShamelessC wrote: post #51

refulgentis wrote: post #33

Earlier quoted context omitted.

Source?

Just theorizing from the top-level post here. No clue if it's legitimate.

I would have presented it as a question versus a statement, in that case.

Re: GPT-4 details leaked?

#638

Post by natpalmer1776 » Fri, Jul 14, 2023, 12:45 AM UTC

drumttocs8 wrote: post #636

natpalmer1776 wrote: post #524

Earlier quoted context omitted.

This is common behavior for inference based learners who don’t hail from strong academic backgrounds. Many developers who are self taught utilize a similar method of learning, essentially using pattern recognition to make “educated guesses” that are then internalized as potential facts and tested at the earliest opportunity. In this instance the test was to project the incorrect information out onto a public forum co…

Are you saying that this was a form of Socratic questioning- intentionally presenting an incorrect statement in order to obtain the correction?

A bit like that, but without prior knowledge of whether the information is incorrect, rather an intuition that it is correct.

Re: GPT-4 details leaked?

#639

Post by s3p » Fri, Jul 14, 2023, 1:58 AM UTC

mt_ wrote: post #338

shahules wrote: post #13

This guy doesn't have any idea what he is talking about. He consistently posts such bullshit on twitter. Mostly copy paste with added spice mix.

It's Twitter, why would you think otherwise?

I'm tired of people just saying 'oh, internet' every time there is a factual inaccuracy somewhere. Yes, we know this is a social media network. Now can we get back to discussing the topic at hand?

Re: GPT-4 details leaked?

#640

Post by growingkittens » Fri, Jul 14, 2023, 1:35 PM UTC

Exuma wrote: post #591

natpalmer1776 wrote: post #529

Earlier quoted context omitted.

The advantages in a social setting lie in the introduction of entropy, that is _creativity_, to a community. In a rigorous academic setting and with proper training these individuals are more likely identify links between ideas or information that may not seem obvious at first, and tend to be your more 'eccentric' academics. For the interests of the wider group, the best outcome is to help these individuals refine th…

You have clarified something I have always thought about very intensely and deeply but haven’t really ever read anyone else who understands that so well or rather put it into words so clearly. I’m an inferenced based learner to an extreme and it definitely has many upsides and also downsides. The upsides are being able to learn extremely rapidly by making connections between pieces of information where there’s gaps a…

We are generalists, as opposed to specialists. Generalists use information from experts across multiple domains. Specialists are the experts who build a particular domain.

As a culture, we look down on generalists - "A jack of all trades is master of none." However, a world full of specialists creates information silos, where experts solve the same problems over and over in isolation. This is where society is at the moment. We need more generalists to navigate these silos.