Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

531–540 of 648 posts

Re: GPT-4 details leaked?

#531

Post by mensetmanusman »

nix0n wrote: post #440
mensetmanusman wrote: post #319

Earlier quoted context omitted.

{Hypothesis, test, loop} is the scientific method, and I can guarantee it is being used when fine tuning an LLM.

The reason that this is Engineering as opposed to Science, is that the hypothesis is just, "hey, maybe this will work". Nobody has really explained why it works.

This is my opinion, so take with a grain of salt:

I believe it's entirely possible that the scientific method breaks down past a certain level of system complexity defined somehow by thermodynamics.

This would in part be due to the infeasibility of running the proper experiments to understand the effects of single variables when tens of thousands of variables might be changing at the micro level.

Re: GPT-4 details leaked?

#532

Post by optimalsolver »

machina_ex_deus wrote: post #525
YeGoblynQueenne wrote: post #479

Earlier quoted context omitted.

Science is not academia. You say you prefer to judge things as "what you can do". Well, how do you judge "what you can do"? Astrologists, homeopaths, podiatrists, Christian scientists (!!!) and other such "heretics and mad men" rejected by the scientific establishment, will all tell you that they "can do" stuff, and so will all their many paying customers. How do we know they can't do what they say? Because science g…

Placebos are known to work. I neither overestimate nor underestimate them. I understand them for what they are: placebos. And some people actually need them. Moreover, in social contexts, religion also "works" in many senses. A person asked me what me what can he do about depression and feeling of meaninglessness. Science would prescribe anti-depressants, medicate him and he would both have side-effects and the probl…

>There is no mistake in Newton's laws. Or in Einstein's. They are just as valid today as they have been back then, in the areas where they were demonstrated and studied.

A model being incomplete is just another way of saying it's wrong.

The models they came up with are wrong, but still useful.

Re: GPT-4 details leaked?

#533

Post by astrange »

ryneandal wrote: post #361
astrange wrote: post #268

Earlier quoted context omitted.

The data, size and training process are what's different.

The point is LLMs are built with Transformer architecture. They're all transformers, attention is an integral part of building worthwhile, contextual answers.

Yeah, but LLMs are more specific, so it's a subset.

Also, LLMs use "in-context learning" ie you actually ask it "hey, translate this" and then it has a conversation with you where you can ask to clarify or provide word definitions.

Google Translate is more hardcoded; a big problem with it is that it can't explain any of the decisions it's made or show its uncertainty about anything.

Of course, neither of these are reliable since they hallucinate.

Re: GPT-4 details leaked?

#534

Post by ly3xqhl8g9 »

notahacker wrote: post #501
ly3xqhl8g9 wrote: post #399

Earlier quoted context omitted.

Hope at least you are getting paid and you don't deploy such ridiculous amounts of bad faith for free. My reply from above "Most of them already don't get paid, hence the strike" was specifically in the context of the streaming platforms not paying residuals in the same manner the studios do, hence the reference to the article. What I find most funny is that you don't even know or care about alternative solutions, yo…

What I find most funny is that you're implying there are alternative solutions that you do know and care about. And yet instead of addressing my actual objection - that screenwriters seeking to get paid more for co-creating IP would be unlikely to see abolition of IP protection as a solution - by articulating those alternatives and how it would allow them to get paid more, you chose to assert they don't get paid.[1]…

[flagged]

Re: GPT-4 details leaked?

#535

Post by optimalsolver »

javajosh wrote: post #512
sdenton4 wrote: post #485

Earlier quoted context omitted.

Ask Feynman why magnets works... And he'll say no one knows. https://www.youtube.com/watch?v=36GT2zI8lVA It's the same in ML. Good predictions do come from real ideas about how the systems work - ideas in information theory, entropy, cognitive science, statistical mechanics and so on - formulated in a context of our existing understanding and prior results. That's science.

Feynman was a genius. It takes a genius to say something like that, and it's surprising how many "scientists" are upset by statements like that, because they are so proud of their hard-won capabilities in modelling. Their curl and divergence operators, their Maxwell's equations... they forget that the map is not the terrain, and the universe is, at it's root, irreducibly mysterious, and the fact that we understand an…

>irreducibly

What are you basing that on?

Re: GPT-4 details leaked?

#536

Post by ly3xqhl8g9 »

JumpCrisscross wrote: post #497
ly3xqhl8g9 wrote: post #307

Earlier quoted context omitted.

"there is no evidence" As quoted above, "Ithaka's total revenue was $105 million in 2019, most of it ($79 million) from JSTOR service fees". All those $79 million and more JSTOR has stolen from the authors. Although JSTOR should have been dissolved long before this for effectively killing Aaron Swartz. But yes, there is no justice in this world. I suppose that's my main contention, why pretend anymore, we live in a s…

This isn’t a cogent argument. You’re identifying troubling behaviour. But it’s not being stitched into anything cohesive. Hiding bad rhetoric behind post-modernist nihilism is in vogue, but unproductive.

Not sure where you picked nihilism in my reply, referencing the "we live in a society" meme I actually smiled. Nevertheless, stopping the pretence today, painful as it is, realizing the state of the world is far from perfect, is the first step for a better tomorrow.

Re: GPT-4 details leaked?

#537

Post by stOneskull »

Xeoncross wrote: post #359
michaelcampbell wrote: post #356

Earlier quoted context omitted.

> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. This is essentially the Capitalist Credo, expressed in practical vs theoretical terms.

> pressure the government into making it Not really, Capitalism is free-trade between two parties. The more government involvement you have the more it moves towards socialism or communism where the government controls trade. At least this was the historical meaning. These days Capitalism is being redefined to mean private (non-government) Communism. That is, power concentrated in the hands of a few.

> Capitalism is being redefined to mean private (non-government) Communism

i've heard this called super-capitalism, yeah.

Re: GPT-4 details leaked?

#538

Post by amelius »

swexbe wrote: post #324
amelius wrote: post #305

Earlier quoted context omitted.

Not sure what you are saying here. Perhaps an analogy helps. Psychology is a science. You can make falsifiable statements about the human brain. You will need experiments to build and test theories. It's the same with deep learning. With computer "science" (and math) it's not the same. You can reason completely about your subjects, i.e. you can determine if something will or will not work just by reasoning, no experi…

Seems like the distinction is mainly between which tools are available to you as a scientist (at least if we stick to comp-sci, math is in a league of it's own). When, or if, we can completely model a human brain, a psychologist would no longer need to perform experiments to test their theories. Given enough computing power, most theories could theoretically by proven or falsified purely through reasoning.

The point is: being able to run a brain inside a computer is not the same as understanding that brain. If you wanted to build a new brain, you'd have to reach for the tool all the time in an iterative way and hope for the best. Only tools that aid in understanding matter. We have only very few tools that help DL researchers better understand what they are doing. Hence DL is more akin towards science than towards math/CS or engineering.

Re: GPT-4 details leaked?

#539

Post by stOneskull »

KingMob wrote: post #481
Xeoncross wrote: post #405

Earlier quoted context omitted.

There is no free market without Capitalism. Capitalism is where the private citizens controls the capital. Not the government. It's not possible to have a free market without competitive markets, price systems, private property, property rights recognition, and voluntary exchange.

??? Bazaars existed for millennia before capitalism arose. > Capitalism is where the private citizens controls the capital. Well, that's a tautology, but based on the examples you cite in the next sentence, you're mixing up "property" with "capital". Capital is specifically the means of production. All capital is property, but not all property is capital. A factory filled with machinery is capital; your toothbrush is…

the word predates karl marx but he made it popular, and the meaning usually is referring to his use. i think it's great there are a million writers with different ideas and i wish marxists would read more of them instead of more of him and his other cult members. they should try and write a manifesto too. it's easy. write two. then three works of fiction each with a different utopia.

Re: GPT-4 details leaked?

#540

Post by hyperthesis »

sdenton4 wrote: post #511
hyperthesis wrote: post #499

Earlier quoted context omitted.

Oh, they worked really well; they were still used for numerical calculation even after Galileo. But they weren't a "demonstration of knowledge and understanding" of planetary motion.

Understanding is a moving target, though. Newtonian mechanics was incorrect for modeling the solar system as well - as it was eventually superseded - but that doesn't mean it wasn't a scientific understanding of planetary motion. Epicycles gave a better description of the movement of the planets, based on the observations available, which was entirely falsifiable. They were eventually superseded, and that's science a…

Interesting to say it's scientific because falsifiable. The objection is that it wasn't a theory, just fitting a function to data. It did "work" in that it captured some pattern: it was extremely good at generalizing/extrapolating/predicting. And was a "model" of something in the data. But there was no operational model behind it, of what was actually happening.

Newtonian mechanics has a model, beyond curve fitting.

BTW: I made this analogy between LLM and epicycles as a joke, but it's looking strangely isomorphic...

It's funny, because epicycles used to be the poster-child for ad hoc, overcomplex models with no conceptual basis... which is an exact match for LLM... but I don't recall the analogy being made. Not even in https://norvig.com/chomsky.html

Post reply on HN