Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs as the new high level language

federicopereiro.com

Post reply on HN
Print view View original

351–360 of 396 posts

Re: LLMs as the new high level language

#351

Post by pjmlp »

cesarb wrote: post #292
pjmlp wrote: post #253

Earlier quoted context omitted.

I certainly already experienced crashes due to JIT miscompilations, even though it was a while back, on Websphere with IBM Java implementation. Also it is almost impossible to guarantee two runs of an application will trigger the same machine code output, unless the JIT is either very dumb on its heuristics and PGO analysis, or one got lucky enough to reproduce the same computation environment.

> Also it is almost impossible to guarantee two runs of an application will trigger the same machine code output As long as the JIT is working properly, it shouldn't matter: the code should always run "as if" it was being run on an interpreter. That is, the JIT is nothing more than a speed optimization; even if you disable the JIT, the result should still be the same.

Same with agent actions.

Re: LLMs as the new high level language

#352

Post by pjmlp »

dgb23 wrote: post #299
pjmlp wrote: post #230

Earlier quoted context omitted.

Anyone doing benchmarks with managed runtimes, or serverless, knows it isn't quite true. Which is exactly one of the AOT only, no GC, crowds use as example why theirs is better.

But there is functional equivalence. While I don't want to downplay the importance of performance, we're talking about something categorically different when comparing LLMs to compilers.

Not when those LLMs are tied to agents, replacing what would be classical programming.

Using low code platforms with AI based automations, like most iPaaS are now doing.

If the agent is able to retrieve the required data from a JSON file, fill an email with the proper subject and body, sending it to another SaaS application, it is one less integration middleware that was required to be written.

For all practical business point of view it is an application.

Re: LLMs as the new high level language

#353

Post by pjmlp »

manuelabeledo wrote: post #283
pjmlp wrote: post #256

Earlier quoted context omitted.

Are you certain to predict the JIT generated machine code given the JVM bytecode? Without taking anything else into account that the JIT uses on its decision tree?

For a single execution, to a certain extent, yes. But that’s not the point I’m trying to make here. JIT compilers are vastly more predictable than LLMs. I can take any two JVMs from any two vendors, and over several versions and years, I’m confident that they will produce the same outputs given the same inputs, to a certain degree, where the input is not only code but GC, libraries, etc. I cannot do the same with two…

Good luck mapping OpenJDK with Azul's cloud JIT, in generated machine code.

Re: LLMs as the new high level language

#354

Post by amelius »

layer8 wrote: post #321
amelius wrote: post #302

Earlier quoted context omitted.

Consider there are 100 upvotes and 100 downvotes. Net votes: 0. The submission would end up with a lower ranking that you wanted it to have.

Submissions don’t have downvotes, only flagging.

Fair. But it still holds for the comments section.

Re: LLMs as the new high level language

#355

Post by redox99 »

skaul wrote: post #347

Programming with LLMs is fundamentally different than going from a lower-level to a higher-level language, even apart from the whole non-determinism thing. With a programming language, you're still writing a for-loop, whether that's in C, Java or Rust. There's language primitives that help you think better in certain languages, but they're still, at the end of the day, code and context that you have to hold in your h…

A programming language does not need to have a for loop. In fact many don't.

Re: LLMs as the new high level language

#356

Post by JackSlateur »

hackyhacky wrote: post #342
JackSlateur wrote: post #327

Earlier quoted context omitted.

We have heard that for years "trust us, it will work soon .. we just need a bit more time and a couple more dozens billions of dollars .. just trust us, bro .."

> We have heard that for years LLMs have been a thing for about three years now, so you can't have been hearing this for very long. In those three years, the rate of progress has been astounding and there is no sign of slowing down.

And god knows that we have heard that a lot in just 3 years.

Re: LLMs as the new high level language

#357

Post by redox99 »

_zagj wrote: post #193
toprerules wrote: post #2

After working with the latest models I think these "it's just another tool" or "another layer of abstraction" or "I'm just building at a different level" kind of arguments are wishful thinking. You're not going to be a designer writing blueprints for a series of workers to execute on, you're barely going to be a product manager translating business requirements into a technical specification before AI closes that gap…

> After working with the latest models I think these "it's just another tool" or "another layer of abstraction" or "I'm just building at a different level" kind of arguments are wishful thinking. You're not going to be a designer writing blueprints for a series of workers to execute on, you're barely going to be a product manager translating business requirements into a technical specification before AI closes that g…

Gen AI for art was different because it would just output a final image with basically 0 control for the artist. It's like if AI programming would output a binary instead of source code.

Re: LLMs as the new high level language

#358

Post by hndc »

mike_hearn wrote: post #199
hndc wrote: post #95

Earlier quoted context omitted.

Deterministic compilation, aka reproducible builds, has been a basic software engineering concept and goal for 40+ years. Perhaps you could provide some examples of compilers that produce non-deterministic output along with your bad news.

JIT compilers.

Compiler artifact is still deterministic. Clearly not referring to runtime behavior that is input-dependent

Re: LLMs as the new high level language

#359

Post by onlyrealcuzzo »

manuelabeledo wrote: post #338
onlyrealcuzzo wrote: post #336

Earlier quoted context omitted.

You're going to pretty hard pressed to do Rust better than Rust. There's minimal opportunity with lifetime annotations. I'm sure very small options elsewhere, too. The idea of replacing Rust with natural language seems insane. Maybe I'm being naive, but I can't see why or how it could possibly be useful. Rust is simply Chinese unless you understand what it's doing. If you translate it to natural language, it's still…

I believe the author thinks of this problem in terms of “the LLM will figure it out”, i.e. it will be trained on enough code that compiles, that the LLM just needs to put the functional blocks together. Which might work to a degree with languages like JavaScript.

That point makes no sense.

If the LLM is not perfect at scale - extraordinarily unlikely that it would be - then it becomes relevant to understand the actual language.

That's either natural language that's supposed to somehow be debuggable - or it's a language like Rust - which actually is.

Re: LLMs as the new high level language

#360

Post by devsda »

afavour wrote: post #317
heikkilevanto wrote: post #226

If we consider the prompts and LLM inputs to be the new source code, I want to see some assurance we get the same results every time. A traditional compiler will produce a program that behaves the same way, given the same source and options. Some even go out of their way to guarantee they produce the same binary output, which is a good thing for security and package management. That is why we don't need to store the…

> If we consider the prompts and LLM inputs to be the new source code, I want to see some assurance we get the same results every time. There’s a related issue that gives me deep concern: if LLMs are the new programming languages we don’t even own the compilers. They can be taken from us at any time. New models come out constantly and over time companies will phase out older ones. These newer models will be better, s…

> New models come out constantly and over time companies will phase out older ones. These newer models will be better, sure, but their outputs will be different.

IIUC, same model with same seed and other parameters is not guaranteed to produce the same output.

If anyone is imagining a future where your "source" git repo is just a bunch of highly detailed prompt files and "compilation" just needs an extra LLM code generator, they are signing up for disappointment.

Post reply on HN