Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs as the new high level language

federicopereiro.com

Post reply on HN
Print view View original

381–390 of 396 posts

Re: LLMs as the new high level language

#381

Post by whoisthemachine »

ChrisGreenHeur wrote: post #332
whoisthemachine wrote: post #320

Earlier quoted context omitted.

Do these compilers sometimes give correct instructions and sometimes incorrect instructions for the same higher level code, and it's considered an intrinsic part of the compiler that you just have to deal with? Because otherwise this argument is bunk.

they in fact do have bugs, yes, inescapably so (no one provides formal proofs for production level compilers)

Ok, but we treat them as bugs that we can reproduce and assume that they are solvable? We don't just assume that it's intrinsic to the compiler that it must have bugs, and that they will occur in random, non-deterministic ways?

Re: LLMs as the new high level language

#382

Post by whoisthemachine »

pjmlp wrote: post #348
whoisthemachine wrote: post #320

Earlier quoted context omitted.

Do these compilers sometimes give correct instructions and sometimes incorrect instructions for the same higher level code, and it's considered an intrinsic part of the compiler that you just have to deal with? Because otherwise this argument is bunk.

Possibly, hence why the discussion regarding security in JavaScript runtimes and JIT, by completely disabling JIT execution. https://microsoftedge.github.io/edgevr/posts/Super-Duper-Sec... Also the exact sequence of generated machine instructions depends of various factors, the same source can have various outputs, depending on code execution, preset hardware, and heuristics.

Sure, but surely you expect a `func add(a, b) { return a + b; }` to actually produce a + b, in whatever way it finds best? And if it doesn't, you can reproduce the error, and file a bug? And then someone can fix that bug?

Re: LLMs as the new high level language

#383

Post by isodev »

orbital-decay wrote: post #81
isodev wrote: post #38

Earlier quoted context omitted.

> 1:1 reproducibility is much easier in LLMs than in software building pipelines What’s a ‘software building pipeline’ in your view here? I can’t think of parts of the usual SDLC that are less reproducible than LLMs, could you elaborate?

Reproducibility across all existing build systems took a decade of work involving everything from compilers to sandboxing, and a hard reproducibility guarantee in completely arbitrary cases is either impossible or needs deterministic emulators which are terribly slow. (e.g. builds that depend on hardware probing or a simulation result) Input-to-output reproducibility in LLMs (assuming the same model snapshot) is a ma…

You’re mixing the terms I think.

Reproducible builds is not the same as reproducible outcomes.

Rebuilding your project with a different version of a dependency is not the same as suddenly a program accepts txt files as attachments instead of pdfs because the model was borrowing a txt example from its training data and got sidetracked.

Re: LLMs as the new high level language

#384

Post by sublinear »

Davidzheng wrote: post #205
sublinear wrote: post #190

Earlier quoted context omitted.

Your example actually perfectly describes why it won't displace anyone. The user still has to be in the loop to instruct the LLM every time, and the tiniest nuances in each execution of that work still matters. If it didn't we'd have replaced people with bash scripts many decades ago. When we've tried to do that, the maintenance of those scripts became a game of whack-a-mole that never ended and they're eventually ab…

Even if your argument is correct here, it would only mean this particular method of replacement doesn't immediately work for this job.

Thanks for the reply. I'm not sure I understand. Why does replacement matter so much? Where do these anxieties come from?

I'm saying that this type of employment is a necessary diffusion layer for making decisions, and isn't about "productivity". Payroll for these kinds of jobs is even considered capex. Efficiency misses the point entirely and is tripping over dollars to pick up pennies.

Re: LLMs as the new high level language

#385

Post by heavyset_go »

hackyhacky wrote: post #345
heavyset_go wrote: post #274

Earlier quoted context omitted.

> Why pay an expensive architect to design your new office building, when AI will do it for peanuts? Why pay an expensive lawyer to review your contract? Why pay a doctor, etc. All of those jobs are mandated by law to done by accredited and liable humans.

> All of those jobs are mandated by law to done by accredited and liable humans. Good point. The jobs I listed will be protected for a little while due to statutory limitations. At first, firms will have one AI-augmented lawyer take on the work of a dozen lawyers. Of course his salary won't increase, and the others will be fired. Eventually, he'll just be rubber-stamping the AI's results, purely for the sake of compl…

Lawyers will always be human and their law firms will always be run by humans because lawyers were smart enough to ensure that they had complete agency and monopoly in their markets by law, not investors, business owners, and what have you.

Re: LLMs as the new high level language

#386

Post by hackyhacky »

heavyset_go wrote: post #385
hackyhacky wrote: post #345

Earlier quoted context omitted.

> All of those jobs are mandated by law to done by accredited and liable humans. Good point. The jobs I listed will be protected for a little while due to statutory limitations. At first, firms will have one AI-augmented lawyer take on the work of a dozen lawyers. Of course his salary won't increase, and the others will be fired. Eventually, he'll just be rubber-stamping the AI's results, purely for the sake of compl…

Lawyers will always be human and their law firms will always be run by humans because lawyers were smart enough to ensure that they had complete agency and monopoly in their markets by law, not investors, business owners, and what have you.

> Then the ruling class will petition the legislature to change the law in the name of "efficiency," and that will be the end of that.

Re: LLMs as the new high level language

#388

Post by orbital-decay »

sarchertech wrote: post #377
orbital-decay wrote: post #303

Earlier quoted context omitted.

>where you check in the spec and regenerate the code then prompt instability/chaotic output makes that infeasible What, why would you want to write the code anew? Identify the changes in the spec and bring the existing code in line with them.

That’s the whole thesis of the article. Using an LLM as a high level language. >The codebase should be reconstructable from the documentation

And this kind of reproducibility is insensitive to prompt instability. If your inference is deterministic (trivial if within the lifetime of a snapshot), you give it the same docs and get the same result.

Re: LLMs as the new high level language

#389

Post by collingreen »

coldtea wrote: post #369
collingreen wrote: post #366

Earlier quoted context omitted.

This is the most handwaving per paragraph I've ever seen. I think a fair summarization of your point is "LLM generated programs work well enough often enough to not need more constraints or validation than natural language", whatever that means. If you take that as a true thing then sure why would you go deeper (eg, I never look at the compiled bytecode my high level languages produce for this exact reason - I'm extr…

> This is the most handwaving per paragraph I've ever seen. Yes: "LLM generated programs work well enough often enough to not need more constraints or validation than natural language" if a fair summarization of my point. Not sure the purpose of "whatever that means" that you added. It's clear what it means. Thought, casual language seems to be a problem for you. Do you only always discuss in formally verified proofs…

The "whatever that means" isn't a judgement jab at your point, merely acknowledging the hand waving of my own with "good enough".

I hope this comment thread helps with your cheeky jab that I might have a problem understanding or using casual language.

I'm not sure if it's moving the goalpost or not to back away from a strong claim that LLMs are at the "good enough" (whatever that means!) level now and instead fall back to "some devs will just ship it and therefore that's good enough, by definition".

Regardless, I think we agree that, if LLMs are "good enough" in this way then we can think a lot less about code and logic and instead focus on prompts and feature requests.

I just don't think we agree on what "good enough" is, if current LLMs produce it with less effort than alternatives, and if most devs already believe the LLM generated code is good enough for that.

I use LLMs for a lot of dev work but I haven't personally seen these things one- or even many- shot things to the level I'd feel comfortable being on call for.

Re: LLMs as the new high level language

#390

Post by coldtea »

collingreen wrote: post #389
coldtea wrote: post #369

Earlier quoted context omitted.

> This is the most handwaving per paragraph I've ever seen. Yes: "LLM generated programs work well enough often enough to not need more constraints or validation than natural language" if a fair summarization of my point. Not sure the purpose of "whatever that means" that you added. It's clear what it means. Thought, casual language seems to be a problem for you. Do you only always discuss in formally verified proofs…

The "whatever that means" isn't a judgement jab at your point, merely acknowledging the hand waving of my own with "good enough". I hope this comment thread helps with your cheeky jab that I might have a problem understanding or using casual language. I'm not sure if it's moving the goalpost or not to back away from a strong claim that LLMs are at the "good enough" (whatever that means!) level now and instead fall ba…

>I just don't think we agree on what "good enough" is, if current LLMs produce it with less effort than alternatives, and if most devs already believe the LLM generated code is good enough for that.

Don't need to consider what they think, one can just see their "revealed preferences", what they actually do. Which for the most part is adopting agents.

>I use LLMs for a lot of dev work but I haven't personally seen these things one- or even many- shot things to the level I'd feel comfortable being on call for.

That's true for many devs one might have working for their team as well. Or even one's self. So we review, we add tests, and so on. So we do that when the programming language is a "real" programming language too, doesn't have to change when it is natural language to an agent. What I'm getting at, is, that this is not a show stopper to the point of TFA.

Post reply on HN