Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT is all you need for the back end

github.com

Post reply on HN
Print view View original

231–240 of 278 posts

Re: GPT is all you need for the back end

#231

Post by auggierose »

Existenceblinks wrote: post #224
auggierose wrote: post #222

Earlier quoted context omitted.

Hehehe. Well, the important ingredient missing from this comic strip is proof. You can see proof as code, if you like, but the important thing here is that you can trust the proof generated by an AI without having to look yourself at it. Most people don't understand proof, but if you don't understand proof in 10 years, you will be out of a job as a programmer.

> if you don't understand proof in 10 years, you will be out of a job as a programmer. No, that's not gonna happen. Can you write a proof of an order for you go to go a fresh market to buy ingredient of a menu I want to cook? "Hey, go buy ingredients for my noodle menu whose result is my taste"

What? You are going to hire a programmer to do that for you?

Re: GPT is all you need for the back end

#232

Post by lelanthran »

chime wrote: post #24
nwah1 wrote: post #5

Earlier quoted context omitted.

Would be ridiculously inefficient, while also being nondeterministic and opaque. Impossible to debug, verify, or test anything, and thus would be unwise to use for almost any kind of important task. But maybe for a very forgiving task you can reduce developer hours. As soon as you need to start doing any kind of custom training of the model, then you are reintroducing all developer costs and then some, while the othe…

> Would be ridiculously inefficient, while also being nondeterministic and opaque. Impossible to debug, verify, or test anything, and thus would be unwise to use for almost any kind of important task. Not to defend a joke app, but I have worked in “serious” production systems that for all intents and purposes were impossible to recreate bugs in to debug. They took data from so many outside sources that the “state” of…

> They took data from so many outside sources that the “state” of the software could not be easily replicated at a later time.

By definition, that's a complex system, and reproducing errors would be equally complex.

A GPT author would produce that for every system. Worse, you would not be able to reproduce bugs in the author itself.

While humans do have bugs that cause them to misunderstand the problem, at least humans are similar enough for us to look at their wrong code and say "Hah, he thought the foobar worked with all frobzes, but it doesn't work with bazzed-up frobzes at all".

IOW, we can point to the reason the bug was written in the first place. With GPT systems it's all opaque - there's no reason or rhyme for why it emitted code that tried to work on bazzed-up frobzes the second time, and not the first time, or why it alternates between the two seemingly randomly ...

Re: GPT is all you need for the back end

#233

Post by fellellor »

Computing is slowly transforming into something out of fantasy or sci-fi. It’s no longer an exact piece of logic but more like “the force”. Something that’s capable of wildly unexpected miracles but only kinda sorta by the chosen one. Maybe.

Re: GPT is all you need for the back end

#234

Post by logifail »

bitsnbytes wrote: post #207
logifail wrote: post #187

Earlier quoted context omitted.

> Just make the AI also generate a proof that its code meets the spec. How would one tell if the AI-created "proof" is both accurate and adequate?

exactly. Just yesterday I was playing with chatgpt and found an error between the code it generated and the explanation of the code. It contradicted itself. However when I caught the error I asked it to further explain since it appears to contradict the code it generated. It then came back with an apology and it did state it made a mistake and was able to understand the error and fix it. Although I was specific about…

> when I caught the error I asked it to further explain since it appears to contradict the code it generated. It then came back with an apology and it did state it made a mistake and was able to understand the error and fix it[..]

I sense that "I'm sorry, Dave..." isn't quite as far away as we thought...

Re: GPT is all you need for the back end

#235

Post by naasking »

alphazard wrote: post #223
naasking wrote: post #202

Earlier quoted context omitted.

> This is going to have all the same problems as letting a bunch of green developers go to town on implementation without a design phase. Except without all the downsides, because GPT can rewrite the whole program nearly instantly. Do you see why our intuitions around maintenance, "good architecture/design" and good processes may now be meaningless? It seems a bit premature to say we don't have anything close when we…

Let's say I'm a bank, how do I know that my APIs don't allow the unintentional creation of money? Presumably because the engineers designed the system to prevent that. They didn't build the system by looking at example API calls and constructing a system which satisfied the examples, but had random behavior elsewhere. They understood this property as an important invariant. More important than matching the timestamps…

> Let's say I'm a bank, how do I know that my APIs don't allow the unintentional creation of money?

How do you know now?

> Presumably because the engineers designed the system to prevent that.

How do you know the engineers understood the invariants? How do you know they didn't make a mistake in coding these invariants? Banks still don't use formal methods to prove these invariants last I checked, so no matter what, you need to write tests to check any invariants, and tests still can't achieve 100% certainty.

> I'm talking about designing for simplicity so that the behavior of the system can be reliably predicted.

From the page, it sounds like the system is fairly predictable, generating a program based on a schema and a descriptive method name. If it's not predictable then the model needs to be tuned to make it more predictable, just like how any other software development advances.

If you can design your schema to ensure any invariants are preserved, even better.

Finally, don't confuse the first preview version of the product with where this is going. The project as it is is fairly simple and predictable, but a bit limited. It does point the direction towards what is possible though.

You could also have separate AI trained to do fuzz testing of an API description and automatically and instantly generate thousands of tests checking all possible corner cases, and in principle, such systems could be even more robust than human written ones simply because of the breadth of testing and the number of iterations you can rapidly go through to converge on a final product.

Re: GPT is all you need for the back end

#236

Post by naasking »

usrbinbash wrote: post #9

Yes I could do that. I could indeed invoke something that requires god-knows how many tensor cores, vram, not to mention the power requirements of all that hardware, in order to power a simple CRUD App. Or, I could not do that, and instead have it done by a sub-100-lines python script, running on a battery powered Pi.

> I could indeed invoke something that requires god-knows how many tensor cores, vram, not to mention the power requirements of all that hardware, in order to power a simple CRUD App.

The app doesn't need to be powered by the LLM for each request, it only needs to generate the code from a description once and cache it until the description changes.

Re: GPT is all you need for the back end

#237

Post by naasking »

elforce002 wrote: post #81
pak wrote: post #63

You know we’re doomed when half the comments here are taking this seriously, and not as the satire it clearly is (1KB of state? come on people) Props to the OP for showing once again how lightheaded everybody gets while gently inhaling the GPT fumes…

I'd assume everyone else is also taking this as satire. There's no way any business will handle business logic to a black box.

> There's no way any business will handle business logic to a black box.

You mean, a black box like a programmer's brain? An AI backend will get used if it's demonstrably better on any dimension. The current iteration is no doubt a bit of a toy, but don't underestimate it.

It seems incredibly obvious that you could turn this into a real product, where the LLM generates the code once based on a high-level description of a schema and an API, and caches it until the description changes somehow.

GPT can generate thousands of lines of code nearly instantly, and can regenerate it all on the fly whenever you want to make a few tweaks. No more worrying about high-architecture designed to keep complexity understandable for mere humans. No code style guides or best practices. No need to manage team sizes to keep communication overheads small.

Then you train another AI to generate a fuzz test suite to check an API for violations of the API contract. Thousands of tests checking every possible corner case, again generated nearly instantly.

Don't underestimate where this could go. The current version linked here is a limited prototype of what's to come.

Re: GPT is all you need for the back end

#238

Post by ProllyInfamous »

bitsnbytes wrote: post #207
logifail wrote: post #187

Earlier quoted context omitted.

> Just make the AI also generate a proof that its code meets the spec. How would one tell if the AI-created "proof" is both accurate and adequate?

exactly. Just yesterday I was playing with chatgpt and found an error between the code it generated and the explanation of the code. It contradicted itself. However when I caught the error I asked it to further explain since it appears to contradict the code it generated. It then came back with an apology and it did state it made a mistake and was able to understand the error and fix it. Although I was specific about…

This will be known as "The No Code Revolution."

Even your single datapoint explanation/POV/understanding will help to accelerate this entire process.

I have to keep reminding programmers that Co-Pilot exists, is real, and makes LESS mistakes than entry-level datagrunt software engineers. And it costs pennies of electricity to run daily.

All capitalism is: the search to maximize efficiency; monotonous human labor (~80%) is the most expensive part of this equation... this is not an "if," rather "when" situation. Putting your head into the sand will be a safe place for lesser programmers to still make money, for at least another few years.

But as was said elsewhere in this thread: if you do not know how to write PROOF code to VERIFY these inevitable AI-assistant-coders' outputs, you will not have a job. Human mindpower cannot compete in the bruteforce arena — all ChatGPT is right now is a bunch of autistic middle-aged asshole trolls with WAY TOO MUCH MONEY, and EVEN MORE TIME (to play around with this).

I encourage you as a more-artistic-than-technical (but still fairly intelligent) person to "just pretend" that this is your new Fiverr-tasker. Because it is already, and will be once more-widely understood / accepted.

Peace.

Re: GPT is all you need for the back end

#239

Post by ProllyInfamous »

oceanplexian wrote: post #150
rco8786 wrote: post #106

And you will almost immediately run into the fundamental problem with current iterations of GPT - You can not trust it to be correct or actually do the thing you want, only something that resembles the thing you want. The description in this link puts some really high hopes on the ability of AI to simply "figure out" what you want with little input. In reality, it will give you something that sorta kinda looks like w…

I keep hearing this assertion, that GPT can be wrong, therefore it’s an unworkable technology. But it’s a bad comparison. LLMs aren’t trying to be computationally correct like a calculator or something, the value is in their ability to semantically process a question. The other issue is assuming that the existing way of doing things is always correct. Engineers frequently get things wrong. If an AI model can complete…

>I’m not sure why folks can’t see the obvious conclusion of where this is heading.

Denial > Anger > [stages of grief]

"First they ignore you, then they laugh at you, then they fight... "and then, you win." —M.Gandhi

Re: GPT is all you need for the back end

#240

Post by ProllyInfamous »

rco8786 wrote: post #173
oceanplexian wrote: post #150

Earlier quoted context omitted.

I keep hearing this assertion, that GPT can be wrong, therefore it’s an unworkable technology. But it’s a bad comparison. LLMs aren’t trying to be computationally correct like a calculator or something, the value is in their ability to semantically process a question. The other issue is assuming that the existing way of doing things is always correct. Engineers frequently get things wrong. If an AI model can complete…

> I keep hearing this assertion, that GPT can be wrong, therefore it’s an unworkable technology. This is a straw man, I did not say any such thing. I am just pointing out the limitations that people like the author of this article seem to be blissfully unaware of. Also I would argue that your premise of AI vs a Jr eng is pretty bad. Junior engineers are not writing things to 85% correctness. If they are, they should…

>...even the worst human programmers have error rates well below 1% FOR CODE THAT ACTUALLY SHIPS.

I added that emphasis, and you should realize that shipping code is the result of teams, and engagement like ChatGPT is demonstrating will replace most of your fellow human teammembers, and the only human input may be "putting it all together..." this is a top-level jobtask, with limited employment opportunity; I worry that the code that I am already able to generate and have real results with (as a non-programmer technician) is quite scary. It is sufficient.

Simply put: the UX/UI here is too addictive and too capable to not be earth-shattering. But this is just an amateur opinion, and certainly "creativity" is already (and already was) an "INhuman" attribute, limited but to the rarest minds...

Post reply on HN