Earlier quoted context omitted.
If you haven't actually used GPT-5 yet, your assessment is irrelevant.
But the real game changer will be GPT-6.
Devin: AI Software Engineer
281–290 of 604 posts
Re: Devin: AI Software Engineer
#282Earlier quoted context omitted.
We're solving the wrong problem. People trying to use cars to pull horse carts are doomed to fail. Trying to use AI to build the software of yesterday is a waste of time.
So what should the AI build instead? Specifically with regard to UI. I don't want my banking app to run on a bunch of non-deterministic prompts.
For me, it's mostly information I want. I don't really need a full app for that. I want to know:
1) How much money do I have? 2) Did a check I cashed clear yet? 3) Are there any unusual charges? How is my spending this month? 4) Anything I should look into?
For actions I'd want to take:
1) Deposit check 2) Transfer money from account to account 3) Make a payment / authenticate with new EFT payee
All of that could be done conversationally, with a flexible level of detail. The data could be shown on any number of shared interfaces (messaging app, dedicated companion app, etc).
It doesn't obviate the need for software, and there are a ton of software ideas that exist today that will still be useful, as well as ideas that have yet to be discovered that will be useful. But...I expect the LOB app to go extinct in the next couple of decades.
Re: Devin: AI Software Engineer
#283Earlier quoted context omitted.
So what should the AI build instead? Specifically with regard to UI. I don't want my banking app to run on a bunch of non-deterministic prompts.
Akin to alchemy, spring up the UI to solve the user's problem. When timelines are shortened from weeks to hours, what can we build? Can a user just talk to a computer and solve their problem regardless of the platform?
Seems like a no-brainer.
Maybe folks LIKE clicking on buttons and going through 10-step procedures to get tasks done.
Some mice like the maze more than the cheese, I guess.
Re: Devin: AI Software Engineer
#284If you need AI to help you program an algorithm, then you shouldn't be using it because you can't tell if AI's solution is correct. If you can tell if a solution is correct or not --- well, then you don't need to have AI write it for you. I think AI programming can only work when the industry begin to treat "almost working" systems backed by human customer service as acceptable.
Did you just solve P=NP?
Many things are trivial to verify, but hard/time consuming to code up.
You probably shouldn't rely on this to write critical software, no matter the amount of manual QA you throw at it afterwards, but there is an abundance of non-critical use cases where you can quickly check if a solution is good enough for what you care about.
Re: Devin: AI Software Engineer
#285Re: Devin: AI Software Engineer
#286Earlier quoted context omitted.
> They produce generic, amateurish copy that reads like it's written by committee. If you were only using GPT 3.5 (free ChatGPT) then your opinion is irrelevant. With GPT-4 you could directly ask it: "rewrite your previous response so that it sounds less generic, less amateurish, and not written by a committee". I'm not even joking. Just provide enough information and tell it what to do. If you don't like the output…
Chat GPT4 is a technological miracle, but it can only produce trite, formulaic text and it's _relentlessly_ polly-anna-ish. Everything reads like ad copy and it's easily identifiable.
Ask it to "rewrite that in the tone of an English professor" or "rewrite that in the style of a redneck rapper" or "make that sound less like generic ad copy". Get into an argument back and forth with the LLM and tell it the previous response is crap because of XYZ.
Re: Devin: AI Software Engineer
#287As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…
Re: Devin: AI Software Engineer
#288As a developer but also product person, I keep trying to use AI to code for me. I keep failing, because of context length, because of shit output from the model, because of lack of any kind of architecture etc etc etc. I'm probably dumb as hell, because I just can't get it to do anything remotely useful, more than helping me with leetcode. Just yesterday I tried to feed it a simple HTML page to extract a selector, I…
The problem with this entire space is that we have VC hype for work that should ultimately still be being done in research labs.
Nearly all LLM results are completely mind blowing from a research perspective but still a long way from production ready for all but a small subset of problems.
The frustrating thing, as someone working in this space awhile, is that VCs want to see game changing products ship overnight. Teams working on the product facing end of these things are all being pushed insanely hard to ship. Most of those teams are solving problems never solved before, but given deadlines as though they are shipping CRUD web apps. The kicker is that despite many teams doing all of this, because the technology still isn't there, they still disappoint "leadership". I've personally seen teams working nights and weekends, implementing solutions to never before seen problems in a few weeks, and still getting a thumbs down when they cross the finish line.
To really solve novel problems with LLMs will take a large amount of research, experimentation and prototyping of ideas, but people funding this hype have no patience for that. I fear we'll get hit by a major AI winter when investors get bored, but we'll end up leaving a lot of value on the table simply because there wasn't enough focus and patience on making these incredible tools work.
Re: Devin: AI Software Engineer
#289I have a few years of experience in backend development, and I have realized that LLMs are incredible productivity boosts for generating code only if you know the underlying libraries/frameworks/languages very well. You can then prompt it with very specific instructions and it can go do that. Helps with the typing, but that's pretty much all. I still have to know everything and it can definitely not do everything on…
It's funny how a year ago I was really excited for how AI can help my everyday coding while fearful that it would replace me. Now I'm not really sure either will happen in the short term.
Re: Devin: AI Software Engineer
#290Earlier quoted context omitted.
You really need to try Opus. Try a provider that works across models (one in my bio).
It's incredible how far behind HN of all places is w.r.t. what the current best tech is. So many people talking about GPT-4 here, or even 3.5 when the SOTA has moved way along. Gemini Advanced is also a great model, but for other reasons. That thing really knows a boat load of low level optimization tricks.
So what is the SOTA, in your opinion?