Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Testing GPT 4's code-writing capabilities with some real world problems

tylerglaiel.substack.com

Post reply on HN
Print view View original

261–270 of 677 posts

Re: Testing GPT 4's code-writing capabilities with some real world problems

#261

Post by illiarian »

nitwit005 wrote: post #58
maxdoop wrote: post #50

Earlier quoted context omitted.

I don’t think you’ve tried GPT-4. If you have, I don’t think you are like majority of devs (maybe not on HN, but in real life). You sound lucky to have true, novel problems to solve each day. I’m with many here commenting that this is quite powerful stuff, especially when my day-to-day is writing simple CRUD apps, or transforming data from one format to another within an API, or configuring some new bit of infra or C…

If all you're doing is very simple crud apps and transforming API responses, your coding is already replaceable with "no code" tools.

I've been a programmer for 23 years now. In all these years every year was the year of no-code tools.

And you know what? We went from having actual no- or low-code tools (UI builders, forms in Access and FoxPro etc.) to zero no-code tools worth a damn [1]. There was a brief dream of Yahoo! Pipes in mid-to-late 2000s, but it's dead as well.

[1] Except some applications like Unreal Blueprints and similar node-based tools in audio and graphical software

Re: Testing GPT 4's code-writing capabilities with some real world problems

#262

Post by illiarian »

joenot443 wrote: post #217
theshrike79 wrote: post #106

Earlier quoted context omitted.

I'm still waiting for the no-code tools that can replace me, the API Glue Coder. They've been promising those since the early 2000's, but I'm still employed. Some stuff has been automated by Zapier, IFTTT and the like, but most of the systems I'm integrating with don't have public APIs.

"I'm still waiting for the automobile that can replace me, the Horse Carriage Driver. They've been promising those since the 1880s, but I've still got passengers in my carriage every day!" Obviously it's a cheeky example, but this would not be the first time in history a previously well-established career was upended in a (relatively) short amount of time. I'm a FAANG dev, I've got skin in the game too and I'm trying…

> I'm still waiting for the automobile that can replace me, the Horse Carriage Driver.

Wow. The hype around "AI"s has entered the crypto stages. Just like crypto was the new car, the new internet etc., so is AI now.

Well, at least AI is useful in certain applications.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#263

Post by yonaguska »

nemo44x wrote: post #254
IKLOL wrote: post #229

Earlier quoted context omitted.

I think the fear should be less about AI taking 100% of jobs but it should be AI making a single programmer do the job of 5, which would wipe a majority of the market out and make it a non-viable career option for most. Companies are already bloated, imagine when they realize one overworked highly paid senior can replace 10 juniors.

Wouldn’t this actually create more demand for programmers? More businesses can exist that have a need for software. To date every advance in programming efficiency has resulted in more tech, not less. Additionally there’s the math behind it. If Company A fires 50% of their staff because AI lets the remaining 50% work at twice the productivity then how will they compete with Company B that keeps their staff and now ge…

Yep, I personally see the cost of doing a start-up going way down for individual developers or even junior developers.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#264

Post by oblio »

anonzzzies wrote: post #164
DaiPlusPlus wrote: post #154

Earlier quoted context omitted.

> I personally know about 10 people who have been fired or assigned to a completely different position because gpt did a faster and better or equal job Please elaborate. And, if true, this would be a major news story that Vox or any number of major newspapers would love to write about - so have you approached the media about this? If not, why not?

> Please elaborate. I simply know firsthand (i'm old, i have manager, cto, ceo friends who I go golf and play squash with) that people in data entry and programming have been let go in the past weeks because 1 person could take over their work using the gtp/chatgpt api's and do their work faster with less errors. I am recommending the same in my company as a lot of my colleagues are doing nothing anymore as the skill…

How does this work from an IP perspective? Aren't you feeding OpenAI and Microsoft with your confidential information?

Also, what license is the code under when you get it back?

Re: Testing GPT 4's code-writing capabilities with some real world problems

#265

Post by blagie »

Off topic: This problem was fun. I would really enjoy a site with a pile of nonstandard, fun, and interesting problems like this one.

Coming up with a working algorithm took about 30 seconds (I got lucky, not brilliant), but it stretched my brain in an interesting way.

That's different from practice sites like leetcode, which have pretty cookie cutter problems. On problems like this one, sometimes:

- I get it in a few seconds, like this case

- Get it in a few minutes

- Get it in a few hours

- Give up and look up the solution

A fun problem a day would be, well, fun.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#266

Post by wantsanagent »

the_af wrote: post #225

The first problem statement that GPT got wrong actually shows a problem with human language. "Avoid fire" means "do not ever go through fire" to me (and GPT thinks the same, apparently). The author thought it meant "avoid fire if you can, but go through it if there's no other way". This was a problem with informal requirements that could have happened in an entirely human context.

I find adding 'before you answer, ask clarifying questions' to a prompt is quite helpful in avoiding these traps.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#267

Post by brookst »

SuoDuanDao wrote: post #239
brookst wrote: post #187

Earlier quoted context omitted.

I think you just declared radio, television, the internet, the macaroon, and mobile phones to be singularities. People generally use the term to mean the point at which progress is automatic, unbounded, and instant. E.g. AIs that can produce smarter AIs themselves, in short order.

'step change' sounds less exciting than 'singularity' but in terms of the original definition there's very little distinction.

Hmm. I see the singularity as a recursive step change; the last step where the average slope becomes vertical.

I’m not a singularity believer (singularitist?); from the perspective of 1000 years ago we’re already at the point of infinite progress in zero time. I think individuals, the culture, and the species adapt to accelerating progress, and I intuit, maybe wrongly, that Gödel‘s theorem means that technology will never become runaway and recursive.

But I think that’s what people mean, more than just a step/paradigm change like the internet.

Re: Testing GPT 4's code-writing capabilities with some real world problems

#268

Post by AstralStorm »

gpderetta wrote: post #209
kykeonaut wrote: post #205

Earlier quoted context omitted.

To paraphrase Harold Abelson: Computer Science is not really very much about computers. And it’s not about computers in the same sense that physics isn’t really about particle accelerators, and biology is not really about microscopes and petri dishes. It is about formalizing intuitions about process: how to do things [0]. [0]: https://www.driverlesscrocodile.com/technology/the-wizard-1-... .

“Computer Science is no more about computers than astronomy is about telescopes” - Edsger Dijkstra

And then some guy called Kepler invented a new kind of telescope.

(And we all atarted writing more quantum algorithms.)

Re: Testing GPT 4's code-writing capabilities with some real world problems

#269

Post by illiarian »

SuoDuanDao wrote: post #232
caddemon wrote: post #68

Earlier quoted context omitted.

You know the shoeshine boy indicator story [1]? That's the vibe I've been getting with ChatGPT lately, I have multiple old high school teachers with 0 computational background who hardly ever spoke about ML before posting regularly on LinkedIn about ChatGPT the last couple months. The part that gets me is they aren't just aimlessly making posts, they're getting involved with webinars targeted at educators in their re…

Teaching is going to be a career affected early by chatgpt because students will use it (are using it) to do assignments. A good teacher doesn't need to know much about machine learning to think about how that will affect his job, he just needs to know his students and have an idea how to work with them given the new tools they have.

> Teaching is going to be a career affected early by chatgpt because students will use it (are using it) to do assignments.

Already happening. A student in Russia submitted his thesis written with the help of GPT, it was accepted, then rejected, then accepted: https://www.themoscowtimes.com/2023/02/02/russian-student-al...

Re: Testing GPT 4's code-writing capabilities with some real world problems

#270

Post by adverbly »

I think having a better understanding about the underlying statistical model of how these AIs are trained is helping me keep back the wave of fear and anxiety associated with AI risks.

The singularity requires AIs to be very good at doing things people have not done before. But this form of machine learning is bad at that. It is like someone who doesn't actually understand anything has somehow managed to memorize their way through whatever topic you're asking about. They have lots of tips and information about things, similar to what you might currently find by doing research. But they don't seem to have what is required to push the boundaries of knowledge for understanding, because they don't actually really have it in the first place. Or maybe what they have is just very minimal when compared to the contribution of their memorization.

Obviously you still have the main risks of breaking capitalism, mass unemployment, pollution of public communications, etc. But honestly, I think each of these are far less scary to me than the existential risk of superintelligence. So in a way I'm actually happy this is happening the way it is right now, and we don't have to deal with both of these risks at the same time.

Our current approach is probably the safest way to progress AI that I can think of: it requires a new model to improve, and it's learning entirely from human data. It might not seem like it, but this is actually pretty slow, expensive, and limited compared to how I expected AI to improve given Sci fi movies or Nick Bostrom's writings(curious what he'd have to say about this resurgence of AI)

Post reply on HN