Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two kinds of AI users are emerging

martinalderson.com

Post reply on HN
Print view View original

61–70 of 358 posts

Re: Two kinds of AI users are emerging

#61

Post by EnPissant »

danpalmer wrote: post #29

I've noticed a huge gap between AI use on greenfield projects and brownfield projects. The first day of working on a greenfield project I can accomplish a week of work. But the second day I can accomplish a few days of work. By the end of the first week I'm getting a 20% productivity gain. I think AI is just allowing everyone to speed-run the innovator's dilemma. Anyone can create a small version of anything, while b…

I have experienced much of the opposite. With an established code base to copy patterns from, AI can generate code that needs a lot less iteration to clean up than on green fields projects.

Re: Two kinds of AI users are emerging

#62

Post by orwin »

danpalmer wrote: post #29

I've noticed a huge gap between AI use on greenfield projects and brownfield projects. The first day of working on a greenfield project I can accomplish a week of work. But the second day I can accomplish a few days of work. By the end of the first week I'm getting a 20% productivity gain. I think AI is just allowing everyone to speed-run the innovator's dilemma. Anyone can create a small version of anything, while b…

Yeah, my observation is that for my usual work, I can maybe get a 20% productivity boot, probably closer to 10% tbh, and for the whole team overall productivity it feels like it has done nothing, as senior use their small productivity gains to fix the tons of issues in PR (or in prod when we miss something).

But last week I had two days where I had no real work to do, so I created cli tools to help with organisation, and cleaning up, I think AI boosted my productivity at least 200%, if not 500.

Re: Two kinds of AI users are emerging

#63

Post by defrost »

tharkun__ wrote: post #38
majormajor wrote: post #24

Earlier quoted context omitted.

One of the dirty secrets of a lot of these "code adjacent" areas is that they have very little testing. If a data science team modeled something incorrectly in their simulation, who's gonna catch it? Usually nobody. At least not until it's too late. Will you say "this doesn't look plausible" about the output? Or maybe you'll be too worried about getting chided for "not being data driven" enough. If an exec tells an i…

This is a pet peeve of mine at work. Any and I mean any statistic someone throws at me I will try and dig in. And if I'm able to, I will usually find that something is very wrong somewhere. As in, the underlying data is usually just wrong, invalidating the whole thing or the data is reasonably sound but the person doing the analysis is making incorrect assumptions about parts of the data and then drawing incorrect co…

I've frequently found, over a few decades, that numerical systems are cyclically 'corrected' until results and performance match prior expectations.

There are often more errors. Sometimes the actual results are wildly different in reality to what a model expects .. but the data treatment has been bug hunted until it does what was expected .. and then attention fades away.

Re: Two kinds of AI users are emerging

#64

Post by doom2 »

I guess this is as good a thread as any to ask what the current meta is for agentic programming (in my case, as applied to data engineering). There are all these posts that make it to the front page talking about productivity gains but very few of them actually detail the setup that's working for the author, just which model is best.

I guess it's like asking for people's vim configs, but hey, there are at least a few popular posts mainly around git/vim/terminal configs.

Re: Two kinds of AI users are emerging

#65

Post by datsci_est_2015 »

linsomniac wrote: post #23
decimalenough wrote: post #18

> I helped one recently almost one-shot[3] converting a 30 sheet mind numbingly complicated Excel financial model to Python with Claude Code. I'm sure Claude Code will happily one-shot that conversion. It's also virtually guaranteed to have messed up vital parts of the original logic in the process.

It depends on how easily testable the Excel is. If Claude has the ability to run both the Excel and the Python with different inputs, and check the outputs, it's stunningly likely to be able to one-shot it.

And also - who understands the system now? Does anyone know Python at this shop? Is it someone’s implicit duty to now learn Python, or is the LLM now the de facto interface for modifying the system?

When shit hits the fan and execs need answers yesterday, will they jump to using the LLM to probabilistically make modifications to the system, or will they admit it was a mistake and pull Excel back up to deterministically make modifications the way they know how?

Re: Two kinds of AI users are emerging

#66

Post by taneq »

simmerup wrote: post #2

Terrifying that people are creating financial models with AI when they don’t have the skills to verify the model does what they expect

If they have the skills to verify the Excel model then they can apply the same approach to the numbers produced by the AI-generated model, even if they can’t inspect it directly.

In my experience a lot of Excel models aren’t really tested, just checked a bit and them deemed correct.

Re: Two kinds of AI users are emerging

#67

Post by datsci_est_2015 »

Thought this was going to be more about programmers, but it was actually about non technical users and Microsoft’s product development failure.

One tidbit I’d disagree with is that only those using the bleeding edge AI tools are reaping the benefits. There seem to be a lot of highly specialized tools and a lot of specific configurations (and mystical incantations) to get them to work, and those are constantly changing and being updated. The bleeding edge is a dangerous place to be if you value your time (and sanity).

Personally, as someone working on moderate-to-highly complex software (live inference of industrial IoT data), I can’t really open a merge / pull request for my colleagues to review unless I 100% understand what I’ve pushed, and can explain to them as well.

My killer app for AI would just be a CLI that gets me to a commit based on moderately technical input:

“Add this configuration variable for this entry point; split this class into two classes, one for each of the responsibilities that are currently crammed together; update the unit tests to reflect these changes, including splitting the tests for the old class into two different test classes; etc”

But, all the hype of the bleeding edge is around abstracting away the entire coding process until you don’t even understand what code is being generated? Hard to see it as anything but a pipe dream. AI is useful, but it’s not a panacea - you can’t fire it and replace it when it fucks up.

Re: Two kinds of AI users are emerging

#68

Post by energy123 »

doom2 wrote: post #64

I guess this is as good a thread as any to ask what the current meta is for agentic programming (in my case, as applied to data engineering). There are all these posts that make it to the front page talking about productivity gains but very few of them actually detail the setup that's working for the author, just which model is best. I guess it's like asking for people's vim configs, but hey, there are at least a few…

I push most work into chat interface (attach full codebase as a single file, paste in specs, describe what I want), then copy the tasklist from chat into codex. This is to reduce codex token usage to avoid breaching weekly limits. I'd use a more agent-heavy process if I didn't care about cost.

Re: Two kinds of AI users are emerging

#69

Post by danpalmer »

EnPissant wrote: post #61
danpalmer wrote: post #29

I've noticed a huge gap between AI use on greenfield projects and brownfield projects. The first day of working on a greenfield project I can accomplish a week of work. But the second day I can accomplish a few days of work. By the end of the first week I'm getting a 20% productivity gain. I think AI is just allowing everyone to speed-run the innovator's dilemma. Anyone can create a small version of anything, while b…

I have experienced much of the opposite. With an established code base to copy patterns from, AI can generate code that needs a lot less iteration to clean up than on green fields projects.

That's a fair observation, there's probably a sweet spot. The difference I've found is that I can reliably keep the model on track with patterns through prompting and documentation if the code doesn't have existing examples, whereas I can't document every single nuance of a big codebase and why it matters.

Re: Two kinds of AI users are emerging

#70

Post by tiangewu »

Microsoft's failure around copilot in Excel gave my partner a very poor impression on AI's ability to help with financial tasks.

It took a lot of convincing, but I finally got her to start using ChatGPT to help her write SQL and walk her through setting up some SaaS accounting software formulas.

It worked so well now she's trying to find more applications at work. Claude code is too scary for her though. That will need to be in some Web UI before she feels comfortable giving it a try.

Post reply on HN