Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two kinds of AI users are emerging

martinalderson.com

Post reply on HN
Print view View original

21–30 of 358 posts

Re: Two kinds of AI users are emerging

#21

Post by lmm »

martinald wrote: post #4
simmerup wrote: post #2

Terrifying that people are creating financial models with AI when they don’t have the skills to verify the model does what they expect

They have an excel sheet next to it - they can test it against that. Plus they can ask questions if something seems off and have it explain the code.

> They have an excel sheet next to it - they can test it against that.

It used to be that we'd fix the copy-paste bugs in the excel sheet when we converted it to a proper model, good to know that we'll now preserve them forever.

Re: Two kinds of AI users are emerging

#22

Post by decimalenough »

defrost wrote: post #12

The "upside" description: On the other you have a non-technical executive who's got his head round Claude Code and can run e.g. Python locally. I helped one recently almost one-shot converting a 30 sheet mind numbingly complicated Excel financial model to Python with Claude Code. Once the model is in Python, you effectively have a data science team in your pocket with Claude Code. You can easily run Monte Carlo simul…

I'm almost certain it will be significantly worse.

The Excel sheet will have been tuned over the years by people who knew exactly what it was doing and fixed countless bugs along the way.

The Claude Code copy will be a simulacrum that may behave the same way with some inputs, but is likely to get many of edge cases wrong, and, when you're talking about 30 sheets of Excel, there will be many, many of these sharp edges.

Re: Two kinds of AI users are emerging

#23

Post by linsomniac »

decimalenough wrote: post #18

> I helped one recently almost one-shot[3] converting a 30 sheet mind numbingly complicated Excel financial model to Python with Claude Code. I'm sure Claude Code will happily one-shot that conversion. It's also virtually guaranteed to have messed up vital parts of the original logic in the process.

It depends on how easily testable the Excel is. If Claude has the ability to run both the Excel and the Python with different inputs, and check the outputs, it's stunningly likely to be able to one-shot it.

Re: Two kinds of AI users are emerging

#24

Post by majormajor »

defrost wrote: post #12

The "upside" description: On the other you have a non-technical executive who's got his head round Claude Code and can run e.g. Python locally. I helped one recently almost one-shot converting a 30 sheet mind numbingly complicated Excel financial model to Python with Claude Code. Once the model is in Python, you effectively have a data science team in your pocket with Claude Code. You can easily run Monte Carlo simul…

One of the dirty secrets of a lot of these "code adjacent" areas is that they have very little testing.

If a data science team modeled something incorrectly in their simulation, who's gonna catch it? Usually nobody. At least not until it's too late. Will you say "this doesn't look plausible" about the output? Or maybe you'll be too worried about getting chided for "not being data driven" enough.

If an exec tells an intern or temp to vibecode that thing instead, then you definitely won't have any checkpoints in the process to make sure the human-language prompt describing process was properly turned into the right simulation. But unlike in coding, you don't have a user-facing product that someone can click around in, or send requests to, and verify. Is there a test suite for the giant excel doc? I'm assuming no, maybe I'm wrong.

It feels like it's going to be very hard for anyone working in areas with less black-and-white verifiability or correctness like that sort of financial modeling.

Re: Two kinds of AI users are emerging

#25

Post by Spivak »

decimalenough wrote: post #18

> I helped one recently almost one-shot[3] converting a 30 sheet mind numbingly complicated Excel financial model to Python with Claude Code. I'm sure Claude Code will happily one-shot that conversion. It's also virtually guaranteed to have messed up vital parts of the original logic in the process.

Doesn't it help you sleep at night that your 401k might be managed by analysts #yoloing their financial modeling tools with an LLM?

Re: Two kinds of AI users are emerging

#26

Post by mkoubaa »

simmerup wrote: post #2

Terrifying that people are creating financial models with AI when they don’t have the skills to verify the model does what they expect

It's not terrifying at all, some shops will fail and some will succeed and in the aggregate it'll be no different for the rest of us

Re: Two kinds of AI users are emerging

#27

Post by martinald »

linsomniac wrote: post #23
decimalenough wrote: post #18

> I helped one recently almost one-shot[3] converting a 30 sheet mind numbingly complicated Excel financial model to Python with Claude Code. I'm sure Claude Code will happily one-shot that conversion. It's also virtually guaranteed to have messed up vital parts of the original logic in the process.

It depends on how easily testable the Excel is. If Claude has the ability to run both the Excel and the Python with different inputs, and check the outputs, it's stunningly likely to be able to one-shot it.

That's exactly what it did (author here).

Re: Two kinds of AI users are emerging

#28

Post by suprstarrd »

fdsf2 wrote: post #15
majormajor wrote: post #8

Earlier quoted context omitted.

The use of specialization of interfaces is apparent if you compare Photoshop with Gemini Pro/Nano Banana for targeted image editing. I can select exactly where I want changes and have targeted element removal in Photoshop. If I submit the image and try to describe my desired changes textually, I get less easily-controllable output. (And I might still get scrambled text, for instance, in parts of the image that it did…

It behooves me that Gemini et al dont have these standard video editing tools. Do the engineers seriously think prompting by text is the way people want videos to be generated? Nope. People want to customise. E.g. Check out capcut in the context of social media. Ive been trying to create a quick and dirty marketing promo via an LLM to visualise how a product will fit into the world of people - it is incredibly painfu…

Speaking as someone who doesn't like the idea of AI art so take my words with a grain of salt, but my theory is that this input method exclusivity is intentional on their part, for exactly the reason you want the change. If you only let people making AI art communicate what they want through text or reference attachments (the latter of which they usually won't have), then they have to spend time figuring out how to put it into words. It IS painful to ask for those refinements, because any human would clearly understands it. In the end, those people get to say that they spent hours, days, or weeks refining "their prompt" to get a consistent and somewhat-okay looking image; the engineers get to train their AI to better understand the context of what someone is saying; and all the while the company gets to further legitimize a false art form.

Re: Two kinds of AI users are emerging

#29

Post by danpalmer »

I've noticed a huge gap between AI use on greenfield projects and brownfield projects. The first day of working on a greenfield project I can accomplish a week of work. But the second day I can accomplish a few days of work. By the end of the first week I'm getting a 20% productivity gain.

I think AI is just allowing everyone to speed-run the innovator's dilemma. Anyone can create a small version of anything, while big orgs will struggle to move quickly as before.

The interesting bit is going to be whether we see AI being used in maturing those small systems into big complex ones that account for the edge cases, meet all the requirements, scale as needed, etc. That's hard for humans to do, and particularly while still moving. I've not see any of this from AI yet outside of either a) very directed small changes to large complex systems, or b) plugins/extensions/etc along a well define set of rails.

Re: Two kinds of AI users are emerging

#30

Post by defrost »

decimalenough wrote: post #22
defrost wrote: post #12

The "upside" description: On the other you have a non-technical executive who's got his head round Claude Code and can run e.g. Python locally. I helped one recently almost one-shot converting a 30 sheet mind numbingly complicated Excel financial model to Python with Claude Code. Once the model is in Python, you effectively have a data science team in your pocket with Claude Code. You can easily run Monte Carlo simul…

I'm almost certain it will be significantly worse. The Excel sheet will have been tuned over the years by people who knew exactly what it was doing and fixed countless bugs along the way. The Claude Code copy will be a simulacrum that may behave the same way with some inputs, but is likely to get many of edge cases wrong, and, when you're talking about 30 sheets of Excel, there will be many, many of these sharp edges…

I won't disagree - I suffered from insufficient damning praise in my last sentence above.

IMHO, earned through years of bleeding eyeballs, the first will be riddled with subtle edge cases curiously patched and fettled such that it'll limp through to the desired goal .. mostly.

The automated AI assisted transcoding will be ... interesting.

Post reply on HN