Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I burned all my tokens researching how to save tokens

quesma.com

Post reply on HN
Print view View original

81–90 of 237 posts

Re: I burned all my tokens researching how to save tokens

#81

Post by Chris2048 »

kittoes wrote: post #72
Chris2048 wrote: post #69

Earlier quoted context omitted.

Then.. why would you link to the readme if you know it's slop? What's the point of a slop README?

'Cause I don't have any quality user-facing documentation yet; that README is for bots. None of the interfaces have had a chance to settle, so any time spent accurately and humanely documenting what the project is capable of right now is an investment I cannot afford. I figured most people would just give it over to their favorite LLM with whatever instructions they prefer for translation because that's what I would'…

What does "that README is for bots" mean? Are you implying it is some kind of special language written by and correctly understood by bots/LLMs, but otherwise semantically coherent?

Re: I burned all my tokens researching how to save tokens

#82

Post by Chris2048 »

throwpoaster wrote: post #78
Chris2048 wrote: post #75

Earlier quoted context omitted.

And the comment text is what they typed into the text box. So just show us the same - the prompt.

A null transform is a transform. Engage seriously with the project and OP will deliver.

Are you confusing whatever you mean by 'null transform' with an identity function?

The project isn't serious, why should I be?

Re: I burned all my tokens researching how to save tokens

#83

Post by nojito »

inigyou wrote: post #74
nojito wrote: post #67

Earlier quoted context omitted.

a massive number of data engineering agents. We are able to stand up, test, audit, deploy data pipelines much more efficiently with the foundational models.

Examples of them being useful?

Happier engineering teams? Happier end users of the data?

What does useful mean to you?

Re: I burned all my tokens researching how to save tokens

#85

Post by ch4s3 »

dominotw wrote: post #30
resonious wrote: post #20

Earlier quoted context omitted.

Trust me there are plenty of us using cloud AI to actually ship stuff. We just aren't writing blog posts about it.

what did you ship?

My team redid a bunch of our ETL. I personally built a programming language that you can download and use, and I’m about to release an iOS app(in a few weeks).

Re: I burned all my tokens researching how to save tokens

#86

Post by edg5000 »

noisy_boy wrote: post #14
edg5000 wrote: post #10

Earlier quoted context omitted.

Yes, but you could have an uncached dynamic tail, this would not kill the cache though. And you could still periodically kill the cache, still keeping the same prefix every N requests. But it's questionable what all this extra complexity delivers. I found that now that the models are better, at least Codex compaction with GPT 5.5+ xhigh is almost unnoticible. So the answer is: just let the context fill up, don't prun…

That's what I have been doing with Cursor with auto mode. Let the model manage the cache through compaction but staying focused on the topic. It has been quite efficient cost wise and quality wise too. It found issuesbat cheaper cost that Sonnet 4.6 medium effort had missed while costing more. I would like to repeat this test with higher tier Anthropic models like Opus effort medium to see the results.

With a coding plan, there is no need to use lower quality models right (GPT 5.6, Fable 5). Only with some kind of batch job (processing PDFs) I could imagine not having enough usage with a 100 or 200 USD coding plan. It sounds like you're on API? I'm dying to know why, since it seems quite common, yet it's about ten times the cost. Sounds like a terrible deal. What am I missing?

Re: I burned all my tokens researching how to save tokens

#87

Post by dofm »

stavros wrote: post #79
dofm wrote: post #71

Earlier quoted context omitted.

Thank you for the details! These are precisely the kind of things I want to hear about because they are self-contained and I can judge against the kind of stuff I do. I am quite interested in experimenting with it for migrations, because for example I have a set of sites written in older Nuxt and Vue and Buefy that need refreshing (and the frontend ported to that UI-agnostic Buefy replacement whose name escapes me at…

You're welcome! LLMs definitely help with the migrations, there were definitely a few iterations because the surface area was massive, but overall it took the project from "will never be prioritised" to "I can work on it on and off when in boring meetings", which was a massive win. I haven't found that LLMs help with CAD at all, but YMMV. As for sharing here, the last time I shared something with "here's something I…

Oh I definitely don't expect them to help with CAD and nor would I want them to — I really enjoy knowing FreeCAD.

It's more the microcontroller choice, broad components, all that.

The main areas I find LLMs struggle with research, I struggle with research too. I am a pretty solid researcher and I find it validating, in fact, to find that a more brute-force-trained thing is not better than me.

I see your point about the snarky reply; I must admit I have probably in the more distant past been somewhat snarky but I stopped commenting like that whenever I spot myself doing it because I know it is dispiriting.

(The other reply, re: early and late stages, is the kind of substantive question that lingers in my mind).

Anyway I just want to reiterate my thanks: I am sort of cynical about this, in that I believe the technologies may have their uses, but I have no time for the cult and the hype train, so it is nice to see anyone talking about actual things.

Re: I burned all my tokens researching how to save tokens

#88

Post by inigyou »

nojito wrote: post #83
inigyou wrote: post #74

Earlier quoted context omitted.

Examples of them being useful?

Happier engineering teams? Happier end users of the data? What does useful mean to you?

So far you've only explained that AI made your coworkers happier. That's fine, but weren't we talking about productivity? Doing more with less?

Re: I burned all my tokens researching how to save tokens

#89

Post by rsfern »

kittoes wrote: post #72
Chris2048 wrote: post #69

Earlier quoted context omitted.

Then.. why would you link to the readme if you know it's slop? What's the point of a slop README?

'Cause I don't have any quality user-facing documentation yet; that README is for bots. None of the interfaces have had a chance to settle, so any time spent accurately and humanely documenting what the project is capable of right now is an investment I cannot afford. I figured most people would just give it over to their favorite LLM with whatever instructions they prefer for translation because that's what I would'…

I don’t mean to pick on you in particular here, but this approach has been bothering me a lot lately, and it seems like it’s super common.

I get that it’s an early prototype and not all the design choices are made yet, but I struggle with “I can’t afford human-readable documentation yet”. Isn’t human readable documentation important for efficiently planning and deciding what you want to build? I feel like I can’t afford not to have human readable docs and plans while the project is taking shape

Related, if a user can prompt an agent to translate to a human-readable summary, wouldn’t it be better to just do this in place? Sure, models can deal with noisy LLM outputs but shouldn’t a document that’s easier for humans also be easier for bots?

Re: I burned all my tokens researching how to save tokens

#90

Post by kittoes »

Chris2048 wrote: post #81
kittoes wrote: post #72

Earlier quoted context omitted.

'Cause I don't have any quality user-facing documentation yet; that README is for bots. None of the interfaces have had a chance to settle, so any time spent accurately and humanely documenting what the project is capable of right now is an investment I cannot afford. I figured most people would just give it over to their favorite LLM with whatever instructions they prefer for translation because that's what I would'…

What does "that README is for bots" mean? Are you implying it is some kind of special language written by and correctly understood by bots/LLMs, but otherwise semantically coherent?

Sort of? The bots almost insist on making sure one exists because that's one of the many patterns they trained on and they do their best to be human-like, but we all know that just doesn't pan out. They're also a side-effect of your long-term interactions with the agents, very much like being in a tight clique where most of your conversations sound like gibberish to outsiders because of all the jargon.
Post reply on HN