Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I burned all my tokens researching how to save tokens

quesma.com

Post reply on HN
Print view View original

71–80 of 237 posts

Re: I burned all my tokens researching how to save tokens

#71

Post by dofm »

stavros wrote: post #64
dofm wrote: post #41

Earlier quoted context omitted.

If you characterise questions as bad faith so you don’t have to answer, sure. Since this is my thread, and since I am an exasperated freelancer who has made most of his post-dot-com career from writing exactly the kind of quite small, simple, unambitious, often internal things for smaller customers that nobody would confuse with anything cool, who is trying to understand how AI is going to make his life any better wh…

I've shipped small, simple, unambitious internal things for our company that saved people lots of time, and I've shipped them in minutes rather than the a day or two they used to take. I've just shipped a migration from our old auth service to the new one in a week, a migration that we've put off for three years because it would take the team months to finish. I've shipped small side-projects that I'm the primary (an…

Thank you for the details! These are precisely the kind of things I want to hear about because they are self-contained and I can judge against the kind of stuff I do.

I am quite interested in experimenting with it for migrations, because for example I have a set of sites written in older Nuxt and Vue and Buefy that need refreshing (and the frontend ported to that UI-agnostic Buefy replacement whose name escapes me at the moment but begins with “ou” I think). These migrations won’t happen any other way (small customers won’t pay for that) so I am open to whether tooling can reduce the cost I have to swallow. LLMs I have tested (including local models) seem to support the idea that this will be doable but I am a little out of the loop with the projects so I haven’t got back to this yet.

The hardware thing looks great; I am a sort of distracted maker and FreeCAD nerd but I often need a nudge to get started and I have found an LLM chat to help there a little bit (though it cannot answer some questions if I don’t have the specific vocabulary).

Writelucid is definitely cute.

I think you should not be put off talking about these things because whether people agree with you or not about whether they are important, they can at least measure against them.

Re: I burned all my tokens researching how to save tokens

#72

Post by kittoes »

Chris2048 wrote: post #69
kittoes wrote: post #61

Earlier quoted context omitted.

That's fair critique, the README itself is indeed slop explicitly written by bots for bots. What it actually does and how it works is actually pretty neat if you care to look.

Then.. why would you link to the readme if you know it's slop? What's the point of a slop README?

'Cause I don't have any quality user-facing documentation yet; that README is for bots. None of the interfaces have had a chance to settle, so any time spent accurately and humanely documenting what the project is capable of right now is an investment I cannot afford. I figured most people would just give it over to their favorite LLM with whatever instructions they prefer for translation because that's what I would've done...

Re: I burned all my tokens researching how to save tokens

#73

Post by inigyou »

alfl wrote: post #63
inigyou wrote: post #56

Earlier quoted context omitted.

I don't think you made that. It looks for sure like an LLM made it. And now that you have typed some words into a chat box to produce a thing, you are confused about why other people won't pay money to you for the output of the chatbox, instead of typing the same thing into their own chat boxes.

I don't think you wrote this comment. I think you typed it into a text box. RTFA - it's about using tokens to research saving tokens.

What?

Re: I burned all my tokens researching how to save tokens

#74

Post by inigyou »

nojito wrote: post #67
inigyou wrote: post #53

Earlier quoted context omitted.

Examples?

a massive number of data engineering agents. We are able to stand up, test, audit, deploy data pipelines much more efficiently with the foundational models.

Examples of them being useful?

Re: I burned all my tokens researching how to save tokens

#75

Post by Chris2048 »

alfl wrote: post #63
inigyou wrote: post #56

Earlier quoted context omitted.

I don't think you made that. It looks for sure like an LLM made it. And now that you have typed some words into a chat box to produce a thing, you are confused about why other people won't pay money to you for the output of the chatbox, instead of typing the same thing into their own chat boxes.

I don't think you wrote this comment. I think you typed it into a text box. RTFA - it's about using tokens to research saving tokens.

And the comment text is what they typed into the text box. So just show us the same - the prompt.

Re: I burned all my tokens researching how to save tokens

#76

Post by [deleted] »

stavros wrote: post #64
dofm wrote: post #41

Earlier quoted context omitted.

If you characterise questions as bad faith so you don’t have to answer, sure. Since this is my thread, and since I am an exasperated freelancer who has made most of his post-dot-com career from writing exactly the kind of quite small, simple, unambitious, often internal things for smaller customers that nobody would confuse with anything cool, who is trying to understand how AI is going to make his life any better wh…

I've shipped small, simple, unambitious internal things for our company that saved people lots of time, and I've shipped them in minutes rather than the a day or two they used to take. I've just shipped a migration from our old auth service to the new one in a week, a migration that we've put off for three years because it would take the team months to finish. I've shipped small side-projects that I'm the primary (an…

[deleted]

Re: I burned all my tokens researching how to save tokens

#77

Post by spudlyo »

camgunz wrote: post #54

TFA says "no hallucinations" but you can't fix hallucinations with rules or other models. I know I'm screaming into the void but whatever.

It's like the frog prompting the scorpion not to sting him as they cross the river. Or like when image models used to routinely draw human hands incorrectly and you'd see these "prompt hacks" that would essentially be stuffed with phrases like NO DEFORMED FINGERS, NO ELDRITCH HORRORS!

Re: I burned all my tokens researching how to save tokens

#78

Post by throwpoaster »

Chris2048 wrote: post #75
alfl wrote: post #63

Earlier quoted context omitted.

I don't think you wrote this comment. I think you typed it into a text box. RTFA - it's about using tokens to research saving tokens.

And the comment text is what they typed into the text box. So just show us the same - the prompt.

A null transform is a transform.

Engage seriously with the project and OP will deliver.

Re: I burned all my tokens researching how to save tokens

#79

Post by stavros »

dofm wrote: post #71
stavros wrote: post #64

Earlier quoted context omitted.

I've shipped small, simple, unambitious internal things for our company that saved people lots of time, and I've shipped them in minutes rather than the a day or two they used to take. I've just shipped a migration from our old auth service to the new one in a week, a migration that we've put off for three years because it would take the team months to finish. I've shipped small side-projects that I'm the primary (an…

Thank you for the details! These are precisely the kind of things I want to hear about because they are self-contained and I can judge against the kind of stuff I do. I am quite interested in experimenting with it for migrations, because for example I have a set of sites written in older Nuxt and Vue and Buefy that need refreshing (and the frontend ported to that UI-agnostic Buefy replacement whose name escapes me at…

You're welcome! LLMs definitely help with the migrations, there were definitely a few iterations because the surface area was massive, but overall it took the project from "will never be prioritised" to "I can work on it on and off when in boring meetings", which was a massive win.

I haven't found that LLMs help with CAD at all, but YMMV.

As for sharing here, the last time I shared something with "here's something I made with LLMs", the reply was "it shows", and that's just the latest instance. It just puts me off the whole thing.

EDIT: https://news.ycombinator.com/item?id=48916013

Re: I burned all my tokens researching how to save tokens

#80

Post by pianopatrick »

Bender wrote: post #65
pianopatrick wrote: post #40

Earlier quoted context omitted.

To run a local AI that is half decent at research at usable speeds requires hardware that costs thousands of dollars. Spending thousands of dollars on hardware to save dollars per month on tokens does not make financial sense. If you run the numbers you'll probably find that using cheaper cloud models makes more financial sense than running those same models locally.

I suppose I was thinking in terms of companies that already had a slew of hardware and are just missing external GPU's, FPGA's or whatever people are using these days. A company might have a $50MM budget for tokens but could save a chunk of that using their existing hardware with some modifications.

Maybe that makes sense if the company already bought the hardware.

But I currently think the fundamental reason the cloud models often make more sense is based in how the models work. My current understanding is that once these models are loaded into GPU memory, the model can respond to multiple prompts in parallel. So a cloud vendor that receives lots of user prompts at the same time can achieve really good hardware utilization. These cloud vendors can then spread the cost of the hardware over many more users than you can with a local model, leading to lower prices per prompt in the cloud.

Post reply on HN