81–90 of 113 posts
Earlier quoted context omitted.
What incredible things can be done with it?
[flagged]
There's empirically someone up your comment saying that, better to address it with arguments rather than calling them brainrotted.
but yeah, its the 80s LOC metric since quality isnt captured
On top of that isn't it strange that if the LLM makes a mistake you're still charged for those tokens? They're selling "intelligence", automation, etc but if the service doesn't work as expected the user has to pay for that.
If I use electricity to do something stupid, I still have to pay for the electricity. Intelligence is just another utility.
I keep trying to convince directors and executives at my company to look past the cost per token amount but they refuse to do so. Those are the only things that actually give any sort of measurement of the monetary value of a token by these labs, and so its what many go by. For example there's some benchmarks that show that Opus for any task that requires a higher than `high` level of effort, may have actually been c…
I keep trying to convince directors and executives at my company to look past the cost per token amount but they refuse to do so. Those are the only things that actually give any sort of measurement of the monetary value of a token by these labs, and so its what many go by. For example there's some benchmarks that show that Opus for any task that requires a higher than `high` level of effort, may have actually been c…
> may have actually been cheaper to use Fable on low even though the cost per token is drastically higher Well that's the problem with these black boxes. You really have no idea beforehand how many tokens a given task is going to take. There's simply too many variables involved. It's therefore only natural for people to assume "the cheaper and older model is probably going to cost less overall to use than the newer,…
I feel we are caught in a "this is fine, pay more and we may turn down the fire" situation. The LLM itself produces one token. Some tool adds that token to the input and runs it again, flogging the horse. Downstream another tool, some kind of harness, tries to control this stream by injecting tokens into the context and then sending it to the inference tool, and then trying to pattern-match the output. Finally, there…
[flagged]
says more than anything else could
Earlier quoted context omitted.
So you are using a “cloud” provider and at 1c per million tokens … Love to hear more about how you structure the orchestrator etc
Yes, it's a "cloud provider" but it's a cloud provider running an open model you can download (and that other cloud providers do host). I just happen to not have a computer big enough to host it. As for the Orchestrator, it's pretty simple. In essence, it's like "Jira/Trello/Kanban on autopilot". Work items have states, a state machine defines how those work items transition between states, states are todo, in progre…
How are you specifiying the graphs? Is this on github (I am still trying to move from concept to how to actually do it (plus Inhave only just woken up and need coffee :-)