11–20 of 383 posts
Earlier quoted context omitted.
In the past month local models have been ramping up in major way meanwhile the namesake providers have upped prices, went offline randomly, and started doing slimier and slimier things. I really think the future is local compute. Or at least self hosted models.
The hosted ones still have the advantage of being able to search the internet for live info rather than being limited to a knowledge cut off date.
This is gross It feels like we’ve been in the golden age and the window is coming to a close Let the enshitification begin, I guess
In the past month local models have been ramping up in major way meanwhile the namesake providers have upped prices, went offline randomly, and started doing slimier and slimier things. I really think the future is local compute. Or at least self hosted models.
Earlier quoted context omitted.
The hosted ones still have the advantage of being able to search the internet for live info rather than being limited to a knowledge cut off date.
I’m not sure why a model needs to be hosted in order to make network calls?
This is gross It feels like we’ve been in the golden age and the window is coming to a close Let the enshitification begin, I guess
Earlier quoted context omitted.
In the past month local models have been ramping up in major way meanwhile the namesake providers have upped prices, went offline randomly, and started doing slimier and slimier things. I really think the future is local compute. Or at least self hosted models.
What's the rough equivalent of a local model? Are we talking GPT-4?
128GB of RAM? Sure, the early to mid 4s releases, except maybe 4o. And on an M5 Max, about the same speed.
I wouldn't really bother under 64GB (meaning 32GB or less) except for entertainment value (chats, summaries, tasky read-only agent things).
Earlier quoted context omitted.
In the past month local models have been ramping up in major way meanwhile the namesake providers have upped prices, went offline randomly, and started doing slimier and slimier things. I really think the future is local compute. Or at least self hosted models.
What's the rough equivalent of a local model? Are we talking GPT-4?
Earlier quoted context omitted.
I’m not sure why a model needs to be hosted in order to make network calls?
Is there a library of good tools for LLMs to call? I have to imagine the bot-detection avoidance mechanisms are a major engineering effort and not likely to work out of the box with a simple harness and random local LLM.
`Error: "The following domains are not accessible to our user agent: ['reddit.com']."`
Since they are served as distinct events then I would think they should be easy to block. Once the ads are injected directly into the main response is when things get interesting.