161–170 of 199 posts
He spend 200 to upgrade his existing setup, great nonetheless. But not "32gb gram for only 200 bucks"
I was very close to buying a retired POWER7+ server with an ungodly amount of memory, but decided being unable to run a modern Linux kernel would be more work than I wanted to have. Modern kernels need POWER8 and above.
I was very close to buying a retired POWER7+ server with an ungodly amount of memory, but decided being unable to run a modern Linux kernel would be more work than I wanted to have. Modern kernels need POWER8 and above.
OTOH, if these chips were fully supported, they wouldn’t hit the second hand market at the prices they do.
Earlier quoted context omitted.
> We might have just shot our most valuable non-AI tech products in the foot. Counterpoint: the fiber buildout during the dotcom boost. That crashed the economy pretty hard when the bubble burst, but we are still benefitting from all the dark fiber that was arranged for and built out back in that era. A lot of today's ISPs were able to grab up that fiber after the bust for cents on the dollar. Assume that OpenAI and…
You're expecting that there's going to be a supply collapse only, but there's a real risk the collapse hits both supply and demand. A lot of the current AI business is FOMO and vanity metrics. Nobody really wants to acknowledge the support tickets where the first three responses are the customer cursing because they didn't appreciate being handed off to a chatbot, or the reworks, or the compliance/policy/privacy conc…
That was what I alluded to in the last paragraph. Semiconductor industry and everything associated with it will get screwed hard.
But in case you mean a demand collapse from the entire economy because even more people get laid off... yes, agreed. Dotcom bubble bust, here we come, full steam ahead.
> We probably don't need AI-powered support experiences that manage to be worse than actually keyword-searching your company's Confluence.
I'd pay good money for an AI that could actually ingest Confluence. In literally every organization that does not have a dedicated team to manage it on all aspects, it inevitably devolves into a tire fire. Unfortunately there's no easy way (yet?) to "after-train" a model - what I'd envision here is a nightly batch job that adds another layer to the AI model from all the information in the Confluence so searches don't incur a giant cost for the AI agent to process everything.
> We probably don't need to be spinning up coding agents to spend 15 minutes discombobulating and bibblewabbling and re-reading 82 billion tokens of context before making a two-line change that an actual developer with learned experience in the code would make in 15 seconds.
Oh we do. The stonk markets don't like it when companies employ people. People need office space, they need associated services (say, IT, fruit baskets and other amenities), they need wages, and in everywhere but the US you can't just go and fire them on a whim. The less people an organization has, the better the company looks on an investor relations press release. That is why the large AI organizations are investing untold billions of dollars... the race to be the first one that can fully replace a class of human employment. Say an SWE makes 130k/y on average - fire 100 of the 150 you have, that's 13 million dollars. That can buy you a looooot of tokens or hardware.
Earlier quoted context omitted.
Where do you think llms learned to write that way?
Because their custom training data contains an emphasis on such verbiage. It doesn't come from the God-knows-how-many TB of web content the model is pre-trained on. There, such phrasing is only a drop in the sea. But the "yes, you're right" phrases, the em dash, etc., come from the later stage, for which content is created according to some (probably overprecise) guidelines.
I have not done the textual and statistical analysis to verify this, but I feel like it's something you could trace back to east coast journalism schools and publishers mediated via television, which long predates mass adoption of AI. Think how many news articles you've read with titles like 'Anatomy of a murder' os 'Inside the meeting that changed everything.' The hooky, slightly pompous tone is something you can find back as far as the 1960s or 1970s; browsing through old issues of Readers Digest and you'll find tons of it. When I say it's mediated through television, I'm talking about both the dramatic and heavily conclusory style of fictional prosecutors and narrators, and the extremely shallow style of TV news reports (often transcribed to the web) which are only one or two sentences per paragraph. And this is before we consider the stylistic impact of ad copywriting on communication in general.
And there's something else.
The one sentence paragraph interjection, designed to refocus your attention in a surprising new direction after two paragraphs of stuff you already know. 'I never thought I'd end upere,' said Sally Nocontext, hooking you in for another paragraph or two where you try to figure out who this woman is, where she ended up, and what it has to do with the article you are already halfway through reading. After all, I've come this far, the reader through. I might as well see it through to the end.
And that's just what publishers wanted.
One sentence can also validate a truism that the reader already suspects, flattering their beliefs in their own analytical powers....
...well you get the idea. When I'm using LLMs for any sort of extended session, I find myself reaching for the same few prompts to break it of such clicheed expression; I'm especially averse to the habit of adding zippy-sounding nicknames to complex or potentially dull concepts. I don't have a favorite starting prompt, but I generally find that asking for 'a concise, academic tone' does wonders to de-fluff its output. Remember, it defaults toward being as widely accessible as possible, and much journalism is aimed at consumers with only a high school education and maybe middle-school reading comprehension, math ability, and appetite for depth over sensation.
This is great! I've been trying to get into local models for a while as I share the sentiment that local models will eventually be so good that there won't be a need to use frontier models for most coding tasks (perhaps that's already true today?). I have zero experience building computers - where would I even start? I mean, aside from the things already well documented and mentioned in the blog post.
I built my first Pentium 4 one when I was like six, so I’m sure someone much older that’s into tech can do it without an issue.
There are also tons of Discord communities that are willing to help you live if you encounter any issues.
The V100 and the 4090 are based on vastly different architectures, the former uses the older Volta while the latter uses Ada. Last I checked you cannot meaningfully combine them. The 3090 is better than the V100, just get two 3090 and a NVLink.
Very interesting, but slight note on the fact that writer sort of forgets to calculate in the price of the 4080 it works together with. He spend 200 to upgrade his existing setup, great nonetheless. But not "32gb gram for only 200 bucks"
Great value for money if you have the time for tinkering and getting the compatibility to work.