611–620 of 661 posts
Earlier quoted context omitted.
Probably not a total confidence; it’s EA/rationalist theory taken to ludicrous extremes in both cases.
*coincidence
Earlier quoted context omitted.
CC being better than Cursor didn't make sense to me until I realized Anthropic trains[0] it's models to use it's own built-in tools[1]. 0 - https://x.com/thisritchie/status/1944038132665454841 1- https://docs.anthropic.com/en/docs/agents-and-tools/tool-use...
It still doesn't make sense. Cursor undoubtedly has smart engineers who could implement the Anthropic text editing tool interface in their IDE. Why not just do that for one of your most important LLM integrations?
Earlier quoted context omitted.
In a centrally banked economy, retail and commercial banks create money when you take out loans. The government doesn't create money except during QE which only happened twice in the US, 2009-2014 and 2020-2021. That's why I was curious what you meant by "they." The Fed has been actively destroying money for the last 4 years.
The government creates money every time it spends more than it taxes. AFAIK, the US has been doing that nonstop since the turn of the century. That new money is different from the new money the central bank creates to push interest rates down. That later one the US has been destroying. But both do many of the same things (but not all).
Think about it this way: if money were just created to fund the deficit why would we have a debt? That's double-counting. You can invalidate your hypothesis very easily: the M2 money supply is about half the size of the debt. It's not possible to square that circle unless deficit spending was re-pledging existing money.
Earlier quoted context omitted.
> So long as there is competition it’ll be available at marginal cost. Most things are not perfect competition, so you get MR=MC not P=MC. We're talking about massive capital costs. Another name for massive capital costs are "barriers to entry".
Granted that capital costs are a barrier to entry and that barriers to entry leads to non-perfect competition, but the exploitability is limited in the case of LLMs because they exist on a sub-linear utility scale. In LLMs 2x the price is not 2x as useful, this means a new entrant can enter the lower end of the market and work their way up. The only way to prevent that is for the incumbent to keep costs as close to m…
> There is a natural monopoly aspect given the ability to train and data mine on private usage data but in general improvements in the algorithms and training seem to be dominating advancements.
There's pretty big economies of scale with inference-- the magic of how to route correctly with experts to conduct batching while keeping latency low. It's an expensive technology to create, and there's a large minimum scale where it works well.
Earlier quoted context omitted.
You need a 100+gigs ram and a top of the line GPU to run legacy models at home. Maybe if you push it that setup will let you handle 2 people maybe 3 people. You think anyone is going to make money on that vs $20 a month to anthropic?
> You need a 100+gigs ram and a top of the line GPU to run legacy models at home. Maybe if you push it that setup will let you handle 2 people maybe 3 people. This doesn't seem correct. I run legacy models with only slightly reduced performance on 32GB RAM with a 12GB VRAM GPU right now . BTW, that's not an expensive setup. > You think anyone is going to make money on that vs $20 a month to anthropic? Why does it hav…
Throwing money and compute at AI strikes me as a very short-term solution. In the end, the human brain does not run off a nuclear power plant, not even when we are learning. I expect the next breakthroughs to be all about efficiency. Granted, that could be tomorrow, or in 5 years, and the AI companies have to stay all at in the meantime.
Earlier quoted context omitted.
Yeah, no hate for kung fu here, but maybe learning to better communicate together, act in ways that allows everyone to thrive in harmony and spread peace among all humanity might be a better thing to start incorporating, might not it?
It's literally a scene from The Matrix.
I mean, it's like the djin giving you three whishes, and not a single character will ask "what's the two best wishes I can do to (ensure mankind will reach perpetually best peaceful harmonious flourishing social dynamics forever| whatever goal the character might have as greatest hope)". When you have a instant perfect knowledge acquisition machine at disposal, the first thing to obviously understand is what the most important things to do to reach your goal.
The film didn't mention everything Neo learned like that though, just that he accumulate straight forward for many hours. Wouldn't be an action movie, certainly you would hope the character first words after such an impressive feat wouldn't be "I know kung fu".
Earlier quoted context omitted.
> Everyone knows how to build these models now, it just takes a ton of cash to do it. This ignores differential quality, efficiency, partnerships, and lots more.
Maybe in enterprise. But in the consumer market segment, for most cases, its all about who is cheapest (free preferably) - aside from the few loonies who care about personality. The true lasting economic benefits within enterprise are yet to play out. The trade off between faster code production vs poorer maintained code is yet to play out.
On what basis do you know this? Or more like your personal impression — based on asking how many people? Your friends?
Earlier quoted context omitted.
Also not all compute was necessary for the final model, a large chunk of it is trial and error research. In theory, for $1B you spent training the latest model, a competitor will be able to do it after 6 months with $100M.
Not only are the actual models rapidly devaluing, the hardware is too. Spend $1B on GPUs and next year there's a much better model out that's massively devalued your existing datacenter. These companies are building mountains of quicksand that they have to constantly pour more cash on else they be reduced to having no advantage rapidly.
I don't know the efficiency gains per generation, but let's just say to get the same compute with this 1.25+delta system requires 2x energy. My impression is that while energy is a substantial cost, the total cost for a training run is still dominated by the actual hardware+infrastructure.
It seems like there must be some break even point where you could use older generation servers and come out ahead. Probably everyone has this figured out and consequently the resale value of previous gen chips is quite high?
What's the lifespan at full load of these servers? I think I read coreweave deprecates them (somewhat controversially) over 4 years.
Assuming the chips last long enough, even if they're not usable for LLM training/serving inference, can't they be reused for scientific loads? I'm not exactly old, but back in my PhD days people were building our own little GPU clusters for MD simulations. I don't think long MD simulations are the best use of compute these days, but there's many similar problems like weather modeling, high dimensional optimization problems, materials/radiation studies, and generic simulations like FEA or simply large systems of ODEs.
Are these big clusters being turned into hand-me-downs for other scientific/engineering problems like above, or do they simply burn them out? What's a realistic expected lifespan for a B200? Or maybe it's as simple as they immediately turn their last gen servers over to serve inference?
Lot of questions, but my main question is just how much the hardware is devalued once it becomes previous gen. Any guidance/references appreciated!
Also, anyone still in the academic computing world, do people like de shaw still exist trying to run massive MD simulations or similar? Do the big national computing centers use the latest greatest big Nvidia AI servers or something a little more modest? Or maybe even they're still just massive CPU servers?
While I have anyone who might know, whatever happened to that fad from 10+ years ago saying a lot of compute/algorithms would be shifting toward more memory-heavy models(2). Seems like it kind of happened in AI at least.
(1) Yes I know it's complicated, especially with memory stuff.
(2) I wanna say it was ibm Almaden championing the idea.