221–230 of 403 posts
Earlier quoted context omitted.
How many tokens per second are you getting from Alpaca?
When llamaCPP came out, I was running 13B at 100ms/token on a base model MacBook Pro 14". Edit: apparently llama.cpp supports running on GPU, so I imagine it's gonna be a bit faster. Maybe a fun evening project for me to get going.
Earlier quoted context omitted.
I agree. I was really impressed that even with regard to the Autocorrect update, they used technical terms like "transformer model" without using hype words. They very clearly labeled it as a predictive text engine rather than some magical pseudo-sentient enigma or something.
Much better than their previous marketing 15 years ago measuring hard drive capacities in "songs".
It seems clear that Apple never wanted to launch the Intel Mac Pro (cheese grater), but they saw a timing gap between the trash-can Mac Pro and the Mac Studio that needed to be filled.
I’ve been draggggggiiinnngggg my feet on a new desktop workstation. Waiting for the store to come back to truly make a decision but I think that I’m gonna go for a Studio. The hacker in me wants to build a beefy Linux workstation but the pragmatist in me wants a machine that just works. I think the Apple tax is worth it here.
I'm surprised they kept the cheese grater case for the new Pro. It is one of my least favorite case designs of any high end mac. I'm really surprised they didn't go with something simpler and more like a tall, scaled up Mac Studio. It's strange that given it is such a big architecture change on the inside isn't mirrored with a physical change on the outside. Mostly I hate the juxtaposition of the chrome legs/handles…
I think the G5 case was peak design for a tower that's hard to top but Apple's surprised me before.
Earlier quoted context omitted.
> It's such a specific use case that I'm not entirely sure what the use case even is. Live TV production, I think. Mostly in the rackmount form-factor. A plethora of "IO breakout boards" is what turns a regular computer into a "video production system" head-unit. Though also, at least three of the PCI-e cards shown on the slide were for fibre-optic networking. So, presumably, this would be the Mac to get if you're tr…
I liked how one of the cards on the screen was a sound card, as if a PCIe slot wasn't 1000x overkill for that amount of I/O. We had USB ports that could handle that when Bill Clinton was still president of the United States.
There was also something there that had DB9 and DB25 connectors, but both female. (I would think this was a weird SuperIO card, but the computer side of a serial port is usually male.) There was also a lot of stuff on that card. Anyone know what that one was?
Congrats to Apple on completing another IA migration! It really is an incredible accomplishment to get such a massive base of customers, partners, and developers to run on a new architecture so quickly and relatively seamlessly. I remember the PPC to Intel move, which was also well done, and they'd improved even on that ... with what must be many many more users. Awesome! P.S. Hopefully this transition frees someone…
(Well there was no memory protection in those days, so everything was in one address space. Still, impressive!)
Earlier quoted context omitted.
Mac Pro = Mac Studio + Expansion Cards I have to imagine this will be a huge disappointment to some, because 192GB of shared memory is way less than the 1.5TB of RAM available on the "old" Mac Pro.
Perhaps someone will come out with a new PCIe card with a load of RAM slots on it, and then writes a kernel driver to map the pcie card pages to appear as regular pages
Earlier quoted context omitted.
Except that my fairly modest upper midrange desktop has ~1.2TB/sec of memory bandwidth... Cause, I just added the GPU and system RAM bandwidth numbers together. Which is what needs to be kept in mind with much of this. Yes that is a lot of memory bandwidth and its hella useful for some subset of users, but its shared, and largely pointless for a lot of CPU bound tasks. But OTOH, may not be enough for many GPU bound o…
I just built a workstation with a 32 core Threadripper Pro, 128 GB ECC RAM, Thunderbolt 4, and an RTX 4090 for $6,500.
Two data points from Apple:
a) M2 Ultra 24 cores, 128GB ram, 2TB storage = $5,200
b) M2 ultra 24 cores, 192GB ram, 4GB storage = $6,600
Earlier quoted context omitted.
> large language models are definitely AI The term “AI” is becoming over inclusive to the point of meaninglessness. Cupertino is smart enough to pick up on that. “Statistical linguistics” is the best general term for LLMs I’ve come across.
LLMs are the apex of NLP research to date, and NLP is obviously a branch of AI. You may have some sophisticated notion of AI, AGI, human-level or human-like AI or whatever, but NLP has been considered AI for generations now.
I'm arguing the term AI has become "inclusive to the point of meaninglessness." That doesn't mean it was always meaningless.