91–100 of 1001 posts
This is easily among the rare highest quality articles/comments I've read in the past weeks, perhaps months (on LLMs/AI since that's what I am particularly interested in). And this was for internal consumption before it was made public. Reinforces my recent impression that so much that's being made for public consumption now is shallow and it is hard to find the good stuff. And sadly, increasing so even on HN. As I w…
So I use ChatGPT every day. I like it a lot and it is useful but it is overhyped. Also from 3.5 to 4 the jump was nice but seemed relatively marginal to me. I think the head start OpenAi has will vanish. Iteration will be slow and painful giving google or whoever more than enough time to catch up. ChatGPT was a fantastic leap getting us say 80% to Agi but as we have seen time and time again the last 20% are excruciat…
Personally, the difference between GPT4 and 3.5 is, pretty immense for what I am using it for. I can use GPT 3.5 for things like summarization tasks (as long as the text isn't too complex), reformatting, and other transformation type tasks alright. I don't even bother with using it for logical or programming tasks though.
Repeating myself from https://news.ycombinator.com/item?id=35164971 : > OpenAI can't build a moat because OpenAI isn't a new vertical, or even a complete product. > Right now the magical demo is being paraded around, exploiting the same "worse is better" that toppled previous ivory towers of computing. It's helpful while the real product development happens elsewhere, since it keeps investors hyped about something. >…
> "Please design a syllabus for a course in Computer Architecture and Assembly language, to be taught at the undergraduate level, over a period of six weeks, from the perspective of an professor teaching the material to beginning students."
> "Please redesign the course as an advanced undergraduate six-month Computer Architecture and Assembly program with a focus on the RISC-V ecosystem throughout, from the perspective of a professional software engineer working in the industry."
> "Under the category of Module 1, please expand on "Introduction to RISC-V ISA and its design principles" and prepare an outline for a one-hour talk on this material"
You can do this with any course, any material, any level of depth - although as you go down into the details, hallucinations do become more frequent so blind faith is unwise, but it's still pretty clear this has incredible educational potential.
Having enough scale to perpetually offer free/low-cost compute is a moat. The primary reason ChatGPT went viral in the first place was because it was free, with no restrictions. Back in 2019, GPT-2 1.5B was made freely accessible by a single developer via the TalkToTransformers website, which was the very first time many people talking about AI text generation...then the owner got hit with sticker shock from the GPU…
Charity is only a moat if it’s not profitable.
Feb 24, 2023: Meta launches LLaMA, a relatively small, open-source AI model.
March 3, 2023: LLaMA is leaked to the public, spurring rapid innovation.
March 12, 2023: Artem Andreenko runs LLaMA on a Raspberry Pi, inspiring minification efforts.
March 13, 2023: Stanford's Alpaca adds instruction tuning to LLaMA, enabling low-budget fine-tuning.
March 18, 2023: Georgi Gerganov's 4-bit quantization enables LLaMA to run on a MacBook CPU.
March 19, 2023: Vicuna, a 13B model, achieves "parity" with Bard at a $300 training cost.
March 25, 2023: Nomic introduces GPT4All, an ecosystem gathering models like Vicuna at a $100 training cost.
March 28, 2023: Cerebras trains an open-source GPT-3 architecture, making the community independent of LLaMA.
March 28, 2023: LLaMA-Adapter achieves SOTA multimodal ScienceQA with 1.2M learnable parameters.
April 3, 2023: Berkeley's Koala dialogue model rivals ChatGPT in user preference at a $100 training cost.
April 15, 2023: Open Assistant releases an open-source RLHF model and dataset, making alignment more accessible.
The ability to connect these models to the web, to pipe up API access to different services and equip LLMs to be the new interface to these services and to the worlds information is the real game changer.
Google cannot out innovate them because they are a big Corp rife with googly politics and challenges of overhead that come with organizational scale.
I would be curious to see if there are plans to spin off the newly consolidated AI unit with their own PnL to stimulate that hunger to grow and survive and then capitalize them accordingly. Otherwise they are en route to die a slow death once better companies come along.
"Many of the new ideas are from ordinary people." Yeah. Google can fuck right off. Maybe this attitude is what got them in the weeds in the first place.
This looks very fake to me. I might be wrong. Yet, there is no "document" that was leaked, the original source is some blog post. If there is a document, share the document. Shared by "anonymous individual on discord who granted permission for republication"... I don't know. If it was shared by anonymous, why ask for permission? Which discord server?
it'll be fun to see the pikachu face when engineers are expected to do more, with the aid of these tools, but are not paid any more money.
If they’re able to produce twice the work in half the time, wouldn’t it make sense to pay them less?
This is easily among the rare highest quality articles/comments I've read in the past weeks, perhaps months (on LLMs/AI since that's what I am particularly interested in). And this was for internal consumption before it was made public. Reinforces my recent impression that so much that's being made for public consumption now is shallow and it is hard to find the good stuff. And sadly, increasing so even on HN. As I w…
But the bulk of the tasks can probably be solved at 3.5 level, another more difficult chunk with 4, I'm wondering how many of the requests will be so complex as to require GPT-5. Probably less than 1%.
There's a significant distinction between web search and generative AI. You can't download "a Google" but you can download "a LLaMA". This marks the end of the centralisation era and increased user freedom. Engaging in chat and image generation without being tracked is now possible while searching, browsing the web or torrenting are still tracked.
Some snippets for folks who come just for the comments: > While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open-source models are faster, more customizable, more private, and pound-for-pound more capable. They are doing things with $100 and 13B params that we struggle with at $10M and 540B. And they are doing so in weeks, not months. > A tremendous outpouring of…
I find it incredibly cathartic to see these massive tech companies and their gatekeepers get their lunch eaten by OSS.