Earlier quoted context omitted.
If they are subsidised they can make a profit while still not making enough money to cover energy costs.
Subsidised by whom?
E.g. tax payers.
Things we learned about LLMs in 2024
131–140 of 615 posts
Re: Things we learned about LLMs in 2024
#132About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…
> best LLMs are able to accelerate you https://www2.math.upenn.edu/~ghrist/preprints/LAEF.pdf - this math textbook was written in just 55 days! Paraphrasing the acknowledgements - ...Begun November 4, 2024, published December 28, 2024. ...assisted by Claude 3.5 sonnet, trained on my previous books... ...puzzles co-created by the author and Claude ...GPT-4o and -o1 were useful in latex configurations...doing proof-rea…
"The story of linear algebra begins with systems of equations,
each line describing a constraint or boundary traced upon abstract space.
These simplest mathematical models of limitation — each equation binding variables in measured proportion — conjoin to shape the realm of
possible solutions. When several such constraints act in concert, their collaboration yields three possible fates: no solution survives their collective
force; exactly one point satisfies all bounds; or infinite possibilities trace
curves and planes through the space of satisfaction. This trichotomy — of
emptiness, uniqueness, and infinity — echoes through all of linear algebra, appearing in increasingly sophisticated forms as our understanding
deepens."
Maybe I'm not the target audience, but... that really doesn't make me interested in continuing to read.
Re: Things we learned about LLMs in 2024
#133In spite of all this progress, I can't find LLMs that solve simple tasks like: Here is my resume. Make it look nice (some design hints). They can spit html and css, but not Google doc. On the other hand, Google results are dominated by SEO spam. You can probably find one usable result on page 10. The problem is not technology. It's a business model that can support the humans feeding data into the LLM.
They can spit out LaTeX, and a PDF from that is going to look much nicer than a Google doc (and display the same everywhere). As an added bonus, the recruiter can't randomly rewrite parts of it (at least not so easily).
The recuiter isn't going to print out your resume. They're going to read in their computer or iPad or phone.
Re: Things we learned about LLMs in 2024
#134Earlier quoted context omitted.
It's justified if AGI is possible. If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives you power. That is of course, assuming AGI is possible and exponential, and that marketshare goes to a single entity instead of a set of entities. Lots of big assumptions. Seems like we're heading towards a slow-lackluster singularity though.
If AGI is possible, then the entire human economy stops making sense as far as money goes, and 'owning' part of OpenAI gives you power. That's if AGI is possible and not easily replicated . If AGI can be copied and/or re-developed like other software then the value of owning OpenAI stock is more like owning stock in copper producers or other commodity sector companies. (It might even be a poorer investment. Even AGI…
The first AGI will have such an advantage. It’ll be the first thing that is smart and tireless, can do anything from continuously hacking enemy networks to trading across all investment classes, to basically taking over the news cycle on social media. It would print money and power.
Re: Things we learned about LLMs in 2024
#135I think John Gruber summed it up nicely: https://daringfireball.net/2024/12/openai_unimaginable OpenAI’s board now stating “We once again need to raise more capital than we’d imagined” less than three months after raising another $6.6 billion at a valuation of $157 billion sounds alarmingly like a Ponzi scheme — an argument akin to “Trust us, we can maintain our lead, and all it will take is a never-ending stream of…
What is funny is that their "lead" is just because of inertia - they were the first to make an LLM publicly available. But they are no longer leaders so their attempts at getting more and more money only prove Altman's skills at convincing people to give him money.
Which models perform better than 4o or o1 for your use cases?
In my limited tests (primarily code) nothing from llama or Gemini have come close, Claude I’m not so sure about.
Re: Things we learned about LLMs in 2024
#136About "people still thinking LLMs are quite useless", I still believe that the problem is that most people are exposed to ChatGPT 4o that at this point for my use case (programming / design partner) is basically a useless toy. And I guess that in tech many folks try LLMs for the same use cases. Try Claude Sonnet 3.5 (not Haiku!) and tell me if, while still flawed, is not helpful. But there is more: a key thing with L…
I don't think people finding LLMs useless is a good representation of the general sentiment though. I feel that more than anything, people are annoyed at LLM slop. Someone uses an LLM too much to write code, they create "slop," which ends up making things worse.
Re: Things we learned about LLMs in 2024
#137Earlier quoted context omitted.
The GP said, "and exponential". If AGI is exponential, then the first one will have a head start advantage that compounds over time. That is going to be hard to overcome.
I believe that AGI cannot be exponential for long because any intelligent agent can only approach nature's limits asymptotically. The first company with AGI will be about as much ahead as, say, the first company with electrical generators [1]. A lot of science fiction about a technological singularity assumes that AGI will discover and apply new physics to develop currently-believed-impossible inventions, but I don't…
I don't recall editing my message, but HN can be wonky sometimes. :)
Nothing is truly exponential for long, but the logistic curve could be big enough to do almost anything if you get imaginative. Without new physics, there are still some places where we can do some amazing things with the equivalent of several trillion dollars of applied R&D, which AGI gets you.
Re: Things we learned about LLMs in 2024
#138Earlier quoted context omitted.
What is funny is that their "lead" is just because of inertia - they were the first to make an LLM publicly available. But they are no longer leaders so their attempts at getting more and more money only prove Altman's skills at convincing people to give him money.
They are still in the lead, and I'd be willing to bet that they have 10x the DAU on chat.com/chatgpt.com than all other providers combined. Barring massive innovation on small sub 10B models - we are all likely to need remote inference from large server farms for the foreseeable future. Even in the case that local inference is possible - it's unlikely it will be desirable from a power perspective in the next 3 years.…
Not sure about 10x DAUs. Google flicked the switch on Gemini and it surfaced in pretty much every GSuite app over night.
Re: Things we learned about LLMs in 2024
#139Earlier quoted context omitted.
Did you try learning HOW to get good code out of them? As with all things LLM there's a whole lot of undocumented and under appreciated depth to getting decent results. Code hallucinations are also the least damaging type of hallucinations, because you get fact checking for free: if you run the code and get an error you know there's a problem. A lot of the time I find pasting that error message back into the LLM gets…
Like all AI simps, your blanket response to pointing out flaws is to tell me to do more prompt engineering and then dismiss the issue entirely. In the time it takes me to coax the model to do the thing I was told it knows how to do, I could just do the task myself. Your examples of LLM code generation are simple, easy to specify, self-contained applications that are not representative of software you can actually bui…
> not representative of software you can actually build a business on
The only people pushing that you can BUILD AN APP WITHOUT WRITING A LINE OF CODE are the Twitter AI hypesters. Simon doesn't assert anything of the sort.
LLMs are more-than-sufficient for code snippets and small self-contained apps, but they are indeed far from replacing software engineers.
Re: Things we learned about LLMs in 2024
#140Earlier quoted context omitted.
> If AGI is possible, then the entire human economy stops making sense as far as money goes, What does this mean in terms of making me coffee or building houses?
If we can simulate a full human intelligence at a reasonable speed, we can simulate 100 of them and ask the AGI to figure out how to make itself 10x faster. Rinse and repeat. That is exponential take off. At the point where you have an army of AIs running at 1000x human speed it can just ask it to design the mechanisms for and write the code to make robots that automate any possible physical task.
This sounds like magic, not science.