721–730 of 827 posts
Earlier quoted context omitted.
[flagged]
> It makes no sense at all, it's cuckooland, are you all on crazy pills? Frequent LLM usage impairs thinking. The LLM has no connection to reality, and it takes over people's minds.
Is there hard evidence on this?
Earlier quoted context omitted.
Reddit and YouTube are such huge social media platforms that it really depends on which bubble (read: subreddits/yt channels) you're looking at. There's the "AGI is here" people over at r/singularity and then the "AI is useless" people at r/programming. I'm simplifying arguments from both sides here but you get my point.
Even looking at r/programming I felt they were less wary of AI, or even comparing the comments here vs those on YouTube for this video
Earlier quoted context omitted.
> What you are describing (ie. ditching formal and using natural language) is moving humanity back towards magical thinking ... "Any sufficiently advanced technology is indistinguishable from magic."
indistinguishable from magic != magic
Earlier quoted context omitted.
> maybe we'll come to view deterministic vs. probabilistic (LLMs) similarly I can't believe someone would seriously write this and not realize how nonsensical it is. "indeterministic programming", you seriously cannot come up with a bigger oxymoron.
Why do people keep having this reaction to something we're already used to? When you're developing against an API, you're already doing the same thing, planning for what happens when the request hangs, or fails completely, or gives a different response, and so on. Same for basically any IO. It's almost not even new, just that it generates text instead of JSON, or whatever. But we've already been doing "indeterministi…
Yes, programming isn’t always deterministic, not just due to the leftpad API endpoint being down, but by design - you can’t deterministically tell which button the user is going to click. So far so good.
But, you program for the things that you expect to happen, and handle the rest as errors. If you look at the branching topology of well-written code, the majority of paths lead to an error. Most strings are not valid json, but are handled perfectly well as errors. The paths you didn’t predict can cause bugs, and those bugs can be fixed.
Within this system, you have effective local determinism. In practice, this gives you the following guarantee: if the program executed correctly until point X, the local state is known. This state is used to build on top of that, and continue the chain of bounded determinism, which is so incredibly reliable on modern CPUs that you can run massive financial transactions and be sure it works. Or, run a weapons system or a flight control system.
So when people point out that LLMs are non-deterministic (or technically unstable, to avoid bike-shedding), they mean that it’s a fundamentally different type of component in an engineering system. It’s not like retrying an HTTP request, because when things go wrong it doesn’t produce “errors”, it produces garbage that looks like gold.
Earlier quoted context omitted.
> maybe we'll come to view deterministic vs. probabilistic (LLMs) similarly I can't believe someone would seriously write this and not realize how nonsensical it is. "indeterministic programming", you seriously cannot come up with a bigger oxymoron.
Why do people keep having this reaction to something we're already used to? When you're developing against an API, you're already doing the same thing, planning for what happens when the request hangs, or fails completely, or gives a different response, and so on. Same for basically any IO. It's almost not even new, just that it generates text instead of JSON, or whatever. But we've already been doing "indeterministi…
I try to avoid those, not celebrate them.
Earlier quoted context omitted.
On the other hand, posts like this are like watching someone writing ask jeeves search queries into google 20 years ago and then gesturing how google sucks while everyone else in the room has figured out how to be productive with it and cringes at his "boomer" queries. If you're still struggling to make LLMs useful for you by now, you should probably ask someone. Don't let other noobs on HN +1'ing you hold you back.
Perhaps consider making some tutorials, then, and share your wealth of knowledge rather than calling people stupid.
Great talk, thanks for putting it online so quickly. I liked the idea of making the generation / verification loop go brrr, and one way to do this is to make verification not just a human task, but a machine task, where possible. Yes, I am talking about formal verification, of course! That also goes nicely together with "keeping the AI on a tight leash". It seems to clash though with "English is the new programming l…
This thread perfectly captures what Karpathy was getting at. We're witnessing a fundamental shift where the interface to computing is changing from formal syntax to natural language. But you can see people struggling to let go of the formal foundations they've built their careers on.
It’s in his financial self interest to over inflate LLM’s beyond their “cool math bar trick” level. They are a lossy text compression technique with stolen text sources.
All this “magic” is just function calls behind the scenes doing web/database/math/etc for the LLM.
Anyone who claims LLMs have a soul either truly doesn’t understand how they work (association rules++) or has hitched their financial wagon to this grift. It’s the crypto coin bruhs looking for their next score.
Earlier quoted context omitted.
No doubt the complete cost of training and to getting where we are today has been significant and I don’t know how the accounting will look years from now but you are just making up the rest based on feelings. We know operationally OpenAI is profitable on purely the runtime side, nobody knows how that will look when accounting for R&D but you have no qualification to say they cannot make a profit in any way.
Except they have to retrain constantly, so why would you not consider the cost of training?
They dont have to retrain constantly and that’s where opinions like yours fall short. I don’t believe anyone has a concrete vision on the economics in the medium to a long term. It’s biased ignorance to hold a strong position in the down or up case.
Earlier quoted context omitted.
Couldn’t believe my eyes. The www is truly bankrupt. If anyone has a browser plugin which automatically redirects to llms.txt sign me up. Website too confusing for humans? Add more design, modals, newsletter pop ups, cookie banners, ads, … Website too confusing for LLMs? Add an accessible, clean, ad-free, concise, high entropy, plain text summary of your website. Make sure to hide it from the humans! PS: it should be…
> If anyone has a browser plugin which automatically redirects to llms.txt sign me up. Not a browser plugin, but you can prefix URLs with `pure.md/` to get the pure markdown of that page. It's not quite a 1:1 to llms.txt as it doesn't explain the entire domain, but works well for one-off pages. [disclaimer: I'm the maintainer]
Earlier quoted context omitted.
This is most definitely not toys for rich people. Now perhaps depending on your country it may be considered rich but I would comfortably say that for most of the developed world, the costs for these tools are absolutely attainable, there is a reason ChatGPT has such a large subscriber base. Also the disconnect for me here is I think back on the cost of electronics, prices for the level of compute have generally gone…
I suspect that economies of scale are different for software and hardware. With hardware, iteration results in optimization of the supply chain, volume discount as the marginal cost is so much less than the fixed cost, and lower prices in time. The purpose of the device remains fixed. With software, the software becomes ever more complex with technical debt - featuritis, patches, bugs, vulnerabilities, and evolution…
As for advertising, it’s possible, but with so many competitors and few defensible moats, there’s real pressure to stay ad-free. These tools are also positioned to command pricing power in a way search never was, given search has been free for decades.
The hardware vs. software angle seems like a distraction. My original point was in response to the claim that LLMs are “toys for the rich.” The C64 was a rich kid’s toy too—and far less capable.