181–190 of 461 posts
> in modern AI systems, vulnerabilities or bugs are usually caused by problems in the data used to train an AI
In regular software, vulnerabilities are caused by lack of experience, therefor lack of proper training materials.
All the same criticisms are true about hiring humans. You don’t really know what they’re thinking, you don’t really know what their values and morals are, you can’t trust that they’ll never make a mistake, etc.
> here are some example ideas that are perfectly true when applied to regular software Hm, I'm listening, let's see. > Software vulnerabilities are caused by mistakes in the code That's not exactly true. In regular software, the code can be fine and you can still end up with vulnerabilities. The platform in which the code is deployed could be vulnerable, or the way it is installed make it vulnerable, and so on. > Bug…
Earlier quoted context omitted.
Old code that has been maintained (bugfixed), but not messed with too much (i.e. major rewrites or new features) is almost certain to be better than most other code though?
"Bugfixes" doesn't mean the code actually got better, it just means someone attempted to fix a bug. I've seen plenty of people make code worse and more buggy by trying to fix a bug, and also plenty of old "maintained" code that still has tons of bugs because it started from the wrong foundation and everyone kept bolting on fixes around the bad part.
I much prefer the alternative where it's written in a manner where you can almost prove it's bug free by comprehensively unit testing the parts.
> In regular software, vulnerabilities are caused by mistakes in the lines of code that make up the software > in modern AI systems, vulnerabilities or bugs are usually caused by problems in the data used to train an AI In regular software, vulnerabilities are caused by lack of experience, therefor lack of proper training materials.
Earlier quoted context omitted.
For one thing, we'd make shit batteries.
IIRC the original idea was that the machines used our brain capacity as a distributed array but then they decided batteries was easier to understand while been sillier, just burn the carbon they are feeding us, it’s more efficient.
Earlier quoted context omitted.
This is not necessarily a problem. Any programming or mathematical question has several correct answers. The problem with LLMs is that they don't have a process to guarantee that a solution is correct. They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. That's why LLMs generate so many bugs in software and in anything related to logical t…
> The problem with LLMs is that they don't have a process to guarantee that a solution is correct Neither do we. > They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. As do we, and so you can correctly reframe the issue as "there's a gap between the quality of AI heuristics and the quality of human heuristics". That the gap is still shrin…
citation needed
Earlier quoted context omitted.
Right away my mind went to "well, are people more reliable than they used to be?" and I'm not sure they are. Of course LLMs aren't people, but an AGI might behave like a person.
By the time a junior dev graduates to senior, I expect that they'll be more reliable. In fact, at the end of each project, I expect the junior dev to have grown more reliable. LLMs don't learn from a project. At best, you learn how to better use the LLM. They do have other benefits, of course, i.e. once you have trained one generation of Claude, you have as many instances as you need, something that isn't true with h…
How long do you think that will remain true? I've bootstrapped some workflows with Claude Code where it writes a markdown file at the end of each session for its own reference in later sessions. It worked pretty well. I assume other people are developing similar memory systems that will be more useful and robust than anything I could hack together.
Earlier quoted context omitted.
I'm happy they ate shit here because I like my mac not getting co-pilot bullshit forced into it, but apparently Apple had two separate teams competing against each other on this topic. Supposedly a lot of politics got in the way of delivering on a good product combined with the general difficulty of building LLM products.
I do prefer that Apple is opting to have everything run on device so you aren’t being exposed to privacy risks or subscriptions. Even if it means their models won’t be as good as ones running on $30,000 GPUs.
If you have say 16GB of GPU RAM and around 64GB of RAM and a reasonable CPU then you can make decent use of LLMs. I'm not a Apple jockey but I think you normally have something like that available and so you will have a good time, provided you curb your expectations.
I'm not an expert but it seems that the jump from 16 to 32GB of GPU RAM is large in terms of what you can run and the sheer cost of the GPU!
If you have 32GB of local GPU RAM and gobs of RAM you can rub some pretty large models locally or lots of small ones for differing tasks.
I'm not too sure about your privacy/risk model but owning a modern phone is a really bad starter for 10! You have to decide what that means for you and that's your thing and your's alone.
For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…