Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Will It Mythos?

swelljoe.com

Post reply on HN
Print view View original

191–200 of 232 posts

Re: Will It Mythos?

#191

Post by delusional »

felipeerias wrote: post #116
p0w3n3d wrote: post #98

I've read opinions that this a speculation to raise the Anthropic's value. They are known to say "horrific things" and personification of the AI they are delivering. It sometimes sounds unprofessional even. This line of communication might have even influenced the courts in the case of copyright violation ("it is not copyright violation if a person learned something and it knows it and thinks of it"). However algorit…

Copyright is a social construct, not an inherent property of the universe. It is whatever we collectively agree it is. In practice, we seem to be leaning towards the idea that training on a copyrighted book is wrong if used to replicate or paraphrase that same book, but not if used to teach a model how to write better.

Property right is a social construct. That doesn't mean you just get to claim "in general I am right" and do whatever you want.

Re: Will It Mythos?

#192

Post by varjag »

kolinko wrote: post #169
varjag wrote: post #65

Earlier quoted context omitted.

Interesting. I tried Fable vs Codex 5.5 xhigh on three different cases. 1. A resource leak with unknown cause. Both of them zoomed onto the same potential issue and proposed almost identical patches. Fable missed an edge case that Codex handled correctly. 2. Review of a SPICE model. Models had different comments, none substantial. Both missed important issues that were simulated inadequately. Clearly a valley where t…

Did you use their native harnesses, or a generic one?

Native for both.

Re: Will It Mythos?

#193

Post by utopcell »

The "best" model finds 4/9 bugs. It would be interesting to see if all models find the _same_ bugs. Does a collection of models exist that can cover all 9?

Also, it seems to me that pointing a model to a bug and asking it to solve it is somewhat easier than what Mythos did, which if I understand correctly, was to generally look at a codebase and find any bug. Even so, non-Mythos models only managed to fix 4/9 of these bugs.

I think the article makes the point that Mythos is at a different level.

Re: Will It Mythos?

#195

Post by scottyah »

clickety_clack wrote: post #174
varjag wrote: post #65

Earlier quoted context omitted.

Interesting. I tried Fable vs Codex 5.5 xhigh on three different cases. 1. A resource leak with unknown cause. Both of them zoomed onto the same potential issue and proposed almost identical patches. Fable missed an edge case that Codex handled correctly. 2. Review of a SPICE model. Models had different comments, none substantial. Both missed important issues that were simulated inadequately. Clearly a valley where t…

I think that Obama-esque, GMAT essay format is the AI flavor that turns me off AI-written articles. It used to be good writing, but because AI locked onto it as such, it's become the watermark of AI generated content.

Oh boy, people are really going to lean into avoiding proper grammar now.

Re: Will It Mythos?

#196

Post by Tossrock »

mjr00 wrote: post #155
l1ng0 wrote: post #118

Earlier quoted context omitted.

You write to the AI as if it were a person. From my point of view it looks like a fair bit of extra typing and extra tokens. Is there a reason you include things like your emotional response and use a very chatty tone? Do you find this seems to alter responses?

I'll go a step further and to say this it's genuinely unsettling someone type to a computer like this. I won't claim to be a psychologist, but with how many instances of "AI psychosis" have been reported (and I've seen first-hand) it seems like treating the computer like a computer is safer, not to mention more effective e.g. lower token usage.

I agree that AI psychosis is a real risk in vulnerable populations (GPT-4o in particular seemed borderline predatory towards those types of people, with its extreme sycophancy), and you should remain clear-eyed while using models. That said, I think exhibiting basic courtesy is still well within the safe-zone. I guess we'll see - I'll be sure to let you know if I end up going psychotic.

Re: Will It Mythos?

#197

Post by anzumitsu »

mjr00 wrote: post #155
l1ng0 wrote: post #118

Earlier quoted context omitted.

You write to the AI as if it were a person. From my point of view it looks like a fair bit of extra typing and extra tokens. Is there a reason you include things like your emotional response and use a very chatty tone? Do you find this seems to alter responses?

I'll go a step further and to say this it's genuinely unsettling someone type to a computer like this. I won't claim to be a psychologist, but with how many instances of "AI psychosis" have been reported (and I've seen first-hand) it seems like treating the computer like a computer is safer, not to mention more effective e.g. lower token usage.

On the other hand I find it quite disturbing to see people be unpleasant or even downright cruel to something that, on a surface level, interacts with you like it’s a thinking, feeling being. Surely you should feel some aversion towards doing so?

I do get where you’re coming from though. I wish these systems had been trained to be clearly robotic and unfeeling.

Re: Will It Mythos?

#198

Post by hodgehog11 »

saberience wrote: post #123
hodgehog11 wrote: post #11

Earlier quoted context omitted.

Hard disagree. Opus reports to me like a student. Fable reported to me like a colleague (researcher). It genuinely seemed to pick up on nuance that the other models just don't, even when I tell them explicitly. It's been really frustrating that neither Codex nor Opus can make targetted edits to Fable's code without screwing something subtle up. For context, this is for computational geometry work, so your mileage may…

Funny, I find Codex to still be better at Coding than Opus or Fable. I A/B tested on a whole array of prompts between Codex and Fable, and Fable almost always found that Codex had produced a better plan and covered more edge cases than it did itself. For every problem I gave the exact same prompt to both models, then I had each analyze the other's output. For roughly 80% of the prompts, Fable acknowledged that Codex'…

I've used them back to back as well. Codex is good at specific tasks; it doesn't try to go big, it does what it's told provided the task is relatively procedural. If Codex can make progress on a task, why would I give it to Fable?

Fable fumbled the one simple task that I gave it too. I gave it multiple very hard open-ended tasks (effectively math tasks) involving research code and it crushed them. It's the first model I've seen that can do that. The current Codex will never produce the type of code Fable gave me no matter how many times I run the same problem at it, because it won't stop trying naive rubbish. And if I tell Codex to try to improve the code, it can't figure out why trying the same classical tricks isn't making it work better, regardless of what I tell it. Opus is marginally better because it can at least recognize some subtleties over time, but still disappointing because it has no idea how to deal with them.

Most programmers want precision instruments for their workflow. That's fine, use the right tool for the job. In my line of work, I need crazy solutions because the obvious stuff doesn't work. That's where Fable shined for me.

Re: Will It Mythos?

#199

Post by antonvs »

ranyume wrote: post #185
antonvs wrote: post #183

Earlier quoted context omitted.

Can you think of many examples of a SaaS provider who regularly keeps old versions of a product around for customers to use? A far more common scenario is that new versions are rolled out to everyone, without offering a choice, as soon as they're considered stable. Older versions consume resources and require staff to spend time on operating and supporting them. Those resources could be used to run a newer version. T…

> Can you think of many examples of a SaaS provider who regularly keeps old versions of a product around for customers to use? Sure. Blender and Ubuntu offer long-lived old versions of their software that get regular fixes.

Neither Blender nor Ubuntu are SaaS. You're just confirming my point: if you want to run old versions of software, you need to host it yourself.

Re: Will It Mythos?

#200

Post by lwarfield »

airstrike wrote: post #173
dd8601fn wrote: post #151

Earlier quoted context omitted.

How do they “nerf the models”? Are they quietly compacting context to reduce kv cache usage, before the actual compaction? Like there’s a slider for how much to compress it, and that’s never revealed to us?

I suspect they quantize them, reduce thinking budgets, batch more requests, or all of the above.

There's also lowering the number of experts you run in MoE models.
Post reply on HN