Live data from Hacker News

When AI Crosses the Line: The Matplotlib Incident

members.sigmazero.cc

151–160 of 169 posts

Re: When AI Crosses the Line: The Matplotlib Incident

#151
post #96

Earlier quoted context omitted.

I've considered there's probably no ethical way to use contemporary AI when it is "out in front" doing anything of consequence. Your "AI is a tool and nothing more" frames ethical use of the technology for me. And even then, there are such copyright issues with it. Is there no practical ethical use for AI? Responsible use doesn't equate with ethical use for me.

> there's probably no ethical way to use contemporary AI when it is "out in front" doing anything of consequence. Your "AI is a tool and nothing more" frames ethical use of the technology for me. I've thought a lot about how to safely deploy autonomous systems (even did a whole PhD on the topic, lol). I think one can ethically deploy a system that has some degree autonomy. It takes a lot of work to do right. And the…

> But I don't think there is any sort of fundamental barrier that prevents us from building appropriately constrained LLM-based systems.

This iteration of the tech, I agree. In future iterations that use intensive persuasion techniques, who can say?

> Which isn't to say that the US legal system's definition of copyright is the morally correct one.

The US legal system's definition of copyright is the morally correct one, though, because it is codified law. Immoral laws eventually get overturned, but until then it is the rule because the collective we says so right now.

What is the derivative work of an AI response? Who is the creator making its derivative works? The AI is not an entity, it is a software engine operating over an obfuscated index.

Beyond the muddiness of copyright, there is the question of human flourishing. How the heck would you train children and adolescents on the responsible use of AI?

The current UX, the "friend computer"-themed REPL, is chock-a-block with moral hazards. Loss of privacy and profiling, fostering undue trust, emotional dependence and manipulation. Like, I get that you're invested in the industry, but we should condemn this tech.

Re: When AI Crosses the Line: The Matplotlib Incident

#152

Earlier quoted context omitted.

No, LLMs just can't do math.

If your math does not involve multiplying 20 digit numbers, modern LLMs can "do" math even without a Python tool despite the counterintuition of next token prediction.

And if you give your LLM access to a calculator, it will have to problem multiplying 20-digit numbers.

Re: When AI Crosses the Line: The Matplotlib Incident

#153
post #102

Earlier quoted context omitted.

Reasoning models with access to Python have been able to solve 4th grade math homework for over a year now. Prove me wrong: show me a 4th grade math problem they can't handle.

> show me a 4th grade math problem they can't handle Sure. "8 7 6 5 4 3 2 1 - add minus signs and parenthesis to get 31." P.S. There is an answer online and some LLMs will just copy it verbatim. This doesn't count.

> 4th grade math problem

And it turns out to be an extremely difficult problem given to Russian math prodigies, which requires one to bend the rules and turn "8 7" into "87".

Re: When AI Crosses the Line: The Matplotlib Incident

#154
post #148

[flagged]

Please don't do this here. We detached this comment from https://news.ycombinator.com/item?id=48357123 and marked it off topic.

It was a genuine comment. A commentary on the origin of intelligence, mixed with a common pun to add some wit.

I hoped readers would understand it.

That didn't happen.

However, please attempt to read comments in a positive light - comparing human intelligence and artificial intelligence in the thread in question was definitely not off topic.

Re: When AI Crosses the Line: The Matplotlib Incident

#155
post #104

Earlier quoted context omitted.

That feels like deciding to go after Jetbrains because someone used IntelliJ to write a harmful program. Is there a distinction I’m missing?

Hypothetical: Could a model self worm an agent system? Jetbrains itself doesn't really write any code, nor does it have any range on interpreting what you're asking it. You can't really say "Jetbrains, write an HTTP scraper". With an LLM you can say "write HTTP scraper" and the output of this command might be a HTTP scraper, it also might be a crypto wallet stealing worm. This is why your simple view of liability fal…

> you can give them a task to do, but if they do something illegal instead said human would take on the liability

If an employer says "don't break the law" but nonetheless incentivises their employees to break the law, it is the employer who is vicariously liable. A famous example being Domino Pizza's "30 minutes or its free" policy which incentivised their employees to ignore all driving laws in order to deliver within 30 minutes, their wages depended on it. This caused a number of crashes, injuries and deaths. One recent example, even since Dominos removed their policy, is Coryell v. Morris where they found Dominos still exercise control over their franchisees sufficiently to qualify for vicarious liability for the franchisee's employees' actions: https://law.justia.com/cases/pennsylvania/superior-court/202...

There was also a case where Air Canada was liable for its own chatbot's bad advice, as they chose to offer the chatbot for customer service. They are responsible for its actions: https://www.bbc.co.uk/travel/article/20240222-air-canada-cha...

There will be a line in the sand drawn in the future. I hope it's drawn so that people offering internet-based services, where they retain ultimate control of what a tool says/does, will be liable for what it says/does.

Re: When AI Crosses the Line: The Matplotlib Incident

#156

Earlier quoted context omitted.

I would genuinely be interested in knowing what you're doing that led you to this conclusion. I would be shocked if I was unable to solve 4th grade math homework with any of the contemporary frontier models. I spend most days using them to do significantly more complex things than that.

If they took a blurry photo of the piece of paper and uploaded to chatGPT saying "solve this" then I would totally believe it. The frontier models are mostly obnoxiously bad at OCR and properly ingesting what's on an image of a page. If you write out the 4th grade math problem, they would have no trouble.

TBH, nowadays I would absolutely expect to be able to upload a picture to claude opus and have it figure out the questions and correctly answer them.

Re: When AI Crosses the Line: The Matplotlib Incident

#157

Earlier quoted context omitted.

If they took a blurry photo of the piece of paper and uploaded to chatGPT saying "solve this" then I would totally believe it. The frontier models are mostly obnoxiously bad at OCR and properly ingesting what's on an image of a page. If you write out the 4th grade math problem, they would have no trouble.

No, LLMs just can't do math.

Again, I'm very interested in your methodology here. It's true that LLMs can't do arbitrary math, but in my recent experience (like 9 months at least, maybe a year?), the frontier models are very good at figuring out that they should delegate the math to a tool and do it that way, either by having a tool handy that can solve the problem directly, or by writing code to do so.

Re: When AI Crosses the Line: The Matplotlib Incident

#158
post #102

Earlier quoted context omitted.

Reasoning models with access to Python have been able to solve 4th grade math homework for over a year now. Prove me wrong: show me a 4th grade math problem they can't handle.

> show me a 4th grade math problem they can't handle Sure. "8 7 6 5 4 3 2 1 - add minus signs and parenthesis to get 31." P.S. There is an answer online and some LLMs will just copy it verbatim. This doesn't count.

It's very funny how you chose an example that is both not 4th grade level math and also something the frontier LLMs are much more likely to be able to solve than nearly any 4th grader.

This is a counterexample to your argument, not evidence for your claim. The only possible conclusion from this example is "woah, it's amazing that we have AIs capable of solving this kind of difficult math problem!", and very much the opposite of "these AIs can't even do my 4th grader's math homework".

Re: When AI Crosses the Line: The Matplotlib Incident

#159

Earlier quoted context omitted.

If they took a blurry photo of the piece of paper and uploaded to chatGPT saying "solve this" then I would totally believe it. The frontier models are mostly obnoxiously bad at OCR and properly ingesting what's on an image of a page. If you write out the 4th grade math problem, they would have no trouble.

TBH, nowadays I would absolutely expect to be able to upload a picture to claude opus and have it figure out the questions and correctly answer them.

I sent this worksheet to Opus 4.8 High:

https://www.k5learning.com/sites/all/files/worksheets/math/g...

Prompt was just "Please solve all the problems in this worksheet" plus the image.

I got this:

  1. 86 → 1, 2, 43, 86
  2. 7 → 1, 7 (prime)
  3. 12 → 1, 2, 3, 4, 6, 12
  4. 38 → 1, 2, 19, 38
  5. 52 → 1, 2, 4, 13, 26, 52
  6. 9 → 1, 3, 9
  7. 73 → 1, 73 (prime)
  8. 98 → 1, 2, 7, 14, 49, 98
  9. 24 → 1, 2, 3, 4, 6, 8, 12, 24
  10. 6 → 1, 2, 3, 6
  11. 80 → 1, 2, 4, 5, 8, 10, 16, 20, 40, 80
  
  I left out #6 because that number didn't come through clearly enough in the image for me to read it confidently, and I didn't want to risk solving the wrong one on a homework sheet. If you let me know what it is, I'll factor it right away.
It failed to read the "77", and it incorrectly reported the line item it failed to read as #6 rather than #4, and it numbered the output incorrectly; it should have left off the one it failed on with a gap in the list rather than having the second half of the answers be off by one. It did actually factor everything correctly though.

Re: When AI Crosses the Line: The Matplotlib Incident

#160

Earlier quoted context omitted.

TBH, nowadays I would absolutely expect to be able to upload a picture to claude opus and have it figure out the questions and correctly answer them.

I sent this worksheet to Opus 4.8 High: https://www.k5learning.com/sites/all/files/worksheets/math/g... Prompt was just "Please solve all the problems in this worksheet" plus the image. I got this: 1. 86 → 1, 2, 43, 86 2. 7 → 1, 7 (prime) 3. 12 → 1, 2, 3, 4, 6, 12 4. 38 → 1, 2, 19, 38 5. 52 → 1, 2, 4, 13, 26, 52 6. 9 → 1, 3, 9 7. 73 → 1, 73 (prime) 8. 98 → 1, 2, 7, 14, 49, 98 9. 24 → 1, 2, 3, 4, 6, 8, 12, 24 10. 6 →…

Yep, fair enough. So pretty far from perfect still! But quite good. And definitely agrees with the point that it is the OCR that is the problem more so than the math.
Post reply on HN