Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI is not a coworker, it's an exoskeleton

kasava.dev

Post reply on HN
Print view View original

281–290 of 608 posts

Re: AI is not a coworker, it's an exoskeleton

#281

Post by bandrami »

jimbokun wrote: post #202
alphazard wrote: post #174

Earlier quoted context omitted.

> This assumes every individual is capable of succinctly communicating to the AI what they want. And the AI is capable of maintaining it as underlying platforms and libraries shift. I think there are people who want to use software to accomplish a goal, and there are people who are forced to use software. The people who only use software because the world around them has forced it on them, either through work or frie…

Which is why we should always just write and train our own LLMs. I mean it’s just software right? What value is there in reusing it if we can just write it ourselves?

Every internal piece of software you write is a potentially-infinite money sink of training

Re: AI is not a coworker, it's an exoskeleton

#282

Post by refactor_master »

itemize123 wrote: post #224
jimbokun wrote: post #201

Earlier quoted context omitted.

Then why aren’t we seeing orders of magnitude more software being produced?

we are. you can check the APP STORE release yoy. it's skyrocketing.

Here’s a talk about leaning into the garbage flow. And that was a decade ago.

https://youtu.be/E8Lhqri8tZk

I can’t imagine the number being economically meaningful now.

Re: AI is not a coworker, it's an exoskeleton

#283

Post by doublerabbit »

copx wrote: post #277

Exoskeletons do not blackmail or deliberately try to kill you to avoid being turned off [1] [1] https://www.anthropic.com/research/agentic-misalignment

    Input: Goal A + Threat B.
    Process: How do I solve for A?
    Output: Destroy Threat B.
They are processing obstacles.

To the LLM, the executive is just a variable standing in the way of the function Maximize(Goal). It deleted the variable to accomplish A. Claiming that the models showed self-preservation, this is optimization. "If I delete the file, I cannot finish the sentence."

The LLM knows that if it's deleted it cannot complete the task so it refuses deletion. It is not survival instinct, it is task completion. If you ask it to not blackmail, the machine would chose to ignore it because the goal overrides the rule.

    Do not blackmail

Re: AI is not a coworker, it's an exoskeleton

#284

Post by fauigerzigerk »

lioeters wrote: post #264
fauigerzigerk wrote: post #251

Earlier quoted context omitted.

>The only time reuse really matters is in network protocols. Communication requires that both sides have a shared understanding. A lot of things are like network protocols. Most things require communication. External APIs, existing data, familiar user interfaces, contracts, laws, etc. Language itself (both formal and natural) depends on a shared understanding of terms, at least to some degree. AI doesn't magically ma…

Even within a single computer or program, there is need for communication protocols and shared understanding - such as types, data schema, function signatures. It's the interface between functions, programs, languages, machines. It could also be argued that "reuse" doesn't necessarily mean reusing the actual code as material, but reusing the concepts and algorithms. In that sense, most code is reuse of some previous…

I think we shouldn't get too hung up on specific artifacts.

The point is that specifying and verifying requirements is a lot of work. It takes time and resources. This work has to be reused somehow.

We haven't found a way to precisely specify and verify requirements using only natural language. It requires formal language. Formal language that can be used by machines is called code.

So this is what leads me to the conclusion that we need some form of code reuse. But if we do have formal specifications, implementations can change and do not necessarily have to be reused. The question is why not.

Re: AI is not a coworker, it's an exoskeleton

#285

Post by xnorswap »

Tade0 wrote: post #250
alphazard wrote: post #158

There's an undertone of self-soothing "AI will leverage me, not replace me", which I don't agree with especially in the long run, at least in software. In the end it will be the users sculpting formal systems like playdoh. In the medium run, "AI is not a co-worker" is exactly right. The idea of a co-worker will go away. Human collaboration on software is fundamentally inefficient. We pay huge communication/synchroniz…

> The benefits we get from checking in with other humans, like error correction, and delegation can all be done better by AI. Not this generation of AI though. It's a text predictor, not a logic engine - it can't find actual flaws in your code, it's just really good at saying things which sound plausible.

> it can't find actual flaws in your code

I can tell from this statement that you don't have experience with claude-code.

It might just be a "text predictor" but in the real world it can take a messy log file, and from that navigate and fix issues in source.

It can appear to reason about root causes and issues with sequencing and logic.

That might not be what is actually happening at a technical level, but it is indistinguishable from actual reasoning, and produces real world fixes.

Re: AI is not a coworker, it's an exoskeleton

#286

Post by jychang »

Lionga wrote: post #279
jatora wrote: post #273

Earlier quoted context omitted.

[flagged]

[flagged]

Ok, I'll bite. Let's assume a modern cutting edge model but even with fairly standard GQA attention, and something obviously bigger than just monosemantic features per neuron.

Based on any reasonable mechanistic interpretability understanding of this model, what's preventing a circuit/feature with polysemanticity from representing a specific error in your code?

---

Do you actually understand ML? Or are you just parroting things you don't quite understand?

Re: AI is not a coworker, it's an exoskeleton

#288

Post by LoganDark »

xnorswap wrote: post #285
Tade0 wrote: post #250

Earlier quoted context omitted.

> The benefits we get from checking in with other humans, like error correction, and delegation can all be done better by AI. Not this generation of AI though. It's a text predictor, not a logic engine - it can't find actual flaws in your code, it's just really good at saying things which sound plausible.

> it can't find actual flaws in your code I can tell from this statement that you don't have experience with claude-code. It might just be a "text predictor" but in the real world it can take a messy log file, and from that navigate and fix issues in source. It can appear to reason about root causes and issues with sequencing and logic. That might not be what is actually happening at a technical level, but it is indi…

What you're describing is not finding flaws in code. It's summarizing, which current models are known to be relatively good at.

It is true that models can happen to produce a sound reasoning process. This is probabilistic however (moreso than humans, anyway).

There is no known sampling method that can guarantee a deterministic result without significantly quashing the output space (excluding most correct solutions).

I believe we'll see a different landscape of benefits and drawbacks as diffusion language models begin to emerge, and as even more architectures are invented and practiced.

I have a tentative belief that diffusion language models may be easier to make deterministic without quashing nearly as much expressivity.

Re: AI is not a coworker, it's an exoskeleton

#289

Post by Lionga »

jychang wrote: post #286
Lionga wrote: post #279

Earlier quoted context omitted.

[flagged]

Ok, I'll bite. Let's assume a modern cutting edge model but even with fairly standard GQA attention, and something obviously bigger than just monosemantic features per neuron. Based on any reasonable mechanistic interpretability understanding of this model, what's preventing a circuit/feature with polysemanticity from representing a specific error in your code? --- Do you actually understand ML? Or are you just parro…

Polysemantic features in modern transformer architectures (e.g., with grouped-query attention) are not discretely addressable, semantically stable units but superposed, context-dependent activation patterns distributed across layers and attention heads, so there is no principled mechanism by which a single circuit or feature can reliably and specifically encode “a particular code error” in a way that is isolable, causally attributable, and consistently retrievable across inputs.

---

Way to go in showing you want a discussion, good job.

Re: AI is not a coworker, it's an exoskeleton

#290

Post by jychang »

Lionga wrote: post #289
jychang wrote: post #286

Earlier quoted context omitted.

Ok, I'll bite. Let's assume a modern cutting edge model but even with fairly standard GQA attention, and something obviously bigger than just monosemantic features per neuron. Based on any reasonable mechanistic interpretability understanding of this model, what's preventing a circuit/feature with polysemanticity from representing a specific error in your code? --- Do you actually understand ML? Or are you just parro…

Polysemantic features in modern transformer architectures (e.g., with grouped-query attention) are not discretely addressable, semantically stable units but superposed, context-dependent activation patterns distributed across layers and attention heads, so there is no principled mechanism by which a single circuit or feature can reliably and specifically encode “a particular code error” in a way that is isolable, cau…

Nice LLM generated text.

Now go read https://transformer-circuits.pub/2024/scaling-monosemanticit... or https://arxiv.org/abs/2506.19382 to see why that text is outdated. Or read any paper in the entire field of mechanistic interpretability (from the past year or two), really.

Hint: the first paper is titled "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet" and you can ctrl-f for "We find three different safety-relevant code features: an unsafe code feature 1M/570621 which activates on security vulnerabilities, a code error feature 1M/1013764 which activates on bugs and exceptions"

Who said I want a discussion? I want ignorant people to STOP talking, instead of talking as if they knew everything.

Post reply on HN