Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone in Seattle hates AI

jonready.com

Post reply on HN
Print view View original

641–650 of 1001 posts

Re: Everyone in Seattle hates AI

#641

Post by sfink »

eru wrote: post #628
pkasting wrote: post #40

Ex-Google here; there are many people both current and past-Google that feel the same way as the composite coworker in the linked post. I haven't escaped this mindset myself. I'm convinced there are a small number of places where LLMs make truly effective tools (see: generation of "must be plausible, need not be accurate" data, e.g. concept art or crowd animations in movies), a large number of places where LLMs make…

> [...] a large number of places where LLMs make apparently-effective tools that have negative long-term consequences (see: anything involving learning a new skill, [...] Don't people learn from imperfect teachers all the time?

Yes, they do. In fact, imperfect teachers can sometimes induce more learning than more perfect ones. And that's what is insidious about learning from AI. It looks like something we've seen before, something where we know how to make it useful and take advantage even of the gaps and inadequacies.

AI can be effective for learning a new skill, but you have to be constantly on your guard to prevent it from hacking your brain and making you helpless and useless. AI isn't the parent holding your bicycle and giving you a push and letting go when you're ready. It's the welded-on training wheels that become larger and more structurally necessary until the bike can't roll forward at all without them. It feeds you the lie that all you need is the theory, you don't ever need to apply it because the AI will do that for you so don't worry your pretty little head over it. AI teaches you that if something requires effort, you're just not relying on the AI enough. The path to success goes only through AI, and those people who try to build their own skills without it are suckers because the AI can effortlessly create things 100x bigger and better and more complex.

Personally, I still believe that human + AI hybrids have enormous potential. It's just that using AI constantly pushes away from beneficial hybridization and towards dependency. You have to constantly fight against your innate impulses, because it hacks them to your detriment.

I'd actually like to see an AI trained to not give answers, but to search out the point where they get you 90% of the way there and then steadfastly refuse to give you the last 10%. An AI built with the goal not of producing artifacts or answers, but of producing learning and growth in the user. (Then again, I'd like to see the same thing in an educational system...)

Re: Everyone in Seattle hates AI

#642

Post by throwaway-0001 »

musebox35 wrote: post #635
throwaway-0001 wrote: post #606

Earlier quoted context omitted.

I think there is a correlation between when you can you expect from something when I know their internals vs someone that doesn’t know but is not like who knows internals is much much better. Example: many people created websites without a clue of how they really work. And got millions of people on it. Or had crazy ideas to do things with them. At the same time there are devs that know how internals work but can’t ge…

I think this is more about mechanistic understanding vs fundamental insight kind of situation. The linear algebra picture is currently very mechanistic since it only tells us what the computations are. There are research groups trying to go beyond that but the insight from these efforts are currently very limited. However, the probabilistic view is very much clearer. You can have many explorable insights, both potent…

If were true then people could predict this AI many years ago

Re: Everyone in Seattle hates AI

#643

Post by musebox35 »

crystal_revenge wrote: post #611
throwaway-0001 wrote: post #606

Earlier quoted context omitted.

I think there is a correlation between when you can you expect from something when I know their internals vs someone that doesn’t know but is not like who knows internals is much much better. Example: many people created websites without a clue of how they really work. And got millions of people on it. Or had crazy ideas to do things with them. At the same time there are devs that know how internals work but can’t ge…

I'm a big fan of the concept of 初心 (Japanese: Shoshin aka "beginners mind" [0] ) and largely agree with Sazuki's famous quote: > “In the beginner’s mind there are many possibilities, but in the expert’s there are few” Experts do tend to be limited in what they see as possible. But I don't think that allows carte blanche belief that a fancy Markov Chain will let you transcend humanity. I would argue one of the key con…

I think this is what is blunted by mass education and most textbooks. We need to discover it again if we want to enjoy our profession with all the signals flowing from social media about all the great things other people are achieving. Staying stupid and hungry really helps.

Re: Everyone in Seattle hates AI

#644

Post by solid_fuel »

insin wrote: post #619
solid_fuel wrote: post #597

Earlier quoted context omitted.

I think the dynamic is different - before, they were writing and testing the functions and features as they went. Now, (some of) my coworkers just push a PR for the first or second thing copilot suggested. They generate code, test it once, it works that time, and then they ship it. So when I am looking through the PR it's effectively the _first_ time a human has actually looked over the suggested code. Anecdote: In t…

Then when you've taken an hour to be the first person to understand how their code works from top to bottom and point out obvious bugs, problems and design improvements (no, I don't think this component needs 8 useEffects added to it which deal exclusively with global state that's only relevant 2 layers down, which are effectively treating React components like an event handling system for data - don't believe people…

Yep, and if you're lucky they actually paste your comments back into the LLM. A lot of times it seems like they just prompted for some generic changes, and the next revision has tons of changes from the first draft. Your job basically becomes playing reviewer to someone else's interactions with an LLM.

It's about as productive as people who reply to questions with "ChatGPT says " except they're getting paid to do it.

Re: Everyone in Seattle hates AI

#645

Post by arresin »

sudoshred wrote: post #420
sirreal14 wrote: post #65

As a Seattle SWE, I'd say most of my coworkers do hate all the time-wasting AI stuff being shoved down our throats. There are a few evangelical AI boosters I do work with, but I keep catching mistakes in their code that they didn't used to make. Large suites of elegant looking unit tests, but the unit tests include large amounts of code duplicating functionality of the test framework for no reason, and I've even seen…

As someone on a team with a less stringent code review culture, AI generated code creates more work when used indiscriminately. Good enough to get approved but full of non-obvious errors that cause expensive rework which only gets prioritized once the shortcomings become painfully obvious (usually) months after the original work was “completed” and once the original author has forgotten the details, or worse, left th…

Exactly it’s the non-obvious errors that are easy to miss—doubly so if you are just scanning the code. Those errors can create very hard to find bugs.

So between the debugging and many times you need to reprompt and redo (if you bother at all, but then that adds debugging time) is any time actually saved?

I think the dust hasn’t settled yet because no one has shipped mostly AI generated code for a non-trivial application. They couldn’t have with its current state. So it’s still unknown whether building on incredibly shaky ground will actually work in real life (I personally doubt it).

Re: Everyone in Seattle hates AI

#646

Post by astrange »

panarky wrote: post #475
josephg wrote: post #421

Earlier quoted context omitted.

Well yeah. And because when an expert looks at the code chatgpt produces, the flaws are more obvious. It programs with the skill of the median programmer on GitHub. For beginners and people who do cookie cutter work, this can be incredible because it writes the same or better code they could write, fast and for free. But for experts, the code it produces is consistently worse than what we can do. At best my pride dem…

> It programs with the skill of the median programmer on GitHub This is a common intuition but it's provably false. The fact that LLMs are trained on a corpus does not mean their output represents the median skill level of the corpus. Eighteen months ago GPT-4 was outperforming 85% of human participants in coding contests. And people who participate in coding contests are already well above the median skill level on…

Chess AI trained at specific human levels performs better than any humans at those levels, because the random mistakes get averaged out.

https://www.maiachess.com

Re: Everyone in Seattle hates AI

#647

Post by Arainach »

CSMastermind wrote: post #223
pkasting wrote: post #40

Ex-Google here; there are many people both current and past-Google that feel the same way as the composite coworker in the linked post. I haven't escaped this mindset myself. I'm convinced there are a small number of places where LLMs make truly effective tools (see: generation of "must be plausible, need not be accurate" data, e.g. concept art or crowd animations in movies), a large number of places where LLMs make…

My friends at Google are some of the most negative about the potential of AI to improve software development. I was always surprised by this and assumed internally at Google would be one of the first places to adopt these.

Being forced to adopt tools regardless of fit to workflow (and being smart enough to understand the limitations of the tools despite management's claims) correlates very well to being negative on them.

Re: Everyone in Seattle hates AI

#648

Post by sfink »

loveparade wrote: post #608
Xeronate wrote: post #429

Earlier quoted context omitted.

Is it true that it's bad for learning new skills? My gut tells me it's useful as long as I don't use it to cheat the learning process and I mainly use it for things like follow up questions.

I think what it comes down to, and where many people get confused, is separating the technology itself from how we use it. The technology itself is incredible for learning new skills, but at the same time it incentivizes people to not learn. Just because you have an LLM doesn't mean you can skip the hard parts of doing textbook exercises and thinking hard about what you are learning. It's a bit similar to passively w…

I fully agree, but to be fair these chatbots hack our reward systems. They present a cost/benefit ratio where for much less effort than doing it ourselves we get a much better result than doing it ourselves (assuming this is a skill not yet learned). I think the analogy to calculators is a good one if you're careful with what you're considering: calculators did indeed make people worse at mental math, yet mental math can indeed be replaced with calculators for most people with no great loss. Chatbots are indeed making people worse at mental... well, everything. Thinking in general. I do not believe that thinking can be replaced with AI for most people with no great loss.

Re: Everyone in Seattle hates AI

#649

Post by musebox35 »

throwaway-0001 wrote: post #642
musebox35 wrote: post #635

Earlier quoted context omitted.

I think this is more about mechanistic understanding vs fundamental insight kind of situation. The linear algebra picture is currently very mechanistic since it only tells us what the computations are. There are research groups trying to go beyond that but the insight from these efforts are currently very limited. However, the probabilistic view is very much clearer. You can have many explorable insights, both potent…

If were true then people could predict this AI many years ago

If you dig ml/vision papers from old, you will see that formulation-wise they actually did, but they lacked the data, compute, and the mechanistic machinery provided by the transformer architecture. The wheels of progress are slow and requires many rotations to finally reach somewhere.

Re: Everyone in Seattle hates AI

#650

Post by Arainach »

dwaltrip wrote: post #153
suprjami wrote: post #104

Earlier quoted context omitted.

I don't see it that way. Tabs, spaces, curly brace placement, Vim, Emacs, VSCode, etc are largely aesthetic choices with some marginal unproven cognitive implications. I find people mostly prefer what they are used to, and if your preference was so superior then how could so many people build fantastic software using the method you don't like? AI isn't like that. AI is a bunch of people telling me this product can do…

You can vibe-code a throwaway UI for investigating some complex data in less than 30 minutes. The code quality doesn't matter, and it will make your life much easier. Rinse and repeat for many "one-off" tasks. It's not going away, you need to learn how to use it. shrugs shoulders

>You can vibe-code a throwaway UI

And then people create non-throwaway things with it and your job, performance report, bonus, and healthcare are tied to being compared to those people who just do what management says without arguing about the correct application of the tool.

If you keep your job, it's now tied to maintaining the garbage those coworkers checked in.

Post reply on HN