Build full “product skills” and you'll probably be fine • Hacker News Board

Re: Build full “product skills” and you'll probably be fine

#741

Post by zizee » Sun, Mar 19, 2023, 9:04 PM UTC

wizofaus wrote: post #244

TheLoafOfBread wrote: post #199

Earlier quoted context omitted.

I am actually finding amusing that managers will generate 100k lines project with AI and then will start figuring out that it does not work as they want to. Then they figured out actual developers are needed to fix it, either in a very strict way telling AI what should happen (i.e. higher level programming) or directly fixing code generated by AI.

You're assuming that the AI is even generating anything that will make sense to a human. It seems inevitable we'll reach the point that for SaaS the AI will do everything directly based some internal model it has of what it believes the requirements are (e.g. it will be capable of acting just like a live web server), whereas for desktop and mobile apps, while that paradigm still remains relevant, it will generate the…

> what it believes the requirements are

It will have to describe these requirements in a way that a human can understand, and verify. The language will have to be unambiguous and structured. A human will need to be able to read that language, build up a mental model, and understand it is correct, or know the way to make corrections. Who do you think that person will be? Hint: it will be a specialist that knows how to think in a structured, logical way.

Re: Build full “product skills” and you'll probably be fine

#742

Post by nextaccountic » Sun, Mar 19, 2023, 9:08 PM UTC

noodlesUK wrote: post #615

coffeebeqn wrote: post #474

Earlier quoted context omitted.

I’ve used no code tools since the 90s and it just has a fatal flaw. For simple demo use cases it looks simple and cool. Then when you go to the real world and start getting pivots and edge cases you have to fix in the interface then it becomes a 4D nightmare and essentially a very bad programming language

I’ve spent a fair bit of time working on interactive chat systems that use a form of visual programming. It’s not good. Once you get past the toy stage (which is good and ergonomic), it’s just the same as programming except the tooling is far worse, you have to invent all your change management stuff from scratch, and it’s like going back 30 years.

What about coding in two languages, one textual and one visual?

Or a single language that has both visual and textual components

Or a single language where each component can be viewed in textual or visual form (and edited in the form that makes most sense)

Re: Build full “product skills” and you'll probably be fine

#743

Post by [deleted] » Sun, Mar 19, 2023, 9:15 PM UTC

kaashif wrote: post #685

antupis wrote: post #553

Earlier quoted context omitted.

if GPT or copilot make developer 5-50x more productive then you need just 2 good programmers instead of team of 10-100 person, so out of window goes all formalized agile processes, safes etc so you don't need anymore those managers who are doing communicating, process planning and other big business stuff. Of course this will also affect for developers but I bet less because there is huge need for software developers…

If C makes developers 5-50x more effective compared to writing assembly, then you just need two devs instead of a team of 10-100. Obviously that didn't happen. And people actually made predictions like that once upon a time. As software becomes cheaper, and the production of software can be done with fewer people, demand has always increased and I don't see any reason that's going to stop. Reminds me of this: https:/…

[deleted]

Re: Build full “product skills” and you'll probably be fine

#744

Post by 908B64B197 » Sun, Mar 19, 2023, 9:15 PM UTC

ChrisMarshallNY wrote: post #259

carlmr wrote: post #29

Looking back we had one CS professor who in 2007 predicted we'd all be jobless in ten years, i.e. 2017. His prediction was based on the trends he was seeing at the time. But it wasn't even AI. Instead he made this prediction because he saw the rise of no-code tools replacing software developers because managers could finally cut out the pesky "translators", i.e. software developers. I said it then and I will say it n…

> So exact in fact that they will need to specify a program in a click and drag interface, or in human language. This. I started programming in Machine Code, where the "editor" was a pad of graph paper. I've watched management- and business-focused people sneer at us geeks for my entire career, and watched them drool over the thought of finally getting rid of us. Hasn't happened yet. > I'd be surprised if the next st…

> I've watched management- and business-focused people sneer at us geeks for my entire career, and watched them drool over the thought of finally getting rid of us. [...] They have nothing but contempt for us "Make It Happen" folks. They treat us as if we grow on trees, and are suckers, to boot.

I've met my fair share as well. Jobs copycats too!

The trick is always to ask them how much they can raise, and do so assuming they raised successfully in the past and that they know VCs.

Either the conversation gets really interesting, or they clam up real quick!

Re: Build full “product skills” and you'll probably be fine

#745

Post by yread » Sun, Mar 19, 2023, 9:16 PM UTC

RivieraKid wrote: post #304

yread wrote: post #291

I work a lot with doctors where there also worries about being replaced with AI. The pioneers there say that won't happen but doctors who use AI will replace the ones that don't. Same thing in our field. And just like in our field there will always be niches where AI output isn't good enough and there isn't enough money to improve it so human specialists will own it.

Doctors? They're safe. They interact with patients, use physical tooling, demand for their work is growing because of demographics. And you really don't want a doctor who sometimes hallucinates advice.

Sorry, should have been more specific. I work with pathologists. They never ever talk to anyone much less a patient :-)

Re: Build full “product skills” and you'll probably be fine

#746

Post by RivieraKid » Sun, Mar 19, 2023, 9:26 PM UTC

mdorazio wrote: post #737

RivieraKid wrote: post #691

Earlier quoted context omitted.

1) Economic theory says that marginal product of labor (value) is equal to wages, at least in simple models. 2) With GPT-4 you still need to know how to program. A product manager can't replace you.

1) The real world is not a simple economic model. The wage rate is roughly equivalent to the rate it costs to replace an employee, not their marginal value. If your argument was true, company profits would tend toward zero as wages rise. 2) I specifically did not say GPT-4. If you think v4 is the peak of what will be possible when looking at how far we have come in just 2 years then I don't know what to tell you. Als…

1) I think it's approximately true in the real world. If you can hire an employee who adds substantially more revenue than what their wage is, you keep hiring employees until value created by an employee is close to their wage.

When discussing economy, it's good to start with understanding the situation through the lens of economic models and than look to what extent are the conclusions applicable to the real world.

Using your argumentation - above you said that xyz is true in a free market, but real-world markets are not free...

2) And I specifically said that I'm assuming no dramatic improvement beyond GPT-4. The 2 misconception I supposedly have... I didn't even make that claim.

Re: Build full “product skills” and you'll probably be fine

#747

Post by josephg » Sun, Mar 19, 2023, 9:56 PM UTC

6510 wrote: post #581

ghaff wrote: post #550

Earlier quoted context omitted.

Good managers shield their reports from a lot of the crap that is inevitable at scale in a large organization. A large company can't just have individuals and small self-managed teams go off and do whatever they feel like. And, yes, if you just cut out all the intermediate layers, the VP with 150 direct reports will simply have to resort to automation based on simplistic metrics. "You didn't produce enough lines of c…

> A large company can't just have individuals and small self-managed teams go off and do whatever they feel like. Right but how much of that is because humans don't scale? With the AI, if it can do a thing well enough it can do it 100 000 times per day.

I think its simply because upper management doesn't trust the people they've hired.

In my mind, having individuals and small self managed teams go off and do whatever they feel drawn to do is exactly how a good company is run. - So long as those teams and individuals talk to (& seek advice) from the rest of the company when their work has impacts outside the team. The book Reinventing Organizations by Laloux talks about this a lot, and how it works in some companies today.

The reason it doesn't happen more is that upper management doesn't trust their employees and they don't feel in control when people just do things.

Re: Build full “product skills” and you'll probably be fine

#748

Post by thomastjeffery » Sun, Mar 19, 2023, 10:03 PM UTC

soraki_soladead wrote: post #689

thomastjeffery wrote: post #652

Real Artificial Intelligence? Yeah, that would definitely factor out a lot of the wasted work we call "engineering". Language Learning Models like GPT? Not even close. We should absolutely stop calling those "AI". They are not intelligent. They model intelligent behavior: human writing. We should probably even stop calling them "Language Learning". They don't know or care what language is: they learn whatever pattern…

> They are not intelligent. Citation needed. Numerous actual citations have demonstrated hallmarks of intelligence for years. Tool use. Comprehension and generalization of grammars. World modeling with spatial reasoning through language. Many of these are readily testable in GPT. Many people have… and I dare say that LLMs reading comprehension, problem solving and reasoning skills do surpass that of many actual human…

> hallmarks of intelligence

All through the lens of personification. It's important to take a step back and ask, "Where do these hallmarks come from?"

The hallmarks of intelligence are literally what is encoded into text. The reason LLMs are so impressive is that they manage to follow those patterns without any explicit direction.

> I dare say that LLMs reading comprehension, problem solving and reasoning skills do surpass that of many actual humans.

People tend to over-optimize reading comprehension by replacing what they are reading with what they predict to be reading. Every person has a worldview built out of prior knowledge that they use to disambiguate language. It takes effort to suspend one's worldview, and it takes effort to write accurate unambiguous language.

An LLM cannot have that problem, because an LLM cannot read. An LLM models text. The most dominant patterns of text are language: either the model aligns with those patterns, or we humans call the result a failure and redirect our efforts.

> Anthropomorphizing LLMs is indeed an issue but is separate from a debate on their intelligence.

How could that even be possible? The very word, "intelligence" is an anthropomorphization. Ignoring that reality moves the argument into pointless territory. If you try to argue that an anthropomorphized LLM is intelligent, then the answer is, "No shit, Sherlock. People are intelligent!" That doesn't answer any questions about a real LLM.

> as these models sprint past goal posts.

Either an LLM succeeds at a goal, or it fails. It has no idea what the difference is. The LLM has no concept of success: no category for failure. An LLM has no goals or intentions, and doesn't make a single logical decision.

So what is its success coming from? The text being modeled. Without humans authoring that text, there is no model at all!

The goals are authored, too. Every subject, every decision, every behavior, and every goal is determined by a human. Without human interaction, the LLM is nothing. Does nothing think? Does an arrow find its target? Of course not.

Re: Build full “product skills” and you'll probably be fine

#749

Post by bitwize » Sun, Mar 19, 2023, 10:05 PM UTC

carlmr wrote: post #29

Looking back we had one CS professor who in 2007 predicted we'd all be jobless in ten years, i.e. 2017. His prediction was based on the trends he was seeing at the time. But it wasn't even AI. Instead he made this prediction because he saw the rise of no-code tools replacing software developers because managers could finally cut out the pesky "translators", i.e. software developers. I said it then and I will say it n…

This is pretty much my standard rant on no-code tools: that out of necessity, the constructs they provide are isomorphic to constructs in some programming language. I even had a similar professor, who assured me that as a programmer my career was at a dead end because "there are tools that can take a UML spec and generate a whole application".

Nevertheless, AI is different. Because now you don't have to specify things at a programming-language level of granularity. It will select the programming language constructs most statistically likely to match a solution to the specified problem.

So now we're in a whole 'nother world where the programming, as such, will be done by machine while the role of designer or architect must still be done by humans. But the legions of programmers employed by companies are about to go the way of farriers.

Re: Build full “product skills” and you'll probably be fine

#750

Post by jiggawatts » Sun, Mar 19, 2023, 10:05 PM UTC

lelandbatey wrote: post #735

wizofaus wrote: post #705

Earlier quoted context omitted.

But why would anyone bother with using AI to generate human readable code if the AI can generate the final desired behavior directly, either on-the-fly or as executable machine code?

Because the AI's, at least right now, can't generate/change code so that it correctly does what's expected with the confidence intervals we expect. I've tried to get it to happen, and it just doesn't. As long as that's true, we'll need to somehow get the correctness to where it needs to be, and that's going to require a person.

A lot of people have already figured out at some tricks to improving code generation.

You can fairly easily update the “next token” choice with a syntax check filter. LLMs like ChatGPT provide a selection of “likely” options, not a single perfect choice. Simply filter the top-n recommendations mechanically for validity. This will improve output a lot.

Similarly, backtracking can be used to fix larger semantic errors.

Last but not least, any scenario where a test case is available can be utilised to automatically iterate the LLM over the same problem until it gets it right. For example, feed it compiler error messages until it fixes the remaining errors.

This will guarantee output that compiles, but it may still be the wrong solution.

As the LLMs get smarter they will do better. Also, they can be fine tuned for specific problems automatically because the labels are available! We can easily determine if a piece of code compiles, or if it makes a unit test pass.