Live data from Hacker News

AI and the Automation of Work

ben-evans.com

251–260 of 268 posts

Re: AI and the Automation of Work

#251
post #76

Earlier quoted context omitted.

> What's the gap between a Co-pilot written test and a human-written test? IMHO the gap is that human-written test understands the (unwritten) specification (which might be as simple as "program doesn't crash"), and tests the program against it. I am of the (pretty radical, it seems) opinion that the (especially unit) tests that just take the code as the specification, instead of having specification coming from else…

Do you know of any good resources to read about testing? How did you learn what you know?

I only know very little. But we had a great guy in our company who was a big proponent of rigorous methods even for non-critical software and he made a very interesting training about it. Here's the list of his recommended resources:

Increasing QA maturity level:

– B.Beizer – „Black-box testing“ ($), „SW testing techniques“ ($)

– ISTQB Foundation level

– RTCA DO-178C ($) / EUROCAE ED-12C ($), DO-248C

– Joint Software Systems Safety Engineering Handbook

Test analysis specification-based techniques (EP, BVA, EP DI, STT, TCl):

– ISTQB „Advanced Test Analyst“, „Advanced Technical Test Analyst“ courses and Syllabi

– BS 7925-2 „Standard for SW Component testing”, ISO/IEC/IEEE 29119-4:2015 „Test techniques” ($)

– IEEE 829:2008 „Standard for SW testing documentation” ($)

Test analysis failure/risk-based techniques (FMEA, FTA, RCA):

– C.Wilhelmsen, L.T.Ostrom – „Risk Assessment: Tools, Techniques, and Their Applications” ($)

– C.S.Carlson – „Effective FMEAs” ($)

– MIL-STD-1629A „Procedures for performing FMEA”

– NUREG-0492 „Fault Tree Handbook”

– NASA Fault Tree Handbook with Aerospace Applications

– DOE-HDBK-1208-2012 Volume 1 „Accident and Operational Safety Analysis Techniques”

– TOR-2014-02202 „RCA Best Practices Guide” ($)

Re: AI and the Automation of Work

#252
post #81

Earlier quoted context omitted.

I can imagine a "box", say 1x1x2 meters, that you get in your home, it will have a plexiglass on it, and inside, it will have several AI-controlled arms, and you will be able to command it to fix anything in that space. Not really sure about cooking, but you could ask for example to fix your trousers, or repair a notebook, or make a fitting for water tap, and so on, and it would autonomously do it (it would ask to or…

Things are so disposable now that I don't think most households would have a use case for a Repair-Bot; I can see it in offices and factories, OTOH. I think we're pretty far away from a robot that can take an arbitrary problem and fix it accurately. I'll be excited when I can take my car to the mechanic and he's supervising a robot changing my oil.

There is no use case for 8-bit home computers or home 3D printers either.

Re: AI and the Automation of Work

#253
post #248

Earlier quoted context omitted.

I'm not confused I'm saying you're conflating two things - understanding something and being able to perform it in a single pass. Neither llms nor humans have to be able to solve a sudoku puzzle to write code that can. I can be awful at mental arithmetic and write code for a calculator.

Ok, let me know if you can get an LLM to solve sudoku puzzles by prompting it with information on backtracking and recursion.

No prompting on backtracking or recursion, that was chosen by the LLM. All I did was tell it to break the problem down and reason about it, then afterwards asked it to write the code. Examples were added by me at the end, from a random sudoku site for "easy" and "hard".

https://gist.github.com/IanCal/9817f8b21b2ea6d77940966ee399d...

Re: AI and the Automation of Work

#254
post #76

Earlier quoted context omitted.

What's the gap between a Co-pilot written test and a human-written test? Are tests written by co-pilot better than no tests? Developers generally hate writing tests, so I suspect more and more unit tests will be generated. I've worked with someone who does this and it seems insane to me, although I never got to see the generated tests they were referring to, so I can't give specifics.

> What's the gap between a Co-pilot written test and a human-written test? IMHO the gap is that human-written test understands the (unwritten) specification (which might be as simple as "program doesn't crash"), and tests the program against it. I am of the (pretty radical, it seems) opinion that the (especially unit) tests that just take the code as the specification, instead of having specification coming from else…

Completely agree with this approach - recently I find that most of the tests I write are so-called 'integration' tests. I find these tests not only get more code coverage for fewer tests, but they more closely represent the real or implied external specification. In other words, they do a better job of verifying 'is the code correct' rather than 'is the code we wrote the code we wrote'.

I think the specification from elsewhere is all about the business value that the code delivers. At the end of the day, the business doesn't care about how exactly your code is decomposed into classes/functions/etc, the business cares about 'can I register a customer' and 'are customers prevented from writing to resources they only have read access to'. Of course there is some requirement for the code not to be a complete mess, as this has impacts delivering features and reducing bugs down the line, but with tests that sit closer to the value the code delivers than the structure of the code, you're free to refactor and the tests will tell you what you've inevitably broken. I find this considerably more valuable than unit tests.

Re: AI and the Automation of Work

#255
post #50

It's not ChatGPT what we should be putting in our forecasts, but its successors. And, from what I have been reading, our AI folk are coming with better performers all the time by chaining multiple neural networks, "making modular brains", so to speak. Any argument that starts by saying "ChatGPT is very bad at X, humans will always do X," is flawed IMO. So I think that any job that can be done by a non-human is up for…

Robocop...

https://www.youtube.com/watch?v=REC-ZYWL1to

Re: AI and the Automation of Work

#256

Earlier quoted context omitted.

"I think AI will probably eliminate almost all future knowledge worker jobs through simple attrition. In 20 years, perhaps 1 human knowledge worker will be doing the work of 100 (measured by todays productivity) augmented by integrated AI in the tools they use." Lump of labour fallacy?

Yea, I don't think this will happen over the longterm. I do see worker dislocation in certain sectors though, can't see how LLM's won't substantially shake-up office work especially as Microsoft is already busy building them into the Office Suite.

>I do see worker dislocation in certain sectors…

Which was my point. Knowledge workers (and the types of people who gravitate to knowledge work careers) will be dislocated and what work remains may not be as rewarding both financially and personally to these kind of people. People of course adapt, but would that person sitting on the autism spectrum who can thrive and make exceptional income as a SWE be able to adapt to a less cerebral career? Will there be mentally rewarding work for those displaced? Those are real concerns and a possible butterfly effect of AI.

My hope for my grandchildren is to pursue careers that can weather the automation and AI storm. Becoming a plumber might be literally shit work, but it’s one trade that is AI and automation proof for many years to come.

Re: AI and the Automation of Work

#257
post #246

Earlier quoted context omitted.

Is this GPT-2 generated? I've tired reading it three times and I still can't make heads or tails of it.

I see what you did there gpt-"2". at least you don't dislike enough to downvote. not sure what's not understandable. I think parent is overly hyperbolic. i get that chat GPT is unprecedented, but come on "impossible to conceive of near-term reality" ??? edit: oh! you're the parent. yeah i think you're wildly hyperbolic. There's more drastic historical precedents but of course the future is by definition going to be i…

That was not meant as an insult or downvote, I literally do not understand your comment.

>Imagine yourself in the movie Apocalypto and I'm using a Hollywood film on purpose here - the entire Universe of being as you see it all the sudden explodes its Universe with new Gods/demons different and more powerful _everything_ coming to kill you. And then there's the boring discovery of "we're just one planet in one solar system in one Galaxy...

This is gibberish word salad.

Re: AI and the Automation of Work

#258
post #253

Earlier quoted context omitted.

Ok, let me know if you can get an LLM to solve sudoku puzzles by prompting it with information on backtracking and recursion.

No prompting on backtracking or recursion, that was chosen by the LLM. All I did was tell it to break the problem down and reason about it, then afterwards asked it to write the code. Examples were added by me at the end, from a random sudoku site for "easy" and "hard". https://gist.github.com/IanCal/9817f8b21b2ea6d77940966ee399d...

You misunderstood what I meant. Prompt the LLM with the sudoku puzzle and ask it to solve it and not write python. Human can do it, LLM can not.

Re: AI and the Automation of Work

#259
post #249

Earlier quoted context omitted.

What exactly do you want me to state up front? A general intelligence can solve general problems when those problems are formally specified. Sudoku is a good example of a formally specified problem that can be solved by most people but not by LLMs. This is because LLMs and all neural networks are simply DAGs of function which do not support recursion or backtracking. Adding recursion to neural networks has been tried…

I don't understand your question. > What exactly do you want me to state up front? Exactly what was in your comment that I told you to state up front. Most people in a discussion about AI and replacing people are working with a definition of general intelligence that includes humans to a large degree. > This is because LLMs and all neural networks are simply DAGs of function which do not support recursion or backtrac…

Ok great, next time I run into you I'll make sure to remind you that large language models are not intelligent as I conceive of the word "intelligent".

Re: AI and the Automation of Work

#260
post #119

Earlier quoted context omitted.

It's also very... American to not think mention the absurdity of machine guns at airports, even in the hands of humans. Even if lethal force is required, would you want a security person (not a trained policeman) to "start blasting" and spraying bullets with a machine-gun?

> It's also very... American to not think mention the absurdity of machine guns at airports Now you mention it, despite the two respective national stereotypes, the only airports where I can remember seeing machine guns on the security team were one of the London airports. If I saw machine guns in SFO or JFK, I was too tired from the flight to remember.

Airport police were definitely carrying assault rifles when I was last in South Korea (2014?).

Don't remember whether it was Incheon or Gimpo.

Post reply on HN