Ask HN: If LLMs are so useful, why haven't we seen any spike in productivity?

> significantly more productive

Try a couple percent. More if you type slowly (magic autocomplete). More if you're doing something where you need to search q&a fora a lot.

Two years ago people on this forum were trying to convince me the world would be a completely different place in no time

This is one of these "in two years" technology, like self driving cars, asteroid mining, &c.

LLMs have been getting better - they were all pretty poor for my programming purposes a year or so ago, recently Perplexity (even the non-Pro version) and GPT4 have been helpful, and 4o is even better. I have been posting Leetcode hard problems into 4o and getting sensible outputs, something I didn't even try previously. Sometimes I do have to have it go through a few iterations, and I give it various qualifications (like keep to such-and-such time and space complexity or better). My usual instruction is to make the class or function more and more compact while keeping to the same functionality and time/space complexity.

I got 4o to give me a 33 line, relatively simple and understandable bidirectional BFS Kotlin function for this Leetcode problem which Perplexity (non-Pro) and GPT4 could solve, but not as well as 4o - https://leetcode.com/problems/word-ladder

Of course, even though these are Leetcode hard level problems, they are well-defined and relatively self-contained. I work at a Fortune 100 company and 99% of the time I can pound out the CRUD I do in my sleep - the difficulties I encounter are distractions, the CI server having some problem, the ticket/story I am working out not being fully specified and the PM is MIA that day, all teams are working on the feature at the same time and I need to find out what feature flags to have set and which test headers have been agreed on, the PM has asked me to work on something but some of what he says does not make sense in context so I have to ask for clarification etc. Then there's the meta-game of knowing what to prioritize, with one important component being what will make my manager happy so I get a good yearly review, and what I need to prioritize may differ from what my PM says to prioritize, or even more complexly, what my manager says to prioritize, but doesn't really mean.

Bottlenecks most often stem from lack of clarity.

The business doesn’t have clarity on what they are trying to achieve. Or they don’t have clarity on what’s important, and constantly change priority (and both of these can cause the most talented engineer to spin their wheels).

LLMs can help gain clarity, the same way a coach, consultant, or therapist can help you work through a scenario. But it’s only as effective as the work you’re willing to put into that endeavor.

So it comes down to:

* Nothing has changed regarding the nature of human work ethic

* Most people don’t want to be a programmer. The idea that ”everyone’s a programmer now” is no different than saying ”everyone’s a carpenter now” because power tools exist. Most people don’t want to do that kind of work and are happy to pay someone else to do it.

There are indeed productivity gains, but those are more scattered to quantify.

Here are some significant productivity gains I get from Mistral/Phind/ChatGPT/office-internal-llm daily.

- throw a messy shell script and ask it to refactor it(works 80% of the time)

- put a sample xml/json/yaml and ask it to generate the class/struct (code generation)

- ask questions and it gives immediate response with example more well suited to my need (previously took time to go into SO/Reddit/SE etc and scroll through several posts, docs or even waste time reading blogspams )

- ask questions about specific topic and get immediate response and citations(this is inhouse trained model) instead of fighting with broken search or ocean of messy documents in Confluence/Notion/Gitlab Pages and what not

- rubber duck when brainstorming a problem(it can sometimes lead to interesting outcomes)

- prepare a bash script to do something and then I simply modify/correct/refine it to fit my needs

- questions about trivial stuff

- generate boiler plates

- generate a throw away project to try something fast

- convert from one language to another(need to work with different teams using different languages such as TS/Java/C++/Scala/Python/Shell/Rust/Erlang etc)

- write a polite email(or response to) which I can copy paste and send when I am too occupied with something else

- documentation of specific feature of something which would take a lot of digging in the original docs

- generate a pure self-contained html/css prototype to send to our UI/UX team to give them an idea of particular concept

- summarize large block of text into bullet forms(useful for presentations)

- get summaries of popular books(because chatgpt has indeed trained on a lot of them somehow!)

- translate a text to another language(works well when it does but still needs some corrections)

Most of these activities save me a lot of time which would previously need some big time investments.