What about a library that doesn’t have any documentation or tutorials, like the Soundfont lib for JUCE where the LLM was essentially I had to get a working product?
> What about a library that doesn’t have any documentation or tutorials This always sucks, but since it is something you have to deal with, the answer is "read the code." Hopefully the code has some test cases somewhere, and that is the best place to start. But if not, you unfortunately got to profile and trace the code yourself. It takes awhile, but you usually come out with a pretty good understanding (after A LOT…
Yeah dude, I’ve got a wife, two young children and a full-time job in legal services. These are the side projects that LLMs give me the time and cognitive energies to work on.
Congratulations on having so much free time!
I do lot of data analysis so I am pretty familiar with awk, et. al. But you know, thanks for the condescending lecture, I guess?
Yes, but do search engines not have a bias towards existing technologies?
Yes but they are much more up-to-date. And they don't have the react issue he mentioned.
I think it may hold for mediocre programmers who brainlessly do what AI tells them. Good programmers will always look for better ways, and for them AI is a great way to explore time-tested ideas in addition to a fast tab-completer. It will certainly not drive them away from new ideas (or at least the arguments in this piece didn't convince me).
If your abstractions match common design patterns then you've solved your problem. It's ridiculous to assume that an LLM will understand 1k LOC of standard library code better than 10 lines of a custom abstraction which uses a common design pattern. It's more prone to hallucinating things if your custom abstraction is not super standard but at least you'd be able to check its mistakes (you're checking the code genera…
LLMs are not human, they see the whole context window at once. On the contrary it’s ridiculous to assume otherwise. I’ll reiterate what I said before: put the whole source of the new library in the context window and tell the LLM to use it. It will, at least if it’s Claude.
I have experienced quite a few of mistakes by claude as documentation grows larger (and not necessarily too large compared to certain standards). Eg some time ago, I fed a whole js documentation for some sensors into the context window and asked to generate code. The documentation mentioned specifically that it does not fully support ES6, and also explicitly that it does not support const. Claude did not bother and used const. And many times I have experienced that Claude makes mistakes using syntax in a (much less common than js or python) language that would make sense in some other language maybe, but not that one. I have inserted instructions not to do the specific mistakes in system prompts, told it to make sure it is valid syntax for X language, but Claude once in a while keeps doing the same mistakes. Negative prompts are hard, especially when probably going against a huge bunch of the training set.
I feel myself taking AI’s base knowledge of a tech stack into account when I work. Otherwise it feels like I am swimming upstream. I can’t be the only one, and this article resonated for me.
Sure, but what I'm saying is that's not where the knowledge bottleneck is. The knowledge bottleneck is in the human population , which is then reflected in Stack Overflow and blogs, which is then reflected in LLM's. LLM's aren't doing anything special to stifle new tech adoption. New tech is harder to adopt because it's new tech, because people in general are less familiar with it . (And there's a little bit of a tra…
Yes, but I think it's also true that LLMs are acting as a force multiplier to the underlying effect you describe.
Something like (my apologies to maths for the abuse) LLMs are adding one to the exponent of the adoption feedback cycle.
The actual truth is that you don’t need any of these frameworks at all. Vanilla JS & HTML has come a long way, and LLMs like Claude can build Vanilla JS with no trouble.
So AI does stifle adoption of new tech then?
Both things can be true. I have also been frustrated trying to get LLMs to work with SwiftUI, given that it’s slightly newer, changing frequently, and has lower adoption relative to something like React.
Honestly this is a short term problem. Eventually the models will have larger context windows and will be able to easily browse brand new documentation or example code.
> What about a library that doesn’t have any documentation or tutorials This always sucks, but since it is something you have to deal with, the answer is "read the code." Hopefully the code has some test cases somewhere, and that is the best place to start. But if not, you unfortunately got to profile and trace the code yourself. It takes awhile, but you usually come out with a pretty good understanding (after A LOT…
Yeah dude, I’ve got a wife, two young children and a full-time job in legal services. These are the side projects that LLMs give me the time and cognitive energies to work on. Congratulations on having so much free time! I do lot of data analysis so I am pretty familiar with awk, et. al. But you know, thanks for the condescending lecture, I guess?
I didn't realize it was a rhetorical question. It was not intended to be condescending. There's a lot of people who do not have that skill
Strong disagree here. I've been trying to learn Zig, and I'm thwarted enough by chatgpt giving me outdated information on Zig's unstable API that if I didn't have a strong incentive to learn it for it's cross compiler, I'd likely turn my efforts towards another language. This effect can greatly alter the adoption curve of a new tech, which can leave it dying on the vine. You're not wrong though, in that Stack Overflo…
Having spent the past week deep in cursor, it's amazing for building out a basic web app.. but getting it to a point of productivity takes a while.. Command line install for latest svelte.. nope npx install is now deprecated, have to do it another way.. ok, let's go old school and read the docs. Great, it's up and running, but nope, Svelte has just hit V5 and the LLM isn't aware of the changes.. ok, do I drop back to…
BTW You can extend cursor out with new knowledge and the svelte community has a config file you can throw at cursor that'll add svelte 5 support.
It seems self-evident it will, and it's largely self-reinforcing. Less documentation/examples of new tech -> New model doesn't have enough info on new tech to be useful -> Less uptake of new technology -> Less documentation/examples to build a corpus.... I do wonder if this problem could get solved by basically providing documentation explicitly written for LLMs to consume and produce more detailed "synthetic" docume…
Since the current chatbots have the ability to tap into Google Search, it's not unlikely they could gather their own up to date documentation on-the-fly. This would create a slew of new attack vectors where malicious actors will try to add backdoors into the documentation, which the LLM would reproduce. A seasoned software engineer will easily pick it up, but the large amount of folks that are just copy pasting chatb…
With people using LLMs instead of googling, there isn't an incentive for people to write tutorial blog posts.
Strong disagree here. I've been trying to learn Zig, and I'm thwarted enough by chatgpt giving me outdated information on Zig's unstable API that if I didn't have a strong incentive to learn it for it's cross compiler, I'd likely turn my efforts towards another language. This effect can greatly alter the adoption curve of a new tech, which can leave it dying on the vine. You're not wrong though, in that Stack Overflo…
Having spent the past week deep in cursor, it's amazing for building out a basic web app.. but getting it to a point of productivity takes a while.. Command line install for latest svelte.. nope npx install is now deprecated, have to do it another way.. ok, let's go old school and read the docs. Great, it's up and running, but nope, Svelte has just hit V5 and the LLM isn't aware of the changes.. ok, do I drop back to…
The deepseek advancements will allow more frequent retraining incorporating updates, it used to likewise take months for the Google index to update: you could tell because they listed how many sites were indexed on the homepage.
Strong disagree here. I've been trying to learn Zig, and I'm thwarted enough by chatgpt giving me outdated information on Zig's unstable API that if I didn't have a strong incentive to learn it for it's cross compiler, I'd likely turn my efforts towards another language. This effect can greatly alter the adoption curve of a new tech, which can leave it dying on the vine. You're not wrong though, in that Stack Overflo…
Have you tried Cursor instead of a chatbot? I don’t trust it to do much coding for me but it has this feature where you give it the URL of the documentation, and it indexes it for you. Then you can reference that documentation in the chat with the @ symbol and Cursor applies RAG to the indexed content to include relevant documentation. I usually index autogenerated API docs and any website docs the library has like g…