This tracks for other areas of AI I am more familiar with. Below average people can use AI to get average results.
>Below average people can use AI to get average results.
But that would shift the average up.
AI is a floor raiser, not a ceiling raiser
81–90 of 218 posts
Re: AI is a floor raiser, not a ceiling raiser
#82Wouldn't it be both by this definition? It raises the bar for people who maybe have a lower IQ ("mastery"), but people who can us AI can then do more than ever before, raising the ceiling as well.
Re: AI is a floor raiser, not a ceiling raiser
#83There are some things that you still can't do with LLMs. For example, if you tried to learn chess by having the LLM play against you, you'd quickly find that it isn't able to track a series of moves for very long (usually 5-10 turns; the longest I've seen it last was 18) before it starts making illegal choices. It also generally accepts invalid moves from your side, so you'll never be corrected if you're wrong about…
It's super obvious even if you try and use something like agent mode for coding, it starts off well but drifts off more and more. I've even had it try and do totally irrelevant things like indent some code using various Claude models.
My favourite example is something that happens quite often even with Opus, where I ask it to change a piece of code, and it does. Then I ask it to write a test for that code, it dutifully writes one. Next, I tell it to run the test, and of course, the test fails. I ask it to fix the test, it tries, but the test fails again. We repeat this dance a couple of times, and then it seemingly forgets the original request entirely. It decides, "Oh, this test is failing because of that new code you added earlier. Let me fix that by removing the new code." Naturally, now the functionality is gone, so it confidently concludes, "Hey, since that feature isn't there anymore, let me remove the test too!"
Re: AI is a floor raiser, not a ceiling raiser
#84Wouldn't it be both by this definition? It raises the bar for people who maybe have a lower IQ ("mastery"), but people who can us AI can then do more than ever before, raising the ceiling as well.
Wouldn't "more" in this house metaphor be like expanding the floor rather than raising the ceiling?
Re: AI is a floor raiser, not a ceiling raiser
#85In things that I am comparatively good at (e.g., coding), I can see that it helps 'raise the ceiling' as a result of allowing me to complete more of the low level tasks more effectively. But it is true as well that it hasn't raised my personal bar in capability, as far as I can measure.
When it comes to things I am not good at at, it has given me the illusion of getting 'up to speed' faster. Perhaps that's a personal ceiling raise?
I think a lot of these upskilling utilities will come down to delivery format. If you use a chat that gives you answers, don't expect to get better at that topic. If you use a tool that forces you to come up with answers yourself and get personalized validation, you might find yourself leveling up.
Re: AI is a floor raiser, not a ceiling raiser
#86Since agents are good only at greenfield projects, the logical conclusion is that existing codebases have to be prepared such that new features are (opinionated) greenfield projects - let all the wiring dangle out of the wall so the intern just has to plug in the appliance. All the rest has to be done by humans, or the intern will rip open the wall to hang a picture.
Hogwash. If you can't figure out how to do something with project Y from npm try checking it out from Github with WebStorm and asking Junie how to do it -- often you get a good answer right away. If not you can ask questions that can help you understand the code base. Don't understand some data structure which is a maze of Map (s) it will scan how it is used and give you draft documentation. Sure you can't point it t…
What you describe is not using agents at all, which my comment was aimed at if you read the first sentence again.
Re: AI is a floor raiser, not a ceiling raiser
#87Earlier quoted context omitted.
I was worried about that a couple of years ago, when there was a lot of hope that deeper reasoning skills and hallucination avoidance would simply arrive as emergent properties of a large enough model. More recently, it seems like that's not the case. Larger models sometimes even hallucinate more [0]. I think the entire sector is suffering from a Dunning Kruger effect -- making an LLM is difficult , and they managed…
really? I find newer models hallucinate less, and I think they have room for improvement, with better training. I believe hallucinations are partly an artifact of imperfect model training, and thus can be ameliorated with better technique.
Yes, really!
Smaller models may hallucinate less: https://www.intel.com/content/www/us/en/developer/articles/t...
The RAG technique uses a smaller model and an external knowledge base that's queried based on the prompt. The technique allows small models to outperform far larger ones in terms of hallucinations, at the cost of performance. That is, to eliminate hallucinations, we should alter how the model works, not increase its scale: https://highlearningrate.substack.com/p/solving-hallucinatio....
Pruned models, with fewer parameters, generally have a lower hallucination risk: https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00695.... "Our analysis suggests that pruned models tend to generate summaries that have a greater lexical overlap with the source document, offering a possible explanation for the lower hallucination risk."
At the same time, all of this should be contrasted with the "Bitter Lesson" (https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...). IMO, making a larger LLMs does indeed produce a generally superior LLM. It produces more trained responses to a wider set of inputs. However, it does not change that it's an LLM, so fundamental traits of LLMs - like hallucinations - remain.
Re: AI is a floor raiser, not a ceiling raiser
#88Only for the people already affluent enough to afford the ever-more expensive subscriptions. Those most in need of a floor-raising don’t have the disposable income to take a bet on AI.
Either you are the item being sold or you are paying for the service. Nothing is free, and I for one prefer a subscription model, if only as a change from the ad model. I am sure we will see the worst of all worlds, but for now, for this moment in history, subscription is better than ads. Let’s also never have ads in GenAi tools. The kind of invasive intent level influence these things can achieve, will make our curr…
I'd never buy anything as overt as an advertisement in an AI tool. I just want to buy influence. Just coincidentally use my product as the example. Just suggest my preferred technology when asked a few % more often than my competitors. I'd never want someone to observe me pulling the strings
Re: AI is a floor raiser, not a ceiling raiser
#89The greatest use of LLMs is the ability to get accurate answers to queries in a normalized format without having to wade through UI distraction like ads and social media. It's the opposite of finding an answer on reddit, insta, tvtropes. I can't wait for the first distraction free OS that is a thinking and imagination helper and not a consumption device where I have to block urls on my router so my kids don't get suc…
"accurate"
Re: AI is a floor raiser, not a ceiling raiser
#90AI will be both a floor and a ceiling raiser, since there is a practical limit to how many domains one person or team can be expert in, and AI does/will have very strong levels of expertise/competency across a large number of domains and will thus offer significant level-ups in areas where cross-domain synthesis is crucial or where the limits of human working memory and pattern recognition make cross-domain synthesis unlikely to occur.
AI also enables much more efficient early stage idea validation, the point at which ideas/projects are the least anchored in established theory/technique. Thus AI will be a great aid in idea generation and early stage refinement, which is where most novel approaches stall or sit on a shelf as a hobby project because the progenitor doesn't have enough spare time to work through it.