181–190 of 215 posts
Earlier quoted context omitted.
TCP/IP is a great example though of something you can get seemingly correct and then be subject to all kinds of failure modes in edge cases you didn’t handle correctly (fragmentation, silly windows, options changing header sizes, etc).
Thanks for telling me how wrong I am! I bet you're fun at parties.
Today, I needed to write a proxy[0] that wraps an object and log all method calls recursively. I asked claude to write the initial version. It came up with a complicated class based solution. I spent more than 30 minutes getting a good abstract to come out. I was copy pasting typescript errors and applying fixes it suggested without thinking much. In the end, I gave up and wrote what I wanted myself in 5 minutes. 0]…
Would you have written it in five minutes has you not just spent 30 minutes ruling out wrong solutions?
Earlier quoted context omitted.
I find that LLMs are almost comically bad at projects that have a hardware component like RaspberryPi or Pico, or Ardunio. I think that its because often the libraries you use are niche or have a a few similar versions, the LLM really commonly hallucinated solutions and would continually suggest that library X did have that capability. I think because often in hardware projects you often hit a point where you can't d…
Actually, it's because many of the people writing tutorials and sharing answers about that stuff don't know what the hell they're doing or grasp the fundamentals of how those systems work and so most of the source material the LLM's are trained on is absolute garbage. Public Arduino, RPi, Pico communities are basically peak cargo cult, with the blind leading the blind through things they don't understand. The noise i…
I'm not sure they will. There's a kind of evaporative cooling effect where once you get to a certain level of understanding you switch around your tools enough that there's not much point interacting with the community anymore.
Counterexample: Ive been able to complete more side projects in the last month leveraging llms than i have ever in my life. One of which I believe to have potential as a viable product, and another which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch. I think the key to being successful here is to realize that you're still at the wheel as an engineer. Th…
Earlier quoted context omitted.
That's almost in the training data: https://www.quora.com/How-many-Humans-can-we-fit-on-the-Moon
I guess coming up with a truly original question is tougher that it seems. Any ideas?
Edit: couldn't resist, and dammit!!
Response: Ah, I see what you're doing! Since the Moon has no atmosphere, there’s technically no air to create any kind of airspeed velocity. So, the answer is... zero miles per hour. Unless, of course, you're asking about the speed of the horse itself! In that case, we’d just have to know how fast the astronaut can gallop without any atmosphere to slow them down.
But really, it’s all about the fun of imagining a moon-riding astronaut, isn’t it?
Earlier quoted context omitted.
My college professor has certifications and has passed tests that weren't in their training data. My college professor was also willing to say "I don't know, ask me next class"
What do you consider 'not in its training data'? I just asked Claude a question I am pretty sure was not in its training data. * https://i.imgur.com/XjvImeT.jpeg
This is a perfect example of where not knowing the “domain” leads you astray. As far as I know “newborn width” is not something typically measured, so Claude is pulling something out of thin air.
Indeed you are showing that something not in the training data leads to failure.
> AI isn’t a co-pilot; it’s a junior dev faking competence. Trust it at your own risk. This is a good take that tracks with my (heavy) usage of LLMs for coding. Leveraging productive-but-often-misguided junior devs is a skill every dev should actively cultivate!
> Leveraging productive-but-often-misguided junior devs is a skill every dev should actively cultivate! Feels like this is only worthwhile because the junior dev learns from the experience; an investment that yields benefits all around, in the broad sense. Nobody wants a junior around that refuses to learn in perpetuity, serving only as a drag on productivity and eventually your sanity.
Earlier quoted context omitted.
I guess coming up with a truly original question is tougher that it seems. Any ideas?
Ask them what's the airspeed velocity of a laden astronaut riding a horse on the moon... Edit: couldn't resist, and dammit!! Response: Ah, I see what you're doing! Since the Moon has no atmosphere, there’s technically no air to create any kind of airspeed velocity. So, the answer is... zero miles per hour. Unless, of course, you're asking about the speed of the horse itself! In that case, we’d just have to know how f…
Earlier quoted context omitted.
What do you consider 'not in its training data'? I just asked Claude a question I am pretty sure was not in its training data. * https://i.imgur.com/XjvImeT.jpeg
It is immediately wrong in Step 1. A newborn is not a 2:1 ratio of height:width. Certainly not 25cm width (what does that even mean? Shoulder to shoulder?). This is a perfect example of where not knowing the “domain” leads you astray. As far as I know “newborn width” is not something typically measured, so Claude is pulling something out of thin air. Indeed you are showing that something not in the training data lead…
Edit: it also doesn't account for the fact the moon is more or less a sphere, and not a flat plane.
Earlier quoted context omitted.
> Claude is like having my own college professor. I don't use Claude, so maybe there's a huge gap in reliability between it and ChatGPT 4o. But with that disclaimer out of the way, I'm always fairly confused when people report experiences like these—IME, LLMs fall over miserably at even very simple pure math questions. Grammatical breakdowns of sentences (for a major language like Japanese) are also very hit-or-miss.…
> LLMs fall over miserably at even very simple pure math questions They are language models, not calculators or logic languages like Prolog or proof languages like Coq. If you go in with that understanding, it makes a lot more sense as to their capabilities. I would understand the parent poster to mean that they are able to ask and rapidly synthesize information from what the LLM tells them, as a first start rather t…