My amazon orgs leadership has been obsessed with spec driven development while individual engineers tell me the only use they have is to placate leadership. I'm tired
--dangerously-skip-reading-code
81–90 of 219 posts
Re: --dangerously-skip-reading-code
#82> We can stop reading LLM-generated code just like we don’t read assembly, or bytecode, or transpiled JavaScript; our high-level language source would now be another form of machine code This is too weird for me. At least with programming languages I can consult the documentation and if the programming language isn’t behaving as documented, it’s obviously a defect and if you’re savvy enough you often have open channe…
If you run a local LLM and an open source agent harness you are pretty close to that.
can you explain how? with a compiler you can rely on the adage "it's never a compiler bug" (until it is! and then you can fix it)
how can a local LLM with an open source agent harness provide the same trustworthiness?
Re: --dangerously-skip-reading-code
#83Earlier quoted context omitted.
If you run a local LLM and an open source agent harness you are pretty close to that.
can you explain how? with a compiler you can rely on the adage "it's never a compiler bug" (until it is! and then you can fix it) how can a local LLM with an open source agent harness provide the same trustworthiness?
> ... then you can fix it
I recall working on a project that used (MSVC) VC++ and a coworker found a bug in the compiler. We reported the issue to Microsoft and they eventually patched it.
You may find yourself arguing explicitly for open source dev tools if you continue down this line. There are many commercial cases where "you can fix it" does not apply to the dev toolchain and you will find yourself reliant on a provider. At that point, the trustworthiness of "compiler provider" and "local LLM provider" is the pertinent discussion (e.g. provider vs. provider instead of LLM vs compiler).
Re: --dangerously-skip-reading-code
#84The lesson I've learned from our new AI age is how little a large number of people who've worked in software development their entire careers understand software development. I suppose all the money floating around AI helps dummify everything, as people glom on to narratives, regardless of merit, that might position them to partake. What we actually have now is the ability to bang out decent quality code really fast…
All the talking points and techniques are those which were used when pushing outsourcing: give better specs, write detailed tests, accept bad code because it works so who cares, we can just rewrite from scratch later, and my favorite "they will get better with more exposure to your code base". None of these takes is wrong, but what they neglect is doing all that work is way more effort than if I wrote the original code myself.
Using an LLM to one shot a small function (something i would do with a very specific search on Google or SO) is handy. Giving it a harness and free access to a code base leads to some terrible code, and doubling down with more instructions and agents in the loop means more time writing the rube Goldberg orchestration rather than just opening up an editor and writing code.
Re: --dangerously-skip-reading-code
#85My amazon orgs leadership has been obsessed with spec driven development while individual engineers tell me the only use they have is to placate leadership. I'm tired
How does spec driven development differ from test driven development?
Re: --dangerously-skip-reading-code
#86Earlier quoted context omitted.
Hate it all you want, but XML is genuinely a good fit there, and Claude is apparently insanely good at working with XML prompts.
Is Claude good at working with XML prompts, or is XML good at convincing users to write more Claude-able specs? I am intensely skeptical that you could write an XML document describing a nontrivial web application in full detail, but I could easily imagine someone who thinks they have to stripping out important details because they don't really map to XML.
They train it with XML even the system prompts that Claude reads are formatted by it.
Re: --dangerously-skip-reading-code
#87[flagged]
Re: --dangerously-skip-reading-code
#88Earlier quoted context omitted.
Hate it all you want, but XML is genuinely a good fit there, and Claude is apparently insanely good at working with XML prompts.
I don’t know why, but I get this feeling whenever someone uses “insanely” or “shockingly” along with AI, I think they’re bot or are writing based on a guideline! No offense, btw, I’m not saying you’re a bot.
You’re absolutely right!
Kidding, nah no worries. I do worry people become overly paranoid of bots as time passes.
Re: --dangerously-skip-reading-code
#89Earlier quoted context omitted.
which sounds like it came from React's "dangerouslySetInnerHTML", per the comment you replied to.
I think people used similar prefixes for a long time. For example, Haskell has had `unsafePerformIO` since the 90's... and MSFT's Hungarian notation was also similar, though it used abbreviations for things like "unsafe" (not "dangerous"). Perhaps React was the most famous case of using "dangerously" though.
"unsafe" seems quite different from the "dangerously [...]" phrasal template. I don't think it's a stretch to suppose it was inspired by React. Still waiting for this one to catch on:
React.__SECRET_INTERNALS_DO_NOT_USE_OR_YOU_WILL_BE_FIRED
Re: --dangerously-skip-reading-code
#90> my first bet would be specifications and tests You are missing another dimension how easy it would be to migrate if adding new feature hits a ceiling and LLM keeps breaking the system. Imagine all tests are passing and code is confirming the spec, but everything is denormalized because LLM thought this was a nice idea at the beginning since no one mentioned that requirement in the spec. After a while you want to ad…
> Don't forget that very very detailed spec is actually the code
Came here to say this, but you said it for me. If the problem were merely one of insufficient rigour or detail in specs, it would have been solved long before LLMs.