I think scaffolds and the app layer are really the two big things needed for the deployment of AI in most use cases. In general, my company says for a given problem, we prefer deterministic software as the solution first, followed by LLMs, followed by humans. That's how we approach pretty much every problem. Yes, there are many things that we do with LLMs that we can eventually get to be done with software, and many things that are done by humans that we can get to be done by LLMs.
Automating AI Away
51–60 of 69 posts
Re: Automating AI Away
#52A dumber but related habit I've gotten into is that if I want to use AI to do some sort of refactoring on a C# codebase, instead of asking it to edit the code directly I ask it to write a code transformation using the Roslyn compiler API, then run that on the code. The result is less likely to have subtle bugs if it appears to work and gets through a light code review on the transformation (i.e., attempts to cheat wi…
This sounds interesting, I am really naive. I don't code in C#, is there an analogy for other programming languages, like GO, or Python or Typescript? Like are you prompting like: --- I need code that does X,Y, and Z. Write it so that the Roslyn compiler on this machine can compile and the code passes the repo's styling and formatting requirements. --- Or something else.
No, they are talking about refactoring, not adding new functionality to code.
So it would be something like:
Rewrite this Python code to use match/case instead of if/elif/else chains, write a script using the ast module to rewrite the code, do not edit it yourself, also write some tests with clear inputs and outputs I can inspect.
Or something.
Re: Automating AI Away
#53Earlier quoted context omitted.
You sound like someone thats never worked in a corporate environment. No, threatened is the correct word. I don't care if you like that or not.
It is not about me liking it or not. The word threaten comes with implications. This means they acted in hostile manner based on your words.
Directly threatening individuals and teams for petty reasons is sadly common practice in US corporate environments and among the reasons I adopted my current HN username.
Re: Automating AI Away
#54Earlier quoted context omitted.
You sound like someone thats never worked in a corporate environment. No, threatened is the correct word. I don't care if you like that or not.
It is not about me liking it or not. The word threaten comes with implications. This means they acted in hostile manner based on your words.
They usually do it by saying there's something wrong with your performance. If your technical work is unimpeachable they'll manufacture some "soft skills" issue--like saying there's something wrong with your communication. They can always find fault if they go looking for it.
Re: Automating AI Away
#55This is a very interesting introduction to a blog post, but... I'm somehow missing the actual blog post. How does this stuff work in practice? What are some concrete examples? How does one get from JavaScript tokenizing things in a commit hook to validating that the LLM didn't disable tests it didn't agree with, or any other helpful property?
You need to always be looking for what can be done deterministically and what can't. If it can, write a script or whatever is needed to make that happen. Your agent can help you figure this out. The agent becomes a glue layer for all your scripts. Use LLM judgement as an extra layer on top of a mechanical baseline.
> validating that the LLM didn't disable tests it didn't agree with
Provide a test runner and force the agent to call it. Have it emit something if you want evidence.
Re: Automating AI Away
#56Yeah, I realized this around may 2024 and started to rail models in deterministic workflows and tooling. LLMs are the new CPU.
Re: Automating AI Away
#57There's a Cambrian explosion of promising-sounding AI tools, all of which seem to work reasonably well for their authors, but it's unclear which ones to try. It seems like what we're missing are in-depth product reviews?
Re: Automating AI Away
#58A dumber but related habit I've gotten into is that if I want to use AI to do some sort of refactoring on a C# codebase, instead of asking it to edit the code directly I ask it to write a code transformation using the Roslyn compiler API, then run that on the code. The result is less likely to have subtle bugs if it appears to work and gets through a light code review on the transformation (i.e., attempts to cheat wi…
This sounds interesting, I am really naive. I don't code in C#, is there an analogy for other programming languages, like GO, or Python or Typescript? Like are you prompting like: --- I need code that does X,Y, and Z. Write it so that the Roslyn compiler on this machine can compile and the code passes the repo's styling and formatting requirements. --- Or something else.
Roslyn API basically lets you plug in to the C# compiler or use parts of it so that you can get the same view of the abstract syntax tree, typechecking, flow analysis, etc. that the compiler does. So it's a way to work with C# code at that level rather than just via textual manipulation. I'm less familiar with other languages but I think many have compilers with analogous capabilities, I know Haskell has the GHC API and I think the LLVM-based Clang (C++) and Rust compilers let you do similar things
Re: Automating AI Away
#59Thanks for posting your ad?
Re: Automating AI Away
#60A dumber but related habit I've gotten into is that if I want to use AI to do some sort of refactoring on a C# codebase, instead of asking it to edit the code directly I ask it to write a code transformation using the Roslyn compiler API, then run that on the code. The result is less likely to have subtle bugs if it appears to work and gets through a light code review on the transformation (i.e., attempts to cheat wi…
Is it open source, can you share? Have been pondering using source generators to achieve a similar effect and have the LLM build the source generator ‘framework’ that code would derive from rather than just generating the code itself, you create the code generation framework.