121–130 of 146 posts
Yeah I quite agree with this take. I don't understand why editors aren't utilizing language servers more for making changes. Crazy to see agents running grep and sed and awk and stuff, all of that should be provided through a very efficient cursor-based interface by the editor itself. And for most languages, they shouldn't even be operating on strings, they should be operating on token streams and ASTs
Earlier quoted context omitted.
Strings are a universal interface with no dependencies. You can do anything in any language across any number of files. Any other abstraction heavily restricts what you can accomplish. Also, LLMs aren't trained on ASTs, they're trained on strings -- just like programmers.
Exactly. LLMs are trained on huge amounts of bash scripts. They “know” how to use grep/awk/whatever. ASTs are, I assume, not really part of that training data. How would they know how to work well with on? LLMs are trained on what humans do to code. Yes, I assume down the road someone will train more efficient versions that can work more closely with the machine. But LLMs work as well as they do because they have a l…
Earlier quoted context omitted.
I agree the current way tools are used seems inefficient. However there are some very good reasons they tend to operate on code instead of syntax trees: * Way way way more code in the training set. * Code is almost always a more concise representation. There has been work in the past training graph neural networks or transformers that get AST edge information. It seems like some sort of breakthrough (and tons of $) w…
Didn't want to bury the lead, but I've done a bunch of work with this myself. It goes fine as long as you give it both the textual representation and the ability to walk along the AST. You give it the raw source code, and then also give it the ability to ask a language server to move a cursor that walks along the AST, and then every time it makes a change you update the cursor location accordingly. You basically have…
The question is not whether it can work, but whether it works better than an edit tool using textual search/replace blocks. I'm curious what you see as the advantage of this approach? One thing that comes to mind is that having a cursor provides some natural integration with LSP signature help
Yes agentic loop with diagnostic feedback is quite powerful. I'd love to have more controllable structured decode from the big llm providers to skip some sources of needing to loop - something like https://github.com/microsoft/aici
Earlier quoted context omitted.
I agree the current way tools are used seems inefficient. However there are some very good reasons they tend to operate on code instead of syntax trees: * Way way way more code in the training set. * Code is almost always a more concise representation. There has been work in the past training graph neural networks or transformers that get AST edge information. It seems like some sort of breakthrough (and tons of $) w…
> * Way way way more code in the training set. Why not convert the training code to AST?
Also, if it is only trained on code, it's likely to miss out on all the world knowledge that comes from the rest of the data.
The promise of MCP is that it “connects your models with the world”[0]. In my experience, it’s actually quite the opposite. By giving an LLM a set of tools, 30 in the Playwright case from the article, you’re essentially restricting what it can do. In this sense, MCP is more of a guardrail/sandbox for an LLM, rather than a superpower (you must choose one of these Stripe commands!). This is good for some cases, where y…
If you're running one of the popular coding agents, they can run commands in bash which is more or less access to the infinite space of tooling I myself use to do my job. I even use it to troubleshoot issues with my linux laptop that in the past I would totally have done myself, but can't be bothered. Which led to the most relatable AI moment I have encountered: "This is frustrating" - Claude Code thought, after 6 tr…
Earlier quoted context omitted.
What you're building makes a lot of sense to me. The communication indirection MCP use frequently introduces bothers me, as well as the duplication of effort when it comes to e.g. the OpenAPI spec. I'll keep an eye on this repo and plan to give it a spin sometime (though I wish there was a typescript version too).
there is a TS version actually, all the SDKs are here: https://github.com/universal-tool-calling-protocol
https://github.com/universal-tool-calling-protocolEarlier quoted context omitted.
Exactly. LLMs are trained on huge amounts of bash scripts. They “know” how to use grep/awk/whatever. ASTs are, I assume, not really part of that training data. How would they know how to work well with on? LLMs are trained on what humans do to code. Yes, I assume down the road someone will train more efficient versions that can work more closely with the machine. But LLMs work as well as they do because they have a l…
treesitter is more or less a universal AST parser you can run queries against. Writing queries against an AST that you incrementally rebuild is massively more powerful and precise in generating the correct context than manually writing infinitely many shell pipeline oneliners and correctly handling all of the edge cases.
Yeah I quite agree with this take. I don't understand why editors aren't utilizing language servers more for making changes. Crazy to see agents running grep and sed and awk and stuff, all of that should be provided through a very efficient cursor-based interface by the editor itself. And for most languages, they shouldn't even be operating on strings, they should be operating on token streams and ASTs
It's so weird that codex/claude code will manually read through sometimes dozens of files in a project because they have no easy way to ask the editor to "Find Usages". Even though efficient use of CLI tools might make the token burn not too bad, the models will still need to spent extra effort thinking about references in comments, readmes, and method overloading.
Earlier quoted context omitted.
> * Way way way more code in the training set. Why not convert the training code to AST?
You could, but it is extremely expensive to train an LLM that is competitive on coding evals. So, I was assuming use of a model someone else trained. Also, if it is only trained on code, it's likely to miss out on all the world knowledge that comes from the rest of the data.