Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Does coding with LLMs mean more microservices?

ben.page

Post reply on HN
Print view View original

61–70 of 73 posts

Re: Does coding with LLMs mean more microservices?

#61

Post by nzach »

monsieurbanana wrote: post #51
nzach wrote: post #43

> It seems to me that existing good practices continue to work well. I haven't seen any radically new approaches to software design and development that only work with LLMs and wouldn't work without them. I've been thinking about it lately and I think you are right. LLMs haven't changed what is 'good software'. But they changed some proxies I used to have for what is 'good software'. In the past I've always loved pro…

I'm not sure llms produce good documentation. I'm open to hear more opinions on this, my feeling is that the documentation of llm-heavy projects is a bit too verbose, a bit off-target, sometimes completely irrelevant, very repetitive. Not terrible, but I'll just point my own llm to it instead of reading it myself like I would for an actual great documentation

If you are willing to point your LLM to the docs instead of actually reading it why not skip it and send your LLM directly to the source code? That is what I've been doing recently, and that is why recently good documentation became less important for me.

Re: Does coding with LLMs mean more microservices?

#62

Post by karmakaze »

I had the same thought as I'm working with LLMs. Then I reached the same conclusion as I did without LLMs: you can get most of the benefits without many of the drawbacks using well-bounded 'modules' within a monolith. The article doesn't distinguish these:

> When coding in a monolith, you have to worry about implicit coupling. The order in which you do things, or the name of a cache key, might be implicitly relied-upon by another part of the monolith. It’s a lot easier to cross boundaries and subtly entangle parts of the application. Of course, you might not do such unmaintainable things, but your coworkers might not be so pious.

What it's saying could also apply to a monorepo with distinctly deployed artifacts. The reason many don't think about clear boundaries between modules is that popular interpreted languages don't support them. Using the Java ecosystem as an example, each module can be a separate .jar containing one or more package namespaces. These must have an explicit X uses Y declaration.

The problem I see isn't so much that misuse it's easy (though that's a part of it), it's that there's to clear indication that boundaries are being crossed since calling from one package to another is normal, and the fact that some packages belong to other modules isn't always obvious.

Re: Does coding with LLMs mean more microservices?

#63

Post by lubujackson »

iainmerrick wrote: post #31

Like almost all of these articles, there's really nothing AI- or LLM-specific here at all. Modularization, microservices, monorepos etc have all been used in the past to help scale up software development for huge teams and complex systems. The only new thing is that small teams using these new tools will run into problems that previously only affected much larger teams. The cadence is faster, sometimes a lot faster,…

Maybe just a refinement of "LLM as app logic" but "LLM as emergent workflow" seems powerful.

For example, a customer service playbook may have certain ways to handle different user problems, but that breaks down as soon as there are complications or compound issues. But an LLM with the ability to address individual concerns may be able to synthesize solution given fundamental constraints. It's kind of lile building mathematics from axioms.

Re: Does coding with LLMs mean more microservices?

#64

Post by staticassertion »

onlyrealcuzzo wrote: post #59
staticassertion wrote: post #32

Earlier quoted context omitted.

> Aren't libraries just "services" without some transport layer / gateway? Libraries can share memory, mutable state, etc. Services can not. > (it inherently does not) That's going to be debatable.

> That's going to be debatable. It's really not. A service adds complexity. If you have no reason to add it besides to "reduce complexity" - that is an oxymoron. There are many concrete reasons to have one. Reducing complexity is not one. That's like arguing you can drive farther forward if you go in reverse. No. There are reasons to drive in reverse. To move forward is not one of them.

> It's really not. A service adds complexity. If you have no reason to add it besides to "reduce complexity" - that is an oxymoron.

No, it really is. I can just as easily say that a system is simpler when it's composite parts are isolated, or that a system is simpler if I can take one component and reason about it in isolation, etc.

Similarly, I could say that libraries add or reduce complexity by making similar appeals like "a single unit of code is simpler" or "separate, smaller units are simpler" etc.

Re: Does coding with LLMs mean more microservices?

#65

Post by FuckButtons »

iainmerrick wrote: post #31

Like almost all of these articles, there's really nothing AI- or LLM-specific here at all. Modularization, microservices, monorepos etc have all been used in the past to help scale up software development for huge teams and complex systems. The only new thing is that small teams using these new tools will run into problems that previously only affected much larger teams. The cadence is faster, sometimes a lot faster,…

It seems entirely logical that if an llm allows each ic to do the work of a 2-3 person team (debatable, but assume it’s true for the sake of argument) then you’ve effectively just added a layer to the org chart, meaning any tool that was effective for the next scale up of org becomes a requirement for managing a smaller team.

What should give anyone pause about this notion is that historically, by far the most effective teams have been small teams of experts focusing on their key competencies and points of comparative advantage. Large organizations tend to be slower, more bureaucratic and less effective at executing because of the added weight of communication and disconnect between execution and intent.

If you want to be effective with llms, it seems like there are a lot of lessons to learn about what makes human teams effective before we turn ourselves into an industry filled with clueless middle managers.

Re: Does coding with LLMs mean more microservices?

#66

Post by FuckButtons »

staticassertion wrote: post #32
onlyrealcuzzo wrote: post #27

Earlier quoted context omitted.

Aren't libraries just "services" without some transport layer / gateway? You should only ever have a separate "service" if there's a concrete reason to. You should never have a "service" to make things simpler (it inherently does not). Libraries on the other hand are much more subjective.

> Aren't libraries just "services" without some transport layer / gateway? Libraries can share memory, mutable state, etc. Services can not. > (it inherently does not) That's going to be debatable.

But debugging mutable state is much easier than debugging a distributed system. Even in C if some global gets mishandled I can just use: gdb, dtrace, strace or even just look at a core dump and know that whatever caused the problem will be discoverable. I have no such guarantee debugging an issue across a distributed systems service boundary.

Re: Does coding with LLMs mean more microservices?

#67

Post by iainmerrick »

FuckButtons wrote: post #65
iainmerrick wrote: post #31

Like almost all of these articles, there's really nothing AI- or LLM-specific here at all. Modularization, microservices, monorepos etc have all been used in the past to help scale up software development for huge teams and complex systems. The only new thing is that small teams using these new tools will run into problems that previously only affected much larger teams. The cadence is faster, sometimes a lot faster,…

It seems entirely logical that if an llm allows each ic to do the work of a 2-3 person team (debatable, but assume it’s true for the sake of argument) then you’ve effectively just added a layer to the org chart, meaning any tool that was effective for the next scale up of org becomes a requirement for managing a smaller team. What should give anyone pause about this notion is that historically, by far the most effect…

That's a very good point. Although I think maybe there are some crucial differences with LLMs here.

First, the extra speed makes a qualitative difference. There is some communication overhead when you're instructing an LLM rather than just doing the work directly, but the LLM is usually so fast that it doesn't necessarily slow you down.

Second, the lack of ego is a big deal. When reviewing LLM code, I have remind myself that it's okay to ask for sweeping changes, or even completely change the design because I'm not happy with how it turned out. The only cost is extra tokens -- it doesn't take much time, nobody's ego gets bruised, team morale doesn't suffer.

This might be an area where LLMs are able to follow human best practices better than humans themselves can. It's good to explore the design space with throwaway prototypes, but I think people are often too reluctant to throw code away and are tempted to try to reuse it.

Re: Does coding with LLMs mean more microservices?

#68

Post by onlyrealcuzzo »

staticassertion wrote: post #64
onlyrealcuzzo wrote: post #59

Earlier quoted context omitted.

> That's going to be debatable. It's really not. A service adds complexity. If you have no reason to add it besides to "reduce complexity" - that is an oxymoron. There are many concrete reasons to have one. Reducing complexity is not one. That's like arguing you can drive farther forward if you go in reverse. No. There are reasons to drive in reverse. To move forward is not one of them.

> It's really not. A service adds complexity. If you have no reason to add it besides to "reduce complexity" - that is an oxymoron. No, it really is. I can just as easily say that a system is simpler when it's composite parts are isolated, or that a system is simpler if I can take one component and reason about it in isolation, etc. Similarly, I could say that libraries add or reduce complexity by making similar appe…

> No, it really is. I can just as easily say that a system is simpler when it's composite parts are isolated

No, a system can not be simpler because you moved A' to B and introduced C (complexity - an added network, gateway, communication layer) for no reason besides to make the entire system simpler.

It is not simpler! You did nothing besides introduce C (complexity)!

Re: Does coding with LLMs mean more microservices?

#69

Post by staticassertion »

onlyrealcuzzo wrote: post #68
staticassertion wrote: post #64

Earlier quoted context omitted.

> It's really not. A service adds complexity. If you have no reason to add it besides to "reduce complexity" - that is an oxymoron. No, it really is. I can just as easily say that a system is simpler when it's composite parts are isolated, or that a system is simpler if I can take one component and reason about it in isolation, etc. Similarly, I could say that libraries add or reduce complexity by making similar appe…

> No, it really is. I can just as easily say that a system is simpler when it's composite parts are isolated No, a system can not be simpler because you moved A' to B and introduced C (complexity - an added network, gateway, communication layer) for no reason besides to make the entire system simpler. It is not simpler! You did nothing besides introduce C (complexity)!

Sounds like there's some debate to be had here...

Re: Does coding with LLMs mean more microservices?

#70

Post by issamG »

esafak wrote: post #49

If this is an issue you must be starting with a blank slate and not giving any direction. If you collaborate on the design you're not going to find it accidentally used microservices.

Makes me wonder if the LLM, out of the box would go one way or another. I am big fan of services, without the micro part and always guide my model in that direction.
Post reply on HN