Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

github.com

Post reply on HN
Print view View original

51–60 of 159 posts

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#51

Post by blainm »

brap wrote: post #39

I don’t really understand agents. I just don’t get why we need to pretend we have multiple personalities, especially when they’re all using the same model. Can anyone please give me a usecase, that couldn’t be solved with a single API call to a modern LLM (capable of multi-step planning/reasoning) and a proper prompt? Or is this really just about building the prompt, and giving the LLM closer guidance by splitting in…

One of the key limitations of even state-of-the-art LLMs is that their coherence and usefulness tend to degrade as the context window grows. When tackling complex workflows, such as customer support automation or code review pipelines - breaking the process into smaller, well-defined tasks allows the model to operate with more relevant and focused context at each step, improving reliability.

Additionally, in self-hosted environments, using an agent-based approach can be more cost-effective. Simpler or less computationally intensive tasks can be offloaded to smaller models, which not only reduces costs but also improves response times.

That being said, this approach is most effective when dealing with structured workflows that can be logically decomposed. In more open-ended tasks, such as "build me an app," the results can be inconsistent unless the task is well-scoped or has extensive precedent (e.g., generating a simple Pong clone). In such cases, additional oversight and iterative refinement are often necessary.

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#52

Post by weego »

brap wrote: post #39

I don’t really understand agents. I just don’t get why we need to pretend we have multiple personalities, especially when they’re all using the same model. Can anyone please give me a usecase, that couldn’t be solved with a single API call to a modern LLM (capable of multi-step planning/reasoning) and a proper prompt? Or is this really just about building the prompt, and giving the LLM closer guidance by splitting in…

I don't get it either. Watching implementations on YouTube etc it primarily it feels like a load of verbiage trying to carve out a sub-industry, but the meat on the bone just seems to be defining discreet units of AI actions that can be chained into workflows that interact with non-ai services.

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#53

Post by bravura »

brap wrote: post #39

I don’t really understand agents. I just don’t get why we need to pretend we have multiple personalities, especially when they’re all using the same model. Can anyone please give me a usecase, that couldn’t be solved with a single API call to a modern LLM (capable of multi-step planning/reasoning) and a proper prompt? Or is this really just about building the prompt, and giving the LLM closer guidance by splitting in…

https://aider.chat/2024/09/26/architect.html

"Aider now has experimental support for using two models to complete each coding task:

An Architect model is asked to describe how to solve the coding problem.

An Editor model is given the Architect’s solution and asked to produce specific code editing instructions to apply those changes to existing source files.

Splitting up “code reasoning” and “code editing” in this manner has produced SOTA results on aider’s code editing benchmark. Using o1-preview as the Architect with either DeepSeek or o1-mini as the Editor produced the SOTA score of 85%. Using the Architect/Editor approach also significantly improved the benchmark scores of many models, compared to their previous “solo” baseline scores (striped bars)."

In particular, recent discord chat suggests that o3m is the most effective architect and Claude Sonnet is the most effective code editor.

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#54

Post by andrewmutz »

brap wrote: post #39

I don’t really understand agents. I just don’t get why we need to pretend we have multiple personalities, especially when they’re all using the same model. Can anyone please give me a usecase, that couldn’t be solved with a single API call to a modern LLM (capable of multi-step planning/reasoning) and a proper prompt? Or is this really just about building the prompt, and giving the LLM closer guidance by splitting in…

Modularity. We could put all code in a single function, it is possible, but we prefer to organize it differently to make it easier to develop and reason about. Agents are similar

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#55

Post by joshstrange »

tybaa wrote: post #44
joshstrange wrote: post #41

Earlier quoted context omitted.

Very cool. Like I said I can make it work with Stdio but I have a SSE MCP proxy I wrote to combine multiple MCP servers (just to make plugging in all my tools to a new client easier to test). That said, I think after looking at the docs that I'll be tempted to move my tools in directly but I probably will keep them behind MCP for portability.

Oh nice, did you write your own proxy or are you using something like https://www.npmjs.com/package/mcp-proxy ?

I have used `mcp-proxy` but (afaik) you can only use it 1-to-1 and I wanted an N-to-1 proxy so that instead of configuring all my MCP servers in the multiple clients I've tested out I could just add 1 server and pull in everything.

I found `mcp-proxy-server` [0] which seemed like it would do what I want but I ran into multiple problems. I added some minor debug logging to it and the ball sort of rolled downhill from there. Now it's more my code than what was there originally but I have tool proxying working for multiple clients (respecting sessionIds, etc) and I think I've solved most all the issues I've run into and added features like optional tool prefixing so there isn't overlap between MCP servers.

Given what I know now, I don't think N-to-1 is quite as useful as I thought. Or rather, it really depends on your "client". If you can toggle on/off tools in your client then it's not a big problem but sometimes you don't want "all" the tools and if you client only allows toggling per MCP server then you will have an issue.

I love the ideas of workflows and how you have defined agents. I think my current issue is almost too many tools and the LLM sometimes gets confused over which ones to use. I'm especially thrilled with your HTTP endpoints you expose for the agents. My main MCP server (my custom tools I wrote, vs the third-party ones) exposes an HTTP GUI for calling the tools (faster iteration vs trying it through LLMs) and I've been using that and 3rd-party chat clients (LibreChat and OpenWebUI) as my "LLM testing" platform (because I wasn't aware of a better options) but neither of those tools let you "re-expose" the agents via an API.

All in all I'm coming to the conclusion that 90% of MCP servers out there are really cool for seeing what's possible but it's probably best to write your own tools/MCP since most all MCP servers are just thin wrappers around an API. Also it's so easy to create an MCP server that they are popping up all over the place and often of low quality (don't fully implement the API, take shortcuts for the authors use-case, etc). Using LLMs to writing the "glue" code from API->Tool is fairly minor and I think is worth "owning". To sum that all up: I think my usage of 3rd party MCP servers is going to trend towards 0 as I "assimilate" MCP servers into my own codebase for more control but I really like MCP as a way to vend tools to various different LLM clients/tools.

[0] https://github.com/adamwattis/mcp-proxy-server

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#56

Post by nilslice »

joshstrange wrote: post #34

This looks awesome! Quick question, are there plans to support SSE MCP servers? I see Stdio [0] are supported and I can always run a proxy but SSE would be awesome. [0] https://mastra.ai/docs/reference/tools/client

we have a tutorial that covers this!

https://docs.mcp.run/tutorials/mcpx-mastra-ts

you don't even need to use SSE, as mcp.run brings the tools directly to your agent, in-process, as secure wasm modules.

mcp.run does have SSE support for all its servlet tools in the registry though too.

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#57

Post by calcsam »

fuddle wrote: post #40

"You may not provide the software to third parties as a hosted or managed service" - The Elastic v2 license isn't actually open source like your title mentions: "Open-source JS agent framework" https://github.com/mastra-ai/mastra/blob/main/LICENSE

I mentioned that in the comment. We’re using Elastic v2 for now because we want users to be able to do anything with us, but protect from eg AWS

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#58

Post by coffeemug »

brap wrote: post #39

I don’t really understand agents. I just don’t get why we need to pretend we have multiple personalities, especially when they’re all using the same model. Can anyone please give me a usecase, that couldn’t be solved with a single API call to a modern LLM (capable of multi-step planning/reasoning) and a proper prompt? Or is this really just about building the prompt, and giving the LLM closer guidance by splitting in…

If you ignore the word "agent" and autocomplete it in your mind to "step", things will make more sense.

Here is an example-- I highlight physical books as I read them with a red pen. Sometimes my highlights are underlines, sometimes I bracket relevant text. I also write some comments in the margins.

I want to photograph relevant pages and get the highlights and my comments into plain text. If I send an image of a highlighted/commented page to ChatGPT and ask to get everything into plain text, it doesn't work. It's just not smart enough to do it in one prompt. So, you have to do it in steps. First you ask for the comments. Then for underlined highlights. Then for bracketed highlights. Then you merge the output. Empirically, this produces much better results. (This is a really simple example; but imagine you add summarization or something, then the steps feed into each other)

As these things get complicated, you start bumping into repeated problems (like understanding what's happening between each step, tweaking prompts, etc.) Having a library with some nice tooling can help with those. It's not especially magical and nothing you couldn't do yourself. But you also could write Datadog or Splunk yourself. It's just convenient not to.

The internet decided to call these types of programs agents, which confuses engineers like you (and me) who tend to think concretely. But if you get past that word, and maybe write an example app or something, I promise these things will make sense.

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#59

Post by jacobr1 »

brap wrote: post #39

I don’t really understand agents. I just don’t get why we need to pretend we have multiple personalities, especially when they’re all using the same model. Can anyone please give me a usecase, that couldn’t be solved with a single API call to a modern LLM (capable of multi-step planning/reasoning) and a proper prompt? Or is this really just about building the prompt, and giving the LLM closer guidance by splitting in…

One way to think about it is job orchestration. You end up with some kind of DAG of work to execute. If all the work you are doing is based on context from the initiation of the workflow, then theoretically you could do everything in a single prompt. But more interesting is when there is some kind of real-world interaction, potentially multiple. Such as a websearch, or executing code, calling an API. Then you take action based on the result of then. Which in turn might trigger another decision to take some other action, iteratively, and potentially branching.

Re: Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby

#60

Post by DSchau »

mplewis wrote: post #45
user9999999999 wrote: post #38

Earlier quoted context omitted.

gatsby was one of the first static react frameworks, now you have things like nextjs remix astro etc... i dont think abandonware is fair, thats just the way software goes

The Gatsby team made a lot of promises upon which they didn't follow through. Not a great way to build confidence in your next big project.

… such as?
Post reply on HN