Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nexus: An Open-Source AI Router for Governance, Control and Observability

nexusrouter.com

Post reply on HN
Print view View original

11–20 of 27 posts

Re: Nexus: An Open-Source AI Router for Governance, Control and Observability

#12

Post by tomhoule »

vid wrote: post #5
CptanPanic wrote: post #3

Earlier quoted context omitted.

Sounds like litellm which I use, I wonder how it compares?

There is also https://github.com/maximhq/bifrost which apparently overcomes some performance issues of litellm and is easy to get going.

Yeah they definitely belong in the same space. Nexus is an LLM Gateway, but early on, the focus has been on MCP: aggregation, authentication, and a smart approach to tool selection. There is that paper, and a lot of anecdotal evidence, pointing to LLMs not coping well with a selection of tools that is too large: https://arxiv.org/html/2411.09613v1

So Nexus takes a tool search based approach to solving that, among other cool things.

Disclaimer: I don't work on Nexus directly, but I do work at Grafbase.

Re: Nexus: An Open-Source AI Router for Governance, Control and Observability

#13

Post by SparkyMcUnicorn »

fbjork wrote: post #8
CptanPanic wrote: post #3

Earlier quoted context omitted.

Sounds like litellm which I use, I wonder how it compares?

Founder of Grafbase here. Here are a few key differentiators vs LiteLLM today: - Nexus does MCP server aggregation and LLM routing - LiteLLM only does LLM routing - The Nexus router is a standalone binary that can run with minimal TOML configuration and optionally Redis - LiteLLM is a whole package with dashboard, database etc. - Nexus is written in Rust - LiteLLM is written in Python That said, LiteLLM is an impress…

What's the difference between "MCP Server Aggregation" and the litellm_proxy endpoint described here?

https://docs.litellm.ai/docs/mcp

Re: Nexus: An Open-Source AI Router for Governance, Control and Observability

#16

Post by tomhoule »

SparkyMcUnicorn wrote: post #13
fbjork wrote: post #8

Earlier quoted context omitted.

Founder of Grafbase here. Here are a few key differentiators vs LiteLLM today: - Nexus does MCP server aggregation and LLM routing - LiteLLM only does LLM routing - The Nexus router is a standalone binary that can run with minimal TOML configuration and optionally Redis - LiteLLM is a whole package with dashboard, database etc. - Nexus is written in Rust - LiteLLM is written in Python That said, LiteLLM is an impress…

What's the difference between "MCP Server Aggregation" and the litellm_proxy endpoint described here? https://docs.litellm.ai/docs/mcp

The main difference is that while you can get Nexus to list all tools, by default the LLM accesses tools by semantic search — Nexus returns only the relevant tools for the what the LLM is trying to accomplish. Also, Nexus speaks MCP to the LLM, it doesn't translate like litellm_proxy seems to do (I wasn't familiar with it previously).

Re: Nexus: An Open-Source AI Router for Governance, Control and Observability

#17

Post by fbjork »

makita34 wrote: post #15

Seems quite similar to the commercial nexos.ai platform, which also focuses on routing, governance, and observability for AI workloads, but as a proprietary solution rather than open source

From what I can tell they don’t offer a self-hosted router?

Re: Nexus: An Open-Source AI Router for Governance, Control and Observability

#19

Post by barbazoo »

fbjork wrote: post #18
owenthejumper wrote: post #14

Another proxy?

MCP aggregation is one of the big differentiators

I'm curious, what issue does that solve? I'm only working on agents that make tool calls via HTTP in a home baked way but I can't imagine how resolving the tools from 2 MCP servers is harder than 1.
Post reply on HN