Live data from Hacker News

Show HN: Airbyte Agents – context for agents across multiple data sources

news.ycombinator.com

41–50 of 63 posts

Re: Show HN: Airbyte Agents – context for agents across multiple data sources

#41
We built something similar an A/B testing framework that measures Unblocked's impact on real AI coding agents.

It spawns agent CLIs (Claude Code, Codex, Cursor, GitHub Copilot) with and without Unblocked's MCP server attached, then statistically compares the results: https://github.com/unblocked/unblocked-harness-compare

We likewise measured token savings, (wall clock) time, # tool calls, and # turns.

Re: Show HN: Airbyte Agents – context for agents across multiple data sources

#43
post #8

(former employee here) congrats Michel! so glad to see you guys adapting to the AI age so well (and using the crap out of Devin!) hmm so airbyte agents could serve as a form of MCP gateway, or a key building block of an MCP gateway, which btw is how anthropic uses mcp themselves for all their internal apps https://www.youtube.com/watch?v=CD6R4Wf3jnY&t=1s&pp=0gcJCd4K... i think my most sad/interesting observation abou…

I think this is right ( a big gap ) but I don’t think data companies even now what the right shape is for AI.

It’s definitely not old school ETL + dbt + BI tool, it might be something like this, but it’s very early

Re: Show HN: Airbyte Agents – context for agents across multiple data sources

#44
post #32

Doesn't Skills solve all of this? OpenClaw, Hermes and other agents have already made skill adoption mainstream? Are you guys still seeing a future where people are dumping entire MCP tool defs into context?

Great question, @Tsarp - Skill and tools work great together. What we've found is that agents generally need both to achieve great results. We're actually not trying to replace skills, but to give them new super powers. Are there any examples you've run into where skills were missing tools (or data) that they needed for a specific task?

Hmm, hoping this isn't a generic LLM generated response.

Skills have the scripts folder and you can precisely describe when and when not to use a script. This can end up directly wrapping API(s), CLIs, generic scripts or even other MCP servers.

CC and codex both have the skill creator and you can have them build the skill for you.

Havent run into any scenarios where skills were missing tools. 1-2 iterations and its usually taken care off quite quickly.

Re: Show HN: Airbyte Agents – context for agents across multiple data sources

#45
post #19

Your billing support email forwards to a google group which rejects the email entirely. So i embedded my question inside the websites sales enquiry form and received multiple rounds of emails that couldn’t be further from human. It’s not why we started using posthog but it definitely sealed the deal when you see how simple and reliable that experience is

Sorry for that experience. We had a bad billing support routing issue and it’s since been fixed. Thank you for calling it out. We'll aim to do better!

Re: Show HN: Airbyte Agents – context for agents across multiple data sources

#46
post #44

Earlier quoted context omitted.

Great question, @Tsarp - Skill and tools work great together. What we've found is that agents generally need both to achieve great results. We're actually not trying to replace skills, but to give them new super powers. Are there any examples you've run into where skills were missing tools (or data) that they needed for a specific task?

Hmm, hoping this isn't a generic LLM generated response. Skills have the scripts folder and you can precisely describe when and when not to use a script. This can end up directly wrapping API(s), CLIs, generic scripts or even other MCP servers. CC and codex both have the skill creator and you can have them build the skill for you. Havent run into any scenarios where skills were missing tools. 1-2 iterations and its u…

Hey, fair enough. (100% human here, btw.) I think I misread your original question to be asking "why do we need a service (whether accessed via API/SDK/MCP/etc.)" vs just having skills (markdown + scripts)".

If you are already leveraging skills as scripts and APIs in your skills, then you understand the distinction. I'll attempt to re-answer your question with now hopefully a better understanding:

I think Airbyte Agents helps your agent by giving access to data across any and all of the systems it may need to get data from, or write data to. While you could hit the service APIs directly (via REST/CLI/etc.), in practice we find that not all use cases are amenable to this. Airbyte Agents does have REST APIs as well as SDKs and of course the MCP interface - so it's not really about MCP tools specifically, more about how you can access the data. The Airbyte Agents interface also reduces the number of creds that the agent needs to handle, giving a single portal (with logging and audit capabilities) for all the actions your agent is taking.

Sorry for the red herring of skills-v-tools. Let me know if you have any additional questions!

Re: Show HN: Airbyte Agents – context for agents across multiple data sources

#47
post #37

The prompts you mentioned here sound like SQL. Is there any way to run actual SQL on these systems? Is "agents need to poke around endlessly" a symptom of the fact that there isn't a way to run an actual query? (I'd guess there is actually SQL at the bottom layer, but there's no way to talk to it?)

Helpful feedback, thank you! And your instincts are spot on. As of now, we have API based search, with filter predicates and field selection in JSON. While we haven't published anything on the backend implementation, I can say it does use a cloud-native storage medium where the filters are indeed pushed down as SQL. We want to be careful about if/when we offer direct SQL access, specifically because SQL dialects can differ drastically and we wouldn't want to break consumers if/when we change which dialect(s) are supported.

That said, please stay tuned - and thank you again for this valuable feedback.

Re: Show HN: Airbyte Agents – context for agents across multiple data sources

#48
post #12

sounds very familiar to what I ended up doing on my internal system - especially anything to do with search - much better to just sync everything to a DB and give the agent access to the DB

That's great to hear - great minds think alike! > give the agent access to the DB This is where Airbyte really can shine, I think, and the total can be more the sum of the parts. Because Airbyte excels at data replication already, we can populate your the Agent Context Store without users or agents ever needing to think about the words "ELT" or "ETL". We're listening carefully to feedback so we hope you will give it…

yeah this is one of the few AI-related products that I have seen that make sense to me

but i also wonder to what extent this needs to be its own thing or if this is just something that it looks like we need but really people just need to shovel more stuff into their data warehouse / data lake that you never had reason to before, because now that's all fodder for agentic search

Re: Show HN: Airbyte Agents – context for agents across multiple data sources

#49
post #42

Congrats, you built an ETL pipeline and called it an agent. The industry has come full circle.

Haha indeed!

On a more serious note, just as swyx mentioned in a comment further up, we do believe a lot of the challenges of reliably operationalising agents boil down to data. All of which is non-obvious to AI engineers (besides Frontier Labs gathering/generating data for model training).

What the right shape is - we are all figuring it out. Happy to trade notes.

Re: Show HN: Airbyte Agents – context for agents across multiple data sources

#50
post #27
post #8

(former employee here) congrats Michel! so glad to see you guys adapting to the AI age so well (and using the crap out of Devin!) hmm so airbyte agents could serve as a form of MCP gateway, or a key building block of an MCP gateway, which btw is how anthropic uses mcp themselves for all their internal apps https://www.youtube.com/watch?v=CD6R4Wf3jnY&t=1s&pp=0gcJCd4K... i think my most sad/interesting observation abou…

Thanks swyx! We'd love to do that session "data engineering for ai engineers", will make you an intro to the right person in the team.

saw your email, will get back!
Post reply on HN