Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

github.com

Post reply on HN
Print view View original

51–60 of 72 posts

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#51

Post by rachelradulo »

jbstack wrote: post #50
rachelradulo wrote: post #27

Earlier quoted context omitted.

Fair question. The core will always stay open source and free. We’ll monetize around it with things like managed hosting, enterprise support, and compliance options (HIPAA, SOC2, etc). Basically, we make money when teams want someone to stand behind it in production, not for using the software itself. But let us know if you have other ideas! We're still new to open source

So to clarify, does this mean you don't plan to go down the route of having a "community edition" vs a "enterprise edition" with missing features in the former?

Correct for now. We've got a ways to go to hammer out the real details of enterprise. We wouldn't want a world where not having the enterprise-y add-ons would hinder the core value prop.

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#52

Post by mhamann »

swyx wrote: post #47

yeah guys look i wish you well and respect for launching and all but this is just not going to ever be a venture scale startup and you should calibrate your expectations. you could be wasting the best years of your life being a wrapper of a wrapper of a wrapper and competing on developer experience in open source for no money, or you could be building agents. build agents. please.

Appreciate the pep talk, but let’s not pretend infra and developer experience plays can’t scale. GitLab, HashiCorp, and Vercel all "just built better DX for open source" and somehow ended up billion-dollar companies.

Agents will come and go (and probably run into the same orchestration headaches), but someone still has to build the reliable, open foundation they’ll stand on.

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#53

Post by mhamann »

swyx wrote: post #48
swyx wrote: post #47

yeah guys look i wish you well and respect for launching and all but this is just not going to ever be a venture scale startup and you should calibrate your expectations. you could be wasting the best years of your life being a wrapper of a wrapper of a wrapper and competing on developer experience in open source for no money, or you could be building agents. build agents. please.

more context - https://x.com/swyx/status/1904256213661192405 i really mean it in as non mean spirited a way as possible

I do really appreciate you taking the time to drop by and leave a comment. But I'm curious...why do you think building agents is so important vs. building more of the "AI infrastructure" (which is really what LlamaFarm is trying to do) that will enable the devs that are building integrated AI systems (including agents).

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#54

Post by zackangelo »

Just a bit of feedback:

> Instead of one brittle giant, we orchestrate a Mixture of Experts…

“mixture of experts” is a specific term of art that describes an architectural detail of a type of transformer model. It’s definitely not using smaller specialized models for individual tasks. Experts in an MoE model are actually routed to on a per token basis, not on a per task or per generation basis.

I know it’s tempting to co-opt this term because it would fit nicely for what you’re trying to do but it just adds confusion.

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#55

Post by rgthelen »

zackangelo wrote: post #54

Just a bit of feedback: > Instead of one brittle giant, we orchestrate a Mixture of Experts… “mixture of experts” is a specific term of art that describes an architectural detail of a type of transformer model. It’s definitely not using smaller specialized models for individual tasks. Experts in an MoE model are actually routed to on a per token basis, not on a per task or per generation basis. I know it’s tempting t…

I hear you and valid. A mixture of Models is probably a better phrase - we are constantly moving between AI experts and very skilled Developers who use OpenAI endpoints and call it AI, so we are constantly working on finding the correct language. This was a miss though - will do better :)

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#56

Post by swyx »

jbstack wrote: post #49
swyx wrote: post #47

yeah guys look i wish you well and respect for launching and all but this is just not going to ever be a venture scale startup and you should calibrate your expectations. you could be wasting the best years of your life being a wrapper of a wrapper of a wrapper and competing on developer experience in open source for no money, or you could be building agents. build agents. please.

This whole comment is predicated on the idea that something is only worthwhile if it will lead to a "venture scale startup" and that to do anything else is to "waste the best years of your life". I, for one, am glad that not everyone shares your ethos. Conversely, I'm also glad that there are people out there "building agents". Diversity is a good thing. Encouraging everyone to only do one thing is a bad thing.

bros identify as YC W22, this is a reasonable expectation

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#57

Post by rgthelen »

swyx wrote: post #47

yeah guys look i wish you well and respect for launching and all but this is just not going to ever be a venture scale startup and you should calibrate your expectations. you could be wasting the best years of your life being a wrapper of a wrapper of a wrapper and competing on developer experience in open source for no money, or you could be building agents. build agents. please.

Our business model is not to compete in open-source; we are just providing this to the community since its the right thing to do and as a signal to enterprises to reduce risk.

Our goal is to target large enterprises with services and support, leveraging deep channel partnerships. We do ship agents as a part of the project and it will be a critical part of the services model in the future.

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#58

Post by swyx »

mhamann wrote: post #53
swyx wrote: post #48

Earlier quoted context omitted.

more context - https://x.com/swyx/status/1904256213661192405 i really mean it in as non mean spirited a way as possible

I do really appreciate you taking the time to drop by and leave a comment. But I'm curious...why do you think building agents is so important vs. building more of the "AI infrastructure" (which is really what LlamaFarm is trying to do) that will enable the devs that are building integrated AI systems (including agents).

both are important, but one will make more money and have higher changes of survival. incidentally i have found that agent companies have better infra than the horizontal ai infra as a service companies anyway, and you can intuit why that is. so: you will struggle to get the good customers because the good customers are increasingly finding it affordable to build rather than buy.

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#59

Post by gus_22 »

Would like to connect. I've got a YC colleague I have asked to setup an intro. Agree with John's point below about the chicken and egg scenario. Our community flagged your post. Fundamental problems like who cares, who pays reinforce why this hasn't already been done (well and at scale). And why this time might be the right time!

Re: Launch HN: LlamaFarm (YC W22) – Open-source framework for distributed AI

#60

Post by johnthecto »

rgthelen wrote: post #21
johnthecto wrote: post #17

So this sounds like an application layer approach, maybe just shy of a replit or base44, with the twist that you can own the pipeline. While there's something to that, I think there are some further questions around differentiation that need to be answered. I think the biggest challenge is going to be the beachead: what client demographic has the cash to want to own the pipeline and not use SaaS, but doesn't have the…

Yeah, that’s a fair framing — it is kind of an “application layer” for AI orchestration, but focused on ownership and portability instead of just convenience. Yeah, the beachead will be our biggest issue - where to find first hard-core users. I was thinking legal (they have a need for AI, but data cannot leave their servers), healthcare (same as legal, but more regualtions), and government (not right now, but normall…

Like your ideas around legal and Healthcare. Both sectors where you've got interest and money. Another way might be to look at partnering with a service org who does transformation/modernization as a way to accelerate delivery. Maybe MSPs? They're always trying to figure out how to lean out.

An idea might be to try and get a vertical sooner rather than later. The only thing better than an interested lawyer would be a selection of curated templates and prompts designed by people in the industry for example. So you get orchestration and industry-specific aligned verts. Much easier sell than a general purpose platform. But then you're fighting with the other vertically integrated offerings.

Maybe there are other differentiators? If this is like bedrock for your network, maybe the angle is private models where you want them. Others are doing that though, so there's pretty active competition there as well.

The more technical and general the audience the more you're going to have to talk them out of just rolling openwebui themselves.

Post reply on HN