Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

news.ycombinator.com

Post reply on HN
Print view View original

161–170 of 206 posts

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#161

Post by dhorthy »

dylan604 wrote: post #141

Isn't this precisely how AI started? It was a bunch of humans under the hood doing the logic when the companies said it was AI. Then we removed the humans and the quality took a hit. To fix that hit, 3rd party companies are putting humans back in the loop? Isn't that kind of like putting a band-aid on the spot where your arm was just blown off?

yeah it's an interesting point. I can only guess that we didn't do a good enough job of learning from the humans while they were doing their jobs...seems like traditional ML or even LLM tech might be good enough that we can take another pass? Overall the thesis of humanlayer is that you should do all this super gradually, move the needle from 1% AI to 99%+, and have strong SLOs/SLAs around when you pause that needle moving because quality took a hit.

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#162

Post by dhorthy »

ejp wrote: post #80

This is exciting. I am an architect in a startup that has long valued bringing humans in the loop for the moments when only humans can do the work. The key thing missing between the potential seen in the last couple years of LLM-based fervor and realizing actual value for us has been the notion of control and oversight. So instead, we have built workflows and manual processes in a custom way throughout the business.…

thanks - fixed! emailed you separately

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#163

Post by soheil »

dhorthy wrote: post #2

P.S. nobody asked but since you made it this far - the next big problem in this space is fast becoming, what else do we need to be able to build these "headless" or "outer loop" AI agents? Most frameworks do a bad job of handling any tool call that would be asynchronous or long running (imagine an agent calling a tool and having to hang for hours or days while waiting for a response from a human). Rewiring existing f…

Just to frame the problem slightly differently, if you had unlimited number of humans who could perform tasks as quickly as a computer this wouldn't be a problem need solving. So since we know that's the end state for any human-in-the-loop system then maybe it's worth solving that problem instead.

A few things come to mind, divide the problem into chunks that can be solved in parallel by many people. Crowd source your platform so there are always people available with a very high SLA, just like cloud servers are today.

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#164

Post by a1j9o94 »

dhorthy wrote: post #146
potatoman22 wrote: post #134

Earlier quoted context omitted.

You could implement some blocking HitL service/tool as an MCP server.

ah okay - I guess in that case, I would like chain a HitL step as an MCP server that wraps/chains to another tool that depends on approval? or is there a cleaner way to do that?

Yeah, exactly. You would define a HitL server and the actions it implements would be API calls to your system.

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#165

Post by mschulkind »

dylan604 wrote: post #141

Isn't this precisely how AI started? It was a bunch of humans under the hood doing the logic when the companies said it was AI. Then we removed the humans and the quality took a hit. To fix that hit, 3rd party companies are putting humans back in the loop? Isn't that kind of like putting a band-aid on the spot where your arm was just blown off?

No, not really.

If you have an AI that can answer 90% of queries correctly AND now this is the key, it knows which 90% it can answer correctly, human in the loop can be incredibly valuable to answer that other 10%.

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#166

Post by dhorthy »

mschulkind wrote: post #165
dylan604 wrote: post #141

Isn't this precisely how AI started? It was a bunch of humans under the hood doing the logic when the companies said it was AI. Then we removed the humans and the quality took a hit. To fix that hit, 3rd party companies are putting humans back in the loop? Isn't that kind of like putting a band-aid on the spot where your arm was just blown off?

No, not really. If you have an AI that can answer 90% of queries correctly AND now this is the key, it knows which 90% it can answer correctly, human in the loop can be incredibly valuable to answer that other 10%.

hah yeah I don't know how soon we will be on great accuracy for the latter, for things like "send an email", people tend to just block everything for approval, because clicking approve 90 times hand editing 10 times is a lot better than copying 90 things from one app to another and then 10 things copy, hand edit, send

although I do have some ideas on how you could use vector similarity against past executions to get a 1-100 score on how likely a given action is to be approved rejected. You could set a dial to "anything below 60 just auto-reject it and provide the past feedback to the model preemptively". This would need a lot of experimentation, might even be a research angle (if it hasn't been tried already)

(thinking like cosine * {1 if approved, -1 if rejected} and normalize the score 1-100. You could maybe even weight rejection in 0 to -1 based on sentiment)

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#167

Post by dhorthy »

ehsanu1 wrote: post #157
dhorthy wrote: post #6

Earlier quoted context omitted.

ah very cool! are there any things you wish it did or any friction points? What are the things that "just work"?

Essentially, you don't need to think about time and space. You just write more or less normal looking code, using the Temporal SDK. Except it actually can resume from arbitrarily long pauses, waiting as long as it needs to for some signal, without any special effort beyond using the SDK. You also automatically get great observability into all running workflows, seeing inputs and outputs at each step, etc. The cost of…

helpful - thanks! I have played with temporal a bit but have this thought that since most AI tools represent state as just a rolling context window, maybe you don't have to serialize and entire call stack and you can cut a bunch of corners.

but we're all probably better off not investing that wheel

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#168

Post by dhorthy »

a1j9o94 wrote: post #164
dhorthy wrote: post #146

Earlier quoted context omitted.

ah okay - I guess in that case, I would like chain a HitL step as an MCP server that wraps/chains to another tool that depends on approval? or is there a cleaner way to do that?

Yeah, exactly. You would define a HitL server and the actions it implements would be API calls to your system.

this is interesting. I will have to think more about how humanlayer can support an MCP integration/wrapper, it's not immediately obvious to me

i do think that AI-calling-tools is insufficient to provide bidirectional communication rails for user input/review though...not disagreeing just maybe thinking out loud a little here

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#169

Post by dhorthy »

TZubiri wrote: post #155
dhorthy wrote: post #130

Earlier quoted context omitted.

i think the slack side is easy. I think an AI-optimized email communication channel is a long ways off. I spent weeks throwing things at my monitor figuring out reliable ways to wire DNS+SES+SNS+Lambda+Webhooks+API+Datastore+Async Workers so that everything would "just work" with a few lines of humanlayer sdk. And what we build still only serves a small subset of use cases (e.g. to support attachments there's a whole…

>DNS+SES+SNS+Lambda+Webhooks+API+Datastore+Async Workers so that everything would "just work" with a few lines of humanlayer sdk. What are you smoking my man? Write a python script that begins with the 2 following lines "import openai import email " Simple is better than complex

hmm, like, i love simplicity, and I'm open to other approaches, but I specifically wanted to solve the "send an email to an AI Agent" sort of concept, and give that agent rails to talk back the the human. Doesn't `import email` require SMTP, DNS, signing infra, etc? can it set up MX infra and receive payloads from a mail exchange?

Re: Launch HN: Human Layer (YC F24) – Human-in-the-Loop API for AI Systems

#170

Post by dhorthy »

foota wrote: post #103
bambax wrote: post #89

The idea is great and necessary. It doesn't seem super hard to replicate but why would anyone build their own solution if something already exists and works fine. The thing that got me thinking... how do you make sure an LLM won't eventually hallucinate approval -- or outright lie about it, to get going? Anyway, congrats, this sounds really cool.

At some point the real tool has to be called, at that point, you can do actual checks that do not rely on the AI output (e.g., store the text that the AI generated and check in code that there was an approval for that text).

yeah I think that's right - we put humanlayer in between the non-deterministic (LLM Decision) and the deterministic code (tool execution logic)
Post reply on HN