Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Find AI – Perplexity Meets LinkedIn

usefind.ai

Post reply on HN
Print view View original

31–40 of 78 posts

Re: Show HN: Find AI – Perplexity Meets LinkedIn

#32

Post by rahimnathwani »

philip1209 wrote: post #28
janalsncm wrote: post #27

Earlier quoted context omitted.

Typically with an LLM you would tokenize strings in a batch and use attention masks to run inference in parallel. OpenAI must have some similar capability. Looks like they have a batch API.

Yes, but the batch API takes up to 24 hours to respond. We use it, but not for user-facing search queries.

I don't think janalalcm is talking about the batch API, but just putting multiple profiles in a single API call.

If your prompt is short, it won't make much difference to the cost. But if your prompt is, say, about the same length as a single profile, you could save almost half your inference costs by analysing ten at a time.

Re: Show HN: Find AI – Perplexity Meets LinkedIn

#34

Post by esafak »

philip1209 wrote: post #26
philip1209 wrote: post #9

Thanks everybody for trying this out. I just looked at our logs and we're doing >2k requests per minute to OpenAI right now. Free users only get partial search results. If there are any you want to see run to completion, reply here or email me and I'll mark it to run to completion. (The code PRODUCTHUNT is also available this week for a free month of access).

Update: >5k requests per minute to OpenAI right now

Don't forget to rate limit every end point.

Re: Show HN: Find AI – Perplexity Meets LinkedIn

#35

Post by philip1209 »

esafak wrote: post #34
philip1209 wrote: post #26

Earlier quoted context omitted.

Update: >5k requests per minute to OpenAI right now

Don't forget to rate limit every end point.

Yup, we have some decent safeguards in place. It’s technically difficult to allow logged out users to try the product, but I think people appreciate it.

Re: Show HN: Find AI – Perplexity Meets LinkedIn

#36

Post by philip1209 »

rahimnathwani wrote: post #32
philip1209 wrote: post #28

Earlier quoted context omitted.

Yes, but the batch API takes up to 24 hours to respond. We use it, but not for user-facing search queries.

I don't think janalalcm is talking about the batch API, but just putting multiple profiles in a single API call. If your prompt is short, it won't make much difference to the cost. But if your prompt is, say, about the same length as a single profile, you could save almost half your inference costs by analysing ten at a time.

Ah I’ll look into that. Thanks.

Re: Show HN: Find AI – Perplexity Meets LinkedIn

#37

Post by 7thpower »

> I started building Find AI to make it easier to search for people. I initially started just having GPT review people's LinkedIn profiles and websites, but it cost thousands of dollars per search (!).

Can you help me connect to what was costing >$1k/search or is that hyperbole? Genuinely interested, not patronizing.

Re: Show HN: Find AI – Perplexity Meets LinkedIn

#40

Post by nielsole »

7thpower wrote: post #37

> I started building Find AI to make it easier to search for people. I initially started just having GPT review people's LinkedIn profiles and websites, but it cost thousands of dollars per search (!). Can you help me connect to what was costing >$1k/search or is that hyperbole? Genuinely interested, not patronizing.

I suppose every search was passing all indexed documents into GPT asking for a rating
Post reply on HN