Live data from Hacker News

A site to share LLMs APIs via Tor

21–22 of 22 posts

Re: A site to share LLMs APIs via Tor

#21

Post by LoganDark » Fri, Jun 23, 2023, 5:03 PM UTC

aortega wrote: post #4

codetrotter wrote: post #3

Earlier quoted context omitted.

Thank you for making this :) I wish more services were made with Tor access in mind, and that such services would feature on HN more often!

Generative AIs are perfect for TOR, as the are low-bandwidth, high-latency services. I should say that I cannot accept all the credits for doing the site, as it was made 90% by the AI itself (under my direction, I hope).

> it was made 90% by the AI itself

It shows.

Re: A site to share LLMs APIs via Tor

#22

Post by 19h » Tue, Jul 04, 2023, 11:15 PM UTC

Unrelated to the actual site — we’ve been testing a tuned version of airoboros-64b on a H100 cluster as a drop in replacement for Claude-100k and GPT4-32k. It’s performing rather well in the generation of text but the 2k context definitely shows, also the reasoning capabilities just as in LLaMA are suboptimal.

For instance, obtaining JSON structured data from freetext is a rather impossible task.

That said, for summarization of leq 2k token texts the model performs extremely well.

This is mostly due to its unfiltered nature, where there would be clear biases visible in models like falcon-40b-instruct or other LLaMA derivatives.

Post reply on HN