Viewing profile — jiayq84

I do a startup called Lepton AI. We provide AI PaaS and fast AI runtimes as a service, so we keep a close eye on the IaaS supply chain. For the last few months we see supply chain …

Full open-source code with Apache license here: https://github.com/leptonai/search_with_lepton

Hi folks - Yangqing from Lepton here. The idea came from a coffee chat with a colleague on the question: how much of the RAG quality comes from the old good search engine, vs LLMs?…

Hi there! We built a Bing-chat-like conversational search engine with less than 500 lines of python code (and a similar amount of frontend script). The source code is fully open-so…

General availability of the structured decoding capability for ALL open-source models hosted on Lepton AI. Simply provide the schema you want the LLM to produce, and all our model …

Super cool exhibition of what a local machine can already do in the AI frenzy!

Oh wow yeah, that is a beast. Let me give it a shot.

Thanks so much for the warm words!

Thanks - we definitely agree that llama.cpp is great. Big fan of their optimizations. We are more or less orthogonal to the engines though - in the sense that we serve as the infra…

Thanks - the policies are listed here: https://www.lepton.ai/policies we'll put a link on our homepage. In short - we do not collect, record, or log any of your prompts and respons…

In theory one can have 640G = 8 * 80G A100s memory and launch it. 180B Falcon with fp16 will be 360G, so there would be enough memory. It's definitely going to be very expensive in…

Great catch! Our cloud machine encountered a cuda error (the GPU fell off PCIe) - had to restart it. It's back to normal now. All the more reason to have a managed version of servi…

It's not only about "building a docker" but also maintaining multiple models, multiple environments and a lot of users. Imagine there is a group of engineers each needing to deploy…

To show some actual coding examples, We have made the python library open-source at https://github.com/leptonai/leptonai/ . With it, launching a common HuggingFace model is as simp…

Hello Hacker News! We're Yangqing, Xiang and JJ from lepton.ai. We are building a platform to run any AI models as easy as writing local code, and to get your favorite models in mi…

I don’t want to be mean, but since you mentioned RCNN - no, you are dead wrong. RCNN was open sourced in 2014, check the repo: https://github.com/rbgirshick/rcnn Not to mention tha…

Just to clarify a little bit... "At the time, very few object detection models had public implementations" - this is wrong. Almost all object detection models had public implementa…

We are moving to Apache 2.0 in a few days. Early draft at https://github.com/Yangqing/caffe2/tree/apache pending double check to make sure we are honoring all existing contributors…

Yangqing (creator and main author of Caffe/Caffe2) here. We are moving to Apache 2.0 in a few days.

Yangqing (creator and main author of Caffe/Caffe2) here. We are moving to Apache 2.0 in a few days.

So what we do is to keep syntax=proto2, but allow users to compile with both protobuf 2.x and protobuf 3.x libraries. Minumum need is 2.6.1. We kind of feel that this gives maximum…