1–10 of 11 posts
Would love some feedback! No data is being used for retraining models.
Show HN: ChatGPT Alternative with LLaMA Models
chat.nbox.ai
Can I ask, are you using a HuggingFace endpoint to hit the LLaMA models, or deploying it yourself? Still new to this and trying to understand how putting the large models in production works...
Is the censorship placed on top of the llama model or is it a part of it?
This looks great! Can I ask, are you using a HuggingFace endpoint to hit the LLaMA models, or deploying it yourself? Still new to this and trying to understand how putting the large models in production works...
We had the tech from our MLOps platform NimbleBox.ai that let us setup a managed service on all major cloud providers so we just frankenstein-ed it to work for LLMs as well :)
The prompt engineering, specially for web search, is powered by our open-source tool ChainFury (https://chainfury.nbox.ai/)
Nice, llama is much better than I expected. It matches ChatGPT quality for everything I'd use it for, though I haven't tested it on code yet because of data issues and such. Is the censorship placed on top of the llama model or is it a part of it?
Feel free to give it a spin for code as well! We're just the infrastructure layer here so we don't use any data for retraining these models. LLaMA 2 70B coming soon! :D
This looks great! Can I ask, are you using a HuggingFace endpoint to hit the LLaMA models, or deploying it yourself? Still new to this and trying to understand how putting the large models in production works...
Hey hey! We have deployed this on our cloud. It’s running on 2 A10Gs on AWS in the background. We had the tech from our MLOps platform NimbleBox.ai that let us setup a managed service on all major cloud providers so we just frankenstein-ed it to work for LLMs as well :) The prompt engineering, specially for web search, is powered by our open-source tool ChainFury ( https://chainfury.nbox.ai/ )
seems to be better than gpt-3.5
this looks awesome. when is LLaMA 2 70B coming?