I've tried it out. Right now it seems more of a proof-of-concept than a real-world application. Having said that, the concepts and ideas in there are definitely reusable.
Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
291–300 of 349 posts
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#292Earlier quoted context omitted.
Llama 2 might by some measures be close to GPT 3.5, but it’s nowhere near GPT 4, nor Anthropic Claude 2 or Cohere’s model. The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. My sense is that the foundation model companies have an edge for now and will probably stay a few steps ahead of the open source realm simply for e…
> The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. Llama2 came out of Meta's AI group. Meta pays researcher salaries competitive with any other group, and their NLP team is one of the top groups in the world. For researchers it is increasingly the most attractive industrial lab because they release the research openly…
There are L5 engineers with 3 YOE making 900k+ at OpenAI right now. Tough to say what they're paying their PhDs, but I'd imagine it's similarly nutty.
https://www.levels.fyi/companies/openai/salaries/software-en...
FAANG pays exceptionally well (I'd know), but what's being offered at OpenAI is eye-popping, even for SWEs. I think they're trying to dig their moat by absorbing the absolute best of the best.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#293Earlier quoted context omitted.
> The closed source players have the best researchers Is that definitely why? GPT 3.5 and GPT 4 are far larger than 70B, right? So if a 70B, local model like LLaMA can even remotely rival them, would that not suggest that LLaMA is fundamentally a better model? For example, would a LLaMA model with even half of GPT 4's parameters be projected to outperform it? Is that how it works? [I'm not super familiar with LLM tec…
Yeah I've been wondering about this too. Word on the street is that GPT4 is several times the size of GPT3.5. Yet I don't feel it's several times as good for sure. Apparently there's a diminishing returns effect on ever enlarging the model.
I believe what they discovered was that 4 is an ensemble model, comprised of (8) GPT3.5s. Things may have changed or been found to not be true on this though.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#294This seems like such an obvious thing to do. I see the use of general purpose LLMs like ChatGPT, but smaller fine tuned models will probably end up being more useful for deployed applications in most companies. Off topic, but I was experimenting with LLongMA-2-7b-16K today, running it very inexpensively in the cloud, and given about 12K of context text it really performed well. This is an easy model to deploy. 7B par…
Is there an easy way to play with these models, as someone who hasn't deployed them? I can download/compile llama.cpp, but I don't know which models to get/where to put them/how to run them, so if someone knows about some automated downloader along with some list of "best models", that would be very helpful.
I usually run them on Google Colab, and occasionally a GPU VPS on Lambda Labs. Hugging Face model card documentation usually have a complete Python example script for loading and running a model.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#295Earlier quoted context omitted.
> The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. Llama2 came out of Meta's AI group. Meta pays researcher salaries competitive with any other group, and their NLP team is one of the top groups in the world. For researchers it is increasingly the most attractive industrial lab because they release the research openly…
There are L5 engineers with 3 YOE making 900k+ at OpenAI right now. Tough to say what they're paying their PhDs, but I'd imagine it's similarly nutty. https://www.levels.fyi/companies/openai/salaries/software-en... FAANG pays exceptionally well (I'd know), but what's being offered at OpenAI is eye-popping, even for SWEs. I think they're trying to dig their moat by absorbing the absolute best of the best.
Write it says no results found for l3
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#296Did this just kill a lot of AI startups that were targeting enterprises?
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#297Earlier quoted context omitted.
> The closed source players have the best researchers Is that definitely why? GPT 3.5 and GPT 4 are far larger than 70B, right? So if a 70B, local model like LLaMA can even remotely rival them, would that not suggest that LLaMA is fundamentally a better model? For example, would a LLaMA model with even half of GPT 4's parameters be projected to outperform it? Is that how it works? [I'm not super familiar with LLM tec…
If you read the Llama2 paper it is very clear that small amounts of data (thousands of records) make vast difference at the instruction turning stage. From the Llama2 paper: > Quality Is All You Need. > Third-party SFT data is available from many different sources, but we found that many of these have insufficient diversity and quality — in particular for aligning LLMs towards dialogue-style instructions. As a result…
SFT?
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#298Earlier quoted context omitted.
OK, fair enough. Please give me an example of a customer facing chatbot that Llama 2 (and unbearable to use) and GPT 4 customer facing chatbot that is a joy to use. I think at the end of the day, you still have customers dreading such interactions.
Using GPT3.5/4 in our language learning app and people seem to enjoy it. [1] Tried Llama2 and it definitely doesn’t even come close for what we’re doing. Would absolutely need fine tuning. Maybe customers don’t enjoy chat bots for customer support, but there are a million other uses for these models. I, for example, LOVE github copilot. 1. https://squidgies.app
Is it cost prohibitive
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#299Earlier quoted context omitted.
IMO Llama2 really isn’t close to 3.5. It still has regular mode collapse (or whatever you call getting repetitive and nonsensical responses after a while), it has very poor mathematical/logical reasoning and is not good at following multi-part instructions. It just sounds like 3.5/4 because it was trained on it.
You're mixing up the language model with the chat bot . The llama2 is a language model. I imagine the language model behind chatgpt is not much different (perhaps it's better, but not by many months AI research time). It likely also suffers from "mode collapse" issues etc. But 3.5 also has a lot of systems around it that detects mode collapse and applies some kind of mitigation, forcing the model to give a more reaso…
So this would be testable by showing that chatGPT makes more mistakes than prompting via API? Or would you consider the API a chatbot, too?
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#300Earlier quoted context omitted.
IMO Llama2 really isn’t close to 3.5. It still has regular mode collapse (or whatever you call getting repetitive and nonsensical responses after a while), it has very poor mathematical/logical reasoning and is not good at following multi-part instructions. It just sounds like 3.5/4 because it was trained on it.
Llama 2 wasn't trained on ChatGPT/GPT4. I think maybe you are thinking of the Vicuna models? https://lmsys.org/blog/2023-03-30-vicuna/
So it’s true that it would violate the OpenAI terms for Llama to be trained with ChatGPT completions, but how do we know? We don’t know the training data for Llama, we just get weights.