Can any one shed light on what "local" means? Local on my own private machine or local in my Azure Tenant?
I am pretty sure it means run the UI locally and access Azure-hosted ChatGPT. The environment vars seem to indicate that as well.
Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
231–240 of 349 posts
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#232Can any one shed light on what "local" means? Local on my own private machine or local in my Azure Tenant?
Assuming you are referring to this section: https://github.com/microsoft/azurechatgpt/blob/main/docs/3-r...
It means you run the front end (the chat-gui) and the backend code from the repo. This code connects to cosmo-db for uploading documents used for "chat with you pdf" and connects to an OpenAI instance on Azure for the chat inferrence.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#233This appears to be a web frontend with authentication for Azure's OpenAI API, which is a great choice if you can't use Chat GPT or its API at work. If you're looking to try the "open" models like Llama 2 (or it's uncensored version Llama 2 Uncensored), check out https://github.com/jmorganca/ollama or some of the lower level runners like llama.cpp (which powers the aforementioned project I'm working on) or Candle, the…
LLaMA2 is still quite a bit behind ChatGPT 3.5 and this mainly get reflected in coding and math. It's easy to beat NLP based benchmark but much much harder to beat NLP+math+coding togather. I think this gap reflects gap in reasoning but we don't have a good non-coding/non-math benchmark to measure it.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#234I'm a little confused by how the relationship works between OpenAI and Microsoft. It is possible for anyone to register for an OpenAI account and use their APIs. Within Azure the same thing is much more difficult as it is necessary to be a "real" business in order to use it. I maintain an open source OpenAI library and would like to add support for Azure but can't because of this restriction. Why can't I just use my regular Azure account?
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#235Earlier quoted context omitted.
Llama 2 might by some measures be close to GPT 3.5, but it’s nowhere near GPT 4, nor Anthropic Claude 2 or Cohere’s model. The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. My sense is that the foundation model companies have an edge for now and will probably stay a few steps ahead of the open source realm simply for e…
> Llama 2 might by some measures be close to GPT 3.5, but it’s nowhere near GPT 4 I think you're right about this, and benchmarks we've run at Anyscale support this conclusion [1]. The caveat there (which I think will be a big boon for open models) is that techniques like fine-tuning makes a HUGE difference and can bridge the quality gap between Llama-2 and GPT-4 for many (but not all) problems. [1] https://www.anysc…
Frankly, number of benchmarks you guys are using are too narrow. In fact these benchmarks are "old world" benchmarks, easy to game through finetuning and we should be stop using them altogether for LLMs. Why are you not using Big Bench Hard or OpenAI evals?
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#236It was really good when the access was enabled via OpenAI, but ever since its moved to Azure subscription, getting preview access is stalled. Wouldn't be a big deal for others, but for smalltime devs like me it becomes a big challenge.. Hope OpenAI provides a developer env or so where we can try things out..
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#237IS it possible for someone to give us the lower bound on the cost of running a 70B model in the cloud? How much memory does Llamba-2 take? What would it cost to fine tune it?
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#238Earlier quoted context omitted.
> While I haven't tested it extensively, 70B model is supposed to rival Chat GPT 3.5 in most areas, and there are now some new fine-tuned versions that excel at specific tasks That has been my experience. Having experimented with both (informally), Llama 2 is similar to GPT-3.5 for a lot of general comprehension questions. GPT-4 is still the best amongst the closed-source, cutting edge models in terms of general conv…
Could you give examples of smaller models trained on specific datasets?
it can be almost anything like your HN comments or some corporate wiki, then get colab pro 10$ month or some juicy gaming machine and fine-tune that using eg this tutorial https://www.philschmid.de/instruction-tune-llama-2 but https://www.reddit.com/r/LocalLLaMA/ is full of different fine tuned models.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#239When will LLMs be good enough to write the code for a competitive or better LLM to themselves?
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#240Earlier quoted context omitted.
> However, ChatGPT risks exposing confidential intellectual property. I don't remember seeing this disclaimer on the ChatGPT website, gee maybe OpenAI should add this so folks stop using it.
If you use ChatGPT through the app or website they can use the data for training, unless you turn it off. https://help.openai.com/en/articles/5722486-how-your-data-is...
Providing my data for training doesn't imply that it risks being exposed.
If you understand what happens on a technical level, it might be possible, but OpenAI has never said this was a risk by using their product.