Darn I just spent a week or so working on a ChatGPT clone that used Azure ChatGPT API due to the privacy aspect. Wasted effort I guess.
This is exactly the same
Welcome to the club :)
Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
151–160 of 349 posts
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#152This appears to be a web frontend with authentication for Azure's OpenAI API, which is a great choice if you can't use Chat GPT or its API at work.
If you're looking to try the "open" models like Llama 2 (or it's uncensored version Llama 2 Uncensored), check out https://github.com/jmorganca/ollama or some of the lower level runners like llama.cpp (which powers the aforementioned project I'm working on) or Candle, the new project by hugging face.
What's are folks' take on this vs Llama 2, which was recently released by Facebook Research? While I haven't tested it extensively, 70B model is supposed to rival Chat GPT 3.5 in most areas, and there are now some new fine-tuned versions that excel at specific tasks like coding (the 'codeup' model) or the new Wizard Math (https://github.com/nlpxucan/WizardLM) which claims to outperform ChatGPT 3.5 on grade school math problems.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#153Earlier quoted context omitted.
ahhhh, but they're pasting the 3 page email into ChatGPT ("summarize this"). The future is here.
Wouldn't be surprised if that was next Outlook feature. Cue someone making some horrible error because some crucial information didn't survive ChatGPT->ChatGPT round-trip
Actually this was in an Azure hacktoon some time ago https://devpost.com/software/amabot
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#154Earlier quoted context omitted.
I'm not sure you realize how proprietary LLMs are being built on. No one is doing secret math in the backend people are building on. The OpenAI API allows you to call functions now, but even that is just a formalized way of passing tokens into the "raw LLM". All the features in the comment you replied to only apply to the web interface , and here you're being given an open interface you can introspect.
It was a contrived example to make a point, one that seems to have flown over your head.
No it was a bad (straight up wrong) example because you don't understand how people are building applications on proprietary LLMs.
If you did you'd also know what evals are.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#155One thing I still don't understand is what _is_ the ChatGPT front end exactly? I've used other "conversational" implementations built with the API and they never work quite as well, it's obvious that you run out of context after a few conversation turns. Is ChatGPT doing some embedding lookup inside the conversation thread to make the context feel infinite? I've noticed anecdotally it definitely isn't infinite, but i…
Logic for azure chatgpt's "infinite context" summarisation is in https://github.com/microsoft/azurechatgpt/blob/main/src/feat... *Edit Azure chatgpt, would be amazed/disappointed if chatgpt used langchain.
That doesn't really look right to me, it looks like that's for responding regarding uploaded documents. I see nothing related to infinite context.
Also this is the azure repo from OP, nothing to do with the actual ChatGPT front-end that was asked about. I highly doubt the official ChatGPT front-end uses langchain, for example.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#156One thing I still don't understand is what _is_ the ChatGPT front end exactly? I've used other "conversational" implementations built with the API and they never work quite as well, it's obvious that you run out of context after a few conversation turns. Is ChatGPT doing some embedding lookup inside the conversation thread to make the context feel infinite? I've noticed anecdotally it definitely isn't infinite, but i…
Logic for azure chatgpt's "infinite context" summarisation is in https://github.com/microsoft/azurechatgpt/blob/main/src/feat... *Edit Azure chatgpt, would be amazed/disappointed if chatgpt used langchain.
I don't see anything related to an infinite context in there. There's only a reference to a server-side `summary` variable which suggests that there is a summary of previous posts which will get sent along with the question for context, as is to be expected. Nothing suggests an infinite context.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#157Curious if anyone has done a side-by-side analysis of this offering vs just running LLaMA? I'm currently running a side-by-side comparison/evaluation of MSFT GPT via Cognitive Services vs LLaMA[7B/13B/70B] and intrigued by the possibility of a truly air-gapped offering not limited by external computer power (nor by metered fees racking up.) Any reads on comparisons would be nice to see. (yes, I realize we'll eventual…
We (at Anyscale) have benchmarked GPT-4 versus the Llama-2 suite of models on a few problems: functional representation, SQL generation, grade-school math question answering.
GPT-4 wins by a lot out of the box. However, surprisingly, fine-tuning makes a huge difference and allows the 7B Llama-2 model to outperform GPT-4 on some (but not all) problems.
This is really great news for open models as many applications will benefit from smaller, faster, and cheaper fine-tuned models rather than a single large, slow, general-purpose model (Llama-2-7B is something like 2% of the size of GPT-4).
GPT-4 continues to outperform even the fine-tuned 70B model on grade-school math question answering, likely due to the data Llama-2 was trained on (more data for fine-tuning helps here).
https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehe...
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#158A lot of companies are already using projects like chatbot-ui with Azure's OpenAI for similar local deployments. Given this is as close to local ChatGPT as any other project can get, this is a huge deal for all those enterprises looking to maintain control over their data. Shameless plug: Given the sensitivity of the data involved, we believe most companies prefer locally installed solutions to cloud based ones at le…
I find it interesting to see how competitive this space got so quickly. How do these stacks differentiate?
[flagged]
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#159Earlier quoted context omitted.
As an example of this. I found that GPT4 wouldn't agree with me that C(A) = C(AA^T) until I explained the proof. A few weeks later it would agree in new chats and would explain using the same proof I did presented the same way.
This is kinda creepy. But at the same time, how do they do that? I thought the training of these models stopped in September 2021/2022. So how do they do these incremental trainings?
All the public and (leaked) private statements I have seen state that this is not happening. As siblings noted, MoE probably explains this variance.
AIUI they are using current chat data for training GPT-5, not re-finetuning the existing models.
Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use
#160Earlier quoted context omitted.
That's mostly because there's very little value in deep speculation there. It's not particularly more fluid than anything you couldn't whip up yourself (and the repo linked proves that) but there's also not much value in trying to compete with ChatGPT's frontend. For most products ChatGPT's frontend is the minimal level of acceptable performance that you need to beat, not an maximal one really worth exploring.
What front end is better than ChatGPT? Is the OP implementation doing running summarization or in-convo embedding lookup?
It sounds like a cop-out but: it's one made for your use-case.
If you're letting people do fun long-form roleplay adventures using summarization alongside some sort of named entity K-V store driven by the LLM would be a good strategy.
If you're building a tool that's mostly for internal data, something that leans heavily into detailed answers with direct verbatim citations and having your frontend create new threads when there's a clear break in the topic of a request is a clever strategy since quality drops with context length and you want to save tokens for citations.
People who are saying LLMs suck or are X or are Y are mostly just completely underutilizing them because LLMs make it super easy to solve problems superficially: when it comes to actually scaling those solutions to production you need more than random RAG vector database wrappers.