Live data from Hacker News

Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

github.com

141–150 of 349 posts

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#141
post #9

Earlier quoted context omitted.

I thought sama said they don’t use data going through the api for training. Guess we can’t trust that statement

That is correct, they do not use the data going through the API for training, but they do use the data from the web and mobile interfaces (unless you explicitly turn it off).

“We don’t water down your beer”.

Oh nice!

“But that is lager”

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#144

What's the practical difference between this and OpenAI API? All I can see is the same product but offered by a larger organization. I.e. they're more likely to get the security details right, and you can potentially win more in a lawsuit should things go bad.

Compliance and customer trust. Azure can sign a BAA, for example. If you are Building LLM capability on top of your SaaS, your customers want assurances about their data.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#145

Earlier quoted context omitted.

it's the SOTA for chat(prove me wrong), and you can always use the API directly even for simple tasks they're less reliable and needs more prompt engineering

> it's the SOTA for chat(prove me wrong) GPT-4 beats ChatGPT on all benchmarks. You can easily google these.

I tried and got nothing useful. What's the difference between GPT-4 and ChatGPT Plus using GPT-4?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#146

What's the practical difference between this and OpenAI API? All I can see is the same product but offered by a larger organization. I.e. they're more likely to get the security details right, and you can potentially win more in a lawsuit should things go bad.

A few months ago my team moved to Azure for capacity reasons. We were constantly dealing with 429 errors and couldn't get in touch with Open AI, while Azure offered more instances.

Eventually got more from Open AI so we load balance both. The only difference is the 3.5 turbo model on Azure is outdated.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#147
post #79

One thing I still don't understand is what _is_ the ChatGPT front end exactly? I've used other "conversational" implementations built with the API and they never work quite as well, it's obvious that you run out of context after a few conversation turns. Is ChatGPT doing some embedding lookup inside the conversation thread to make the context feel infinite? I've noticed anecdotally it definitely isn't infinite, but i…

Logic for azure chatgpt's "infinite context" summarisation is in https://github.com/microsoft/azurechatgpt/blob/main/src/feat...

*Edit Azure chatgpt, would be amazed/disappointed if chatgpt used langchain.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#148
post #129

Earlier quoted context omitted.

Azure is hosting and operating the service themselves rather then for OpenAI, with all the security requirements that come with that. I assume this comes with different data and access restrictions as well and ability to run in secured instances (and nothing sent to OpenAI the company). Most companies use cloud already for their data, processing, etc. and aren’t running anything major locally, let alone ML models, th…

Ah that's fair. But it is my impression that the bulk of privacy/confidentiality concerns (e.g. law/health/..) would require "end to end" data safety. Not sure if I'm making sense. I guess microsoft is somehow more trustworthy than openai themselves... EDIT: what you say about existing cloud customers being able to extend their trust to this new thing makes sense, thanks.

Right. If I was an European company worried about, say, industrial espionage, this wouldn't be nearly enough to reassure me.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#149

This is potentially a huge deal. Companies are concerned using ChatGPT might violate data privacy policies if someone puts in user data or invalidate trade secrets protections if someone uploads sections of code. I suspect many companies have been waiting for an enterprise version.

This is a web UI that talks to a (separate) Azure OpenAI resource that you can deploy into your subscription as a SaaS instance.

So how is it any different

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#150
post #85

Earlier quoted context omitted.

This is one of the things that make me uncomfortable about proprietary llm. They get task performance by doing a lot more than just feeding a prompt straight to an llm, and then we performance compare them to raw local options. The problem is, as this secret sauce changes, your use case performance is also going to vary in ways that are impossible for you to fix. What if it can do math this month and next month the h…

I'm not sure you realize how proprietary LLMs are being built on. No one is doing secret math in the backend people are building on. The OpenAI API allows you to call functions now, but even that is just a formalized way of passing tokens into the "raw LLM". All the features in the comment you replied to only apply to the web interface , and here you're being given an open interface you can introspect.

It was a contrived example to make a point, one that seems to have flown over your head.
Post reply on HN