Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

github.com

Post reply on HN
Print view View original

131–140 of 349 posts

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#131

Post by nmstoker »

dbish wrote: post #129
padolsey wrote: post #127

I'm confused. If this is just a front-end for the OpenAI API then how does it remove the data privacy concern? Your data still ends up with Azure/OpenAI, right? It doesn't stay localized to your instance; it's not your GPU running the transformations. You have no way of knowing whether your data is being used to train models. If customer data is sensitive, I'm pretty sure running a 70B llama (or similar) on a bunch o…

Azure is hosting and operating the service themselves rather then for OpenAI, with all the security requirements that come with that. I assume this comes with different data and access restrictions as well and ability to run in secured instances (and nothing sent to OpenAI the company). Most companies use cloud already for their data, processing, etc. and aren’t running anything major locally, let alone ML models, th…

Yes, this was my understanding.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#132

Post by padolsey »

dbish wrote: post #129
padolsey wrote: post #127

I'm confused. If this is just a front-end for the OpenAI API then how does it remove the data privacy concern? Your data still ends up with Azure/OpenAI, right? It doesn't stay localized to your instance; it's not your GPU running the transformations. You have no way of knowing whether your data is being used to train models. If customer data is sensitive, I'm pretty sure running a 70B llama (or similar) on a bunch o…

Azure is hosting and operating the service themselves rather then for OpenAI, with all the security requirements that come with that. I assume this comes with different data and access restrictions as well and ability to run in secured instances (and nothing sent to OpenAI the company). Most companies use cloud already for their data, processing, etc. and aren’t running anything major locally, let alone ML models, th…

Ah that's fair. But it is my impression that the bulk of privacy/confidentiality concerns (e.g. law/health/..) would require "end to end" data safety. Not sure if I'm making sense. I guess microsoft is somehow more trustworthy than openai themselves...

EDIT: what you say about existing cloud customers being able to extend their trust to this new thing makes sense, thanks.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#133

Post by EGreg »

We just have to trust them and take their word for it? Or what?

https://azure.microsoft.com/en-us/explore/trusted-cloud/priv...

https://azure.microsoft.com/en-us/blog/3-reasons-why-azure-s...

I guess I would trust them, since they're big and they make these promises and other big companies use them.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#135

Post by p1esk »

Q6T46nT668w6i3m wrote: post #111
tikkun wrote: post #41

Earlier quoted context omitted.

I did one. I took a few dozen prompts from my ChatGPT history and ran them through a few LLMs. GPT-4, Bard and Claude 2 came out on top. Llama 2 70b chat scored similarly to GPT-3.5, though GPT-3.5 still seemed to perform a bit better overall. My personal takeaway is I’m going to continue using GPT-4 for everything where the cost and response time are workable. Related: A belief I have is that LLM benchmarks are all…

I don’t know what you mean by “too research oriented.” A common complaint in LLM research is the poor quality of evaluation metrics. There’s no consensus. Everyone wants new benchmarks but designing useful metrics is very much an open problem.

I think he wants to limit evaluations to the most frequent question types seen in the real world.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#136

Post by stavros »

mark_l_watson wrote: post #90

This seems like such an obvious thing to do. I see the use of general purpose LLMs like ChatGPT, but smaller fine tuned models will probably end up being more useful for deployed applications in most companies. Off topic, but I was experimenting with LLongMA-2-7b-16K today, running it very inexpensively in the cloud, and given about 12K of context text it really performed well. This is an easy model to deploy. 7B par…

Is there an easy way to play with these models, as someone who hasn't deployed them? I can download/compile llama.cpp, but I don't know which models to get/where to put them/how to run them, so if someone knows about some automated downloader along with some list of "best models", that would be very helpful.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#137

Post by littlestymaar »

dijital wrote: post #33
tharwan wrote: post #15

Earlier quoted context omitted.

Huh. I missed this one. Got a link?

At a guess it's this story: https://www.vice.com/en/article/xweqbq/microsoft-contractors...

Ah yes it was Skype and not Teams, my bad.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#138

Post by PoignardAzur »

> However, ChatGPT risks exposing confidential intellectual property. One option is to block corporate access to ChatGPT, but people always find workarounds

Pretty bold thing to say to your potential clients. "You can always tell your employees not to use our product, but they won't listen to you."

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#139

Post by H8crilA »

What's the practical difference between this and OpenAI API?

All I can see is the same product but offered by a larger organization. I.e. they're more likely to get the security details right, and you can potentially win more in a lawsuit should things go bad.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#140

Post by quickthrower2 »

samrolken wrote: post #92
Buttons840 wrote: post #32

Earlier quoted context omitted.

As an example of this. I found that GPT4 wouldn't agree with me that C(A) = C(AA^T) until I explained the proof. A few weeks later it would agree in new chats and would explain using the same proof I did presented the same way.

I’ve found that the behavior of ChatGPT can vary widely from session to session. The recent information about GPT4 being a “mixture of experts” might also be relevant. Do we know that it wouldn’t have varied in its answer by just as much, if you had tried in a new session at the same time?

There is randomness even at t=0, there was another HN submission about that
Post reply on HN