Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

github.com

Post reply on HN
Print view View original

281–290 of 349 posts

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#281

Post by qrios »

oceanplexian wrote: post #215
xcdzvyn wrote: post #197

Earlier quoted context omitted.

> The closed source players have the best researchers Is that definitely why? GPT 3.5 and GPT 4 are far larger than 70B, right? So if a 70B, local model like LLaMA can even remotely rival them, would that not suggest that LLaMA is fundamentally a better model? For example, would a LLaMA model with even half of GPT 4's parameters be projected to outperform it? Is that how it works? [I'm not super familiar with LLM tec…

LLamA 2 at 70B is, let’s say pessimistically 70% as good as GPT3.5. This makes me think that OpenAI is lying about their parameter count, are vastly less efficient than LLaMA, or, the lager model sizes have diminishing returns. Either way, your point is a good one. Something doesn’t add up.

Without questioning the statement '70% as good as GPT3.5', but wouldn't that be quantifying a quality, and a Turing test? Also: maybe these missing 30% are the hard part.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#282

Post by Xenoamorphous »

simonbutt wrote: post #147
extr wrote: post #79

One thing I still don't understand is what _is_ the ChatGPT front end exactly? I've used other "conversational" implementations built with the API and they never work quite as well, it's obvious that you run out of context after a few conversation turns. Is ChatGPT doing some embedding lookup inside the conversation thread to make the context feel infinite? I've noticed anecdotally it definitely isn't infinite, but i…

Logic for azure chatgpt's "infinite context" summarisation is in https://github.com/microsoft/azurechatgpt/blob/main/src/feat... *Edit Azure chatgpt, would be amazed/disappointed if chatgpt used langchain.

This is Azure's docs to create a conversation: https://learn.microsoft.com/en-us/azure/cognitive-services/o...

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#283

Post by agentgumshoe »

pietz wrote: post #253
Roark66 wrote: post #244

Private and secure? I thought the main issue with privacy and security of (not at all)OpenAI models is that by using their products you agree for them to retain all the data you send and receive from the models forever for whatever they choose to use it for. Or is this just a thing for free use? If you pay, do you get a Ts&Cs that don't contain any wording like this? Still, even if there was no specific "we own every…

The Azure SLAs state that neither the chats are stored nor used for training in any way. They are private and protected in the same way all the other sensitive data is stored on Azure. On top, you might argue that Microsoft and Azure are easier to trust than a still rather new AI startup.

So what do they train it on then?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#284

Post by jokethrowaway »

nl wrote: post #217
ttul wrote: post #162

Earlier quoted context omitted.

Llama 2 might by some measures be close to GPT 3.5, but it’s nowhere near GPT 4, nor Anthropic Claude 2 or Cohere’s model. The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. My sense is that the foundation model companies have an edge for now and will probably stay a few steps ahead of the open source realm simply for e…

> The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. Llama2 came out of Meta's AI group. Meta pays researcher salaries competitive with any other group, and their NLP team is one of the top groups in the world. For researchers it is increasingly the most attractive industrial lab because they release the research openly…

hearsay, but I've heard OpenAI pays significantly more

I agree that Meta hired some amazing researchers so we'll see what the future holds

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#285

Post by e12e »

CodeCompost wrote: post #254
pyrophane wrote: post #181

Earlier quoted context omitted.

> While I haven't tested it extensively, 70B model is supposed to rival Chat GPT 3.5 in most areas, and there are now some new fine-tuned versions that excel at specific tasks That has been my experience. Having experimented with both (informally), Llama 2 is similar to GPT-3.5 for a lot of general comprehension questions. GPT-4 is still the best amongst the closed-source, cutting edge models in terms of general conv…

Can it handle other languages besides English?

Not anywhere near as well as ChatGPT 4 (for chat anyway - maybe the model is better)?

Prompt:

> Hvad tycks om at fika nu?

ChatGPT 4

> Det låter som en trevlig idé! Fika är ju alltid gott. Vad skulle du vilja ha till din fika? (Oj, ursäkta för emojis! )

https://chat.openai.com/share/8e89a16f-f182-4f62-b9fa-f93cd5...

Llama2:

> I apologize, but I don't understand what you mean by "fika nu." Could you please provide more context or clarify your question so I can better assist you?

https://hf.co/chat/r/kOF2qst

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#286

Post by [deleted] »

mark_l_watson wrote: post #90

This seems like such an obvious thing to do. I see the use of general purpose LLMs like ChatGPT, but smaller fine tuned models will probably end up being more useful for deployed applications in most companies. Off topic, but I was experimenting with LLongMA-2-7b-16K today, running it very inexpensively in the cloud, and given about 12K of context text it really performed well. This is an easy model to deploy. 7B par…

[deleted]

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#287

Post by e12e »

actionfromafar wrote: post #258
kiratp wrote: post #252

Earlier quoted context omitted.

> Starting on March 1, 2023, we are making two changes to our data usage and retention policies: > OpenAI will not use data submitted by customers via our API to train or improve our models, unless you explicitly decide to share your data with us for this purpose. You can opt-in to share data. > Any data sent through the API will be retained for abuse and misuse monitoring purposes for a maximum of 30 days, after whi…

Unless required by law… I wonder what law.

> Unless required by law… I wonder what law.

Any law. It just makes explicit that a contract can't supercede laws. Even if it was left out, Microsoft is still subject to laws.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#288

Post by e12e »

laserbeam wrote: post #263
actionfromafar wrote: post #258

Earlier quoted context omitted.

Unless required by law… I wonder what law.

"Unless required by law" is wording required to enable a mechanism called "legal hold". If an authority or lawyer discovers some documents for a case they get to prevent their automatic deletion until that case gets closed. Basically, you don't want to lose evidence if there's a warrant or ongoing lawsuit. I really see no problem with that clause in most ToS documents. Now, I think you can do shady stuff with that wo…

> is wording required to enable a mechanism called "legal hold"

I don't think this is accurate. At least in Norway you can't "just not" keep records required by law - any section in a contract in conflict with current law would simply be invalid?

I think the section just clarifies that Microsoft will comply with laws requiring them to keep data (eg the "anti-terror" laws that might require data retention).

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#289

Post by dalbasal »

Roark66 wrote: post #244

Private and secure? I thought the main issue with privacy and security of (not at all)OpenAI models is that by using their products you agree for them to retain all the data you send and receive from the models forever for whatever they choose to use it for. Or is this just a thing for free use? If you pay, do you get a Ts&Cs that don't contain any wording like this? Still, even if there was no specific "we own every…

Privacy and security... in practice, can mean different things.

In HN-space, it is at its most abstract, idealistic, etc. At the practical level this services is aimed at... it might mean compliance, or CYA. Less cynically, it might mean something mundane. MSFT's guarantee, a responsive place to report security issues.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#290

Post by 37112741 »

RockyMcNuts wrote: post #211
37112741 wrote: post #186

Nothing in the repo details how this addresses privacy concerns of running inference on someone else's LLM. To be isolated from other users of the service is not the same thing as having a private inference engine. > Private: Built-in guarantees around the privacy of your data and fully isolated from those operated by OpenAI. Do tell.

So where do you draw the line? No cloud instances, no cloud SQL like Snowflake, no Teams or Office 365, no S3/blob storage? Run everything on-prem like 10 years ago? It's only going to get more impossible. All that VC money going in at 100x revenue needs a return and they aren't going leave money on the table with full-featured open-source or CentOS type alternatives. All those data engineering startups, database pro…

They should clearly spell out what is and is not "private". As it is we simply have a blurp about some undefined guarantees. And some comments here in thread saying "this is as close as you're going to get to local GPT" are deeply wrong. But then there is easy VC money (just like with ADs ..) and certain "clever" geeks throw social responsibility out the window as usual and are pushing all sorts of deeply invasive applications ("let our proxy for Microsoft hoover your inbox!") based on these undefined "Privacy guarantees".

If we accept this just we accepted the very flawed solutions we were given by corporation regarding social networking and ads, we are going to be stuck with it, suffer the consequences, and there will be no incentive to develop alternatives that actually address issues and work.

Homomorphic Encryption works. It just doesn't work very efficiently right now but that is an intellectual problem that can be solved if we push for actual privacy for this critical technology as it will be fully enmeshed in all parts of our lives.

"Think of the children" if that helps.

Post reply on HN