Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

github.com

Post reply on HN
Print view View original

91–100 of 349 posts

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#91

Post by TillE »

tikkun wrote: post #41
TuringNYC wrote: post #16

Curious if anyone has done a side-by-side analysis of this offering vs just running LLaMA? I'm currently running a side-by-side comparison/evaluation of MSFT GPT via Cognitive Services vs LLaMA[7B/13B/70B] and intrigued by the possibility of a truly air-gapped offering not limited by external computer power (nor by metered fees racking up.) Any reads on comparisons would be nice to see. (yes, I realize we'll eventual…

I did one. I took a few dozen prompts from my ChatGPT history and ran them through a few LLMs. GPT-4, Bard and Claude 2 came out on top. Llama 2 70b chat scored similarly to GPT-3.5, though GPT-3.5 still seemed to perform a bit better overall. My personal takeaway is I’m going to continue using GPT-4 for everything where the cost and response time are workable. Related: A belief I have is that LLM benchmarks are all…

I think tests like "can this LLM pass an English literature exam it's never seen before" are probably useful, but yeah there's a lot of silly stuff like math tests.

I suppose the question is where are they most commercially viable. I've found them fantastic for creative brainstorming, but that's sort of hard to test and maybe not a huge market.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#92

Post by samrolken »

Buttons840 wrote: post #32
theptip wrote: post #14

Earlier quoted context omitted.

The concern is that ChatGPT is training on your chats (by default, you can opt out but you lose chat history last I checked). So in general enterprises cannot allow internal users to paste private code into ChatGPT, for example.

As an example of this. I found that GPT4 wouldn't agree with me that C(A) = C(AA^T) until I explained the proof. A few weeks later it would agree in new chats and would explain using the same proof I did presented the same way.

I’ve found that the behavior of ChatGPT can vary widely from session to session. The recent information about GPT4 being a “mixture of experts” might also be relevant.

Do we know that it wouldn’t have varied in its answer by just as much, if you had tried in a new session at the same time?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#93

Post by sebzim4500 »

candiddevmike wrote: post #10
Y_Y wrote: post #3

So the public access one isn't private and secure?

> However, ChatGPT risks exposing confidential intellectual property. I don't remember seeing this disclaimer on the ChatGPT website, gee maybe OpenAI should add this so folks stop using it.

It's pretty clear in the FAQ to be fair.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#95

Post by SOLAR_FIELDS »

extr wrote: post #79

One thing I still don't understand is what _is_ the ChatGPT front end exactly? I've used other "conversational" implementations built with the API and they never work quite as well, it's obvious that you run out of context after a few conversation turns. Is ChatGPT doing some embedding lookup inside the conversation thread to make the context feel infinite? I've noticed anecdotally it definitely isn't infinite, but i…

They definitely do some proprietary running summarization to rebuild the context with each chat. Probably a RAG like approach that has had a lot of attention and work

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#96

Post by sebzim4500 »

refulgentis wrote: post #5

Crappy clone of ChatGPT frontend, half missing, half direct copy. Implied and overly vast claims of insecurity + lack of privacy, that are narrowly true, i.e. for _Chat_GPT. Really surprised to see this aggressive of language 1) written down 2) on Github. I'd be pretty pissed if I was OpenAI, regardless of the $10B.

I think OpenAI is entirely on board with the idea that OpenAI sells to consumers and Azure/Microsoft sells the same product to enterprise.

That's how it's been working for months, and if OpenAI objected they would have done something about it.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#97

Post by asabla »

Havoc wrote: post #94

How does this work in terms of utilization? The isolation presumably means buying gpu capacity and only using a %?

Basically you get N tokens/second (or if it was minute, can check tomorrow if you're really interested) per deployment. So if you would outgrow on deployment, just deploy another one (with the associated costs of course).

One deployment = a deployed model which you can query

On top of that, depending on the model you're using, you also see a cost increment for each 1000 request you make.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#98

Post by jmathai »

Normal_gaussian wrote: post #78
judge2020 wrote: post #71

Earlier quoted context omitted.

I imagine most companies serious about this created their own wrappers around the API or contracted it out, likely using private Azure GPUs.

Most companies are either not tech companies, or do not have the knowledge to manage such a project within reasonable cost bounds.

Most companies are trying to figure out exactly what generative AI is and how to use it in their business. Given how new this is - I doubt any large company has done much besides ban the public ChatGPT. So this is probably very relevant for them.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#99

Post by Ecstatify »

mritchie712 wrote: post #60
Ecstatify wrote: post #46

Our company is pushing everyone to use a similar offering. Most of the company is doing low value work … still using excels even though we have a custom ERP. Now seeing people who couldn’t write a coherent email before write 3 page emails. The illusion of being productive by doing more work even though it has zero impact on the bottom line. It’s insane how inefficient organisations are. No doubt we’ll have some KPI s…

What ERP are you using? We've found some early success selling to companies with older "long-tail" ERP's. I've been finding a new one every day.

It’s a proprietary ERP completely custom. Think it was deployed through an acquisition. The problem isn’t the ERP it’s the business. “We want custom processes” but hire the cheapest developers possible to maintain the ERP and then complain about bugs. “We’re agile™” … but have the same inefficient processes for the last 3 years. Cargo cult org, the CEO was taking about Black Swans during COVID … even though Nassim Taleb explicitly said COVID wasn’t a black swan event.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#100

Post by extr »

MaxLeiter wrote: post #83
extr wrote: post #79

One thing I still don't understand is what _is_ the ChatGPT front end exactly? I've used other "conversational" implementations built with the API and they never work quite as well, it's obvious that you run out of context after a few conversation turns. Is ChatGPT doing some embedding lookup inside the conversation thread to make the context feel infinite? I've noticed anecdotally it definitely isn't infinite, but i…

It uses a sliding context windows. Older tokens are dropped as new ones stream in

I don't believe that's the whole story. Other conversational implementations use sliding context windows and it's very noticable as context drops off. Whereas ChatGPT seems to retain the "gist" of the conversation much longer.
Post reply on HN