Live data from Hacker News

Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

github.com

241–250 of 349 posts

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#241
post #218

Earlier quoted context omitted.

> The closed source players have the best researchers Is that definitely why? GPT 3.5 and GPT 4 are far larger than 70B, right? So if a 70B, local model like LLaMA can even remotely rival them, would that not suggest that LLaMA is fundamentally a better model? For example, would a LLaMA model with even half of GPT 4's parameters be projected to outperform it? Is that how it works? [I'm not super familiar with LLM tec…

If you read the Llama2 paper it is very clear that small amounts of data (thousands of records) make vast difference at the instruction turning stage. From the Llama2 paper: > Quality Is All You Need. > Third-party SFT data is available from many different sources, but we found that many of these have insufficient diversity and quality — in particular for aligning LLMs towards dialogue-style instructions. As a result…

This quote is quite funny taken out of context like this. Top AI researchers find that garbage in === garbage out.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#243

Earlier quoted context omitted.

> However, ChatGPT risks exposing confidential intellectual property. I don't remember seeing this disclaimer on the ChatGPT website, gee maybe OpenAI should add this so folks stop using it.

It's pretty clear in the FAQ to be fair.

The comment you are responding to is sarcastic

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#244
Private and secure? I thought the main issue with privacy and security of (not at all)OpenAI models is that by using their products you agree for them to retain all the data you send and receive from the models forever for whatever they choose to use it for. Or is this just a thing for free use?

If you pay, do you get a Ts&Cs that don't contain any wording like this? Still, even if there was no specific "we own everything" statement there could be pretty much standard statement of "we'll retain data as required for the delivery and improvement of the service" which is essentially the same thing.

So, any company that allows it's employees to use chatgpt for work stuff (writing emails with company secrets etc) is definitely not engaging in "secure and private" use.

Unless there is very clear data ownership, for example, customer owns the data going in and going out. I can't see how it can be any different. The problem (not at all)OpenAI has in delivery such service is that in contrast to open source models I'm told there is a lot of "secret sauce" around the model(not just the model itself). Specifically input/output processing, result scoring and so on.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#246

Earlier quoted context omitted.

I mean, I explicitly have the LLM summarize content that's about to fall out of the window as a form of pre-emptive token compression. I'd expect maybe they do something similar.

I feel like we're describing short vs long term memory.

That’s exactly what it is. It’s just it turns out you need very good generalized or focused simple reasoning to do accurate compression or else the abstraction and movement to long term memory doesn’t include the most important content. Or worse distracting details.

I’ve been working on short and long term memory windows at allofus.ai for about 6 months now and it’s way more complex than I had originally thought it would be.

Even if you can magically extend the content window, the added data confuses and waters down the reasoning of the LLM. You must do layered abstraction and compression with goal based memory for it to continue to reason without distraction of irrelevant data.

It’s an amazing realization, almost like a proof that memory is a kind of layered reasoning compression system. Intelligence of any kind can’t understand everything forever. It must cull the irrelevant details, process the remains and reason on a vector that arises from them.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#247

Earlier quoted context omitted.

> The closed source players have the best researchers Is that definitely why? GPT 3.5 and GPT 4 are far larger than 70B, right? So if a 70B, local model like LLaMA can even remotely rival them, would that not suggest that LLaMA is fundamentally a better model? For example, would a LLaMA model with even half of GPT 4's parameters be projected to outperform it? Is that how it works? [I'm not super familiar with LLM tec…

LLamA 2 at 70B is, let’s say pessimistically 70% as good as GPT3.5. This makes me think that OpenAI is lying about their parameter count, are vastly less efficient than LLaMA, or, the lager model sizes have diminishing returns. Either way, your point is a good one. Something doesn’t add up.

IMO Llama2 really isn’t close to 3.5. It still has regular mode collapse (or whatever you call getting repetitive and nonsensical responses after a while), it has very poor mathematical/logical reasoning and is not good at following multi-part instructions.

It just sounds like 3.5/4 because it was trained on it.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#248
post #162

Earlier quoted context omitted.

Llama 2 might by some measures be close to GPT 3.5, but it’s nowhere near GPT 4, nor Anthropic Claude 2 or Cohere’s model. The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. My sense is that the foundation model companies have an edge for now and will probably stay a few steps ahead of the open source realm simply for e…

You seriously underestimate just how much _not_ having to tune your llm for SF sensibilities benefits performance. As an example from the last six months: people on tor are producing better than state of the art stable diffusion because they want porn without limitations. I haven't had the time to look at llm's but the degenerates who enjoy that sort of thing have said they can get the Llama2 model to role play their…

What do you mean by "tune for SF" ?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#249

I'm a little confused by how the relationship works between OpenAI and Microsoft. It is possible for anyone to register for an OpenAI account and use their APIs. Within Azure the same thing is much more difficult as it is necessary to be a "real" business in order to use it. I maintain an open source OpenAI library and would like to add support for Azure but can't because of this restriction. Why can't I just use my…

Microsoft owns enough of OpenAI that their endgame goal of putting GPT like features into Azure and Office365 for enterprise customers is what we’re likely to see happen.

OpenAI will likely target private consumers while Microsoft focuses on enterprise. I can use my own organisation as an example. We’re an investment bank that does green energy within the EU. We would absolutely use GPT if it was legal, but it isn’t, and it likely never will be considering their finance model is partly to steal as much data as they can. Even if it’s not so polite to say that. This is where Microsoft comes into the picture. In non-tech enterprise you’re buying Microsoft products because everyone wants windows, outlook and office. We can wish it wasn’t like that, but where is the realistic alternative? I’m not anti Microsoft by the way, in all my decades in the enterprise business they’ve easily been the best and most consistent business partner for any IT. When Amazon saw how much money there was on the operations side of EU enterprise they quickly caught up, but Amazon doesn’t sell a Office365 product. So anyway, once you have Office365, you’re also likely to use Teams as your communications platform (which is why there is an anti-trust case against it), Sharepoint as your document platform, and, well, Azure as your cloud platform. Except you might use AWS because Amazon is also great. In some ways they are even more compliant with EU legislation than Microsoft.

But if Microsoft can throw GPT products into Azure the same way they put Teams and Sharepoint into Office365… well, then where is their competition? And having GPT features within Office365 will only further their advantage on the office platform. I mean, there are companies which won’t use Outlook, but there won’t be when ChatGPT writes your e-mails.

So this isn’t necessarily for you. It’s just part of Microsoft’s over all strategy for total IT domination in Enterprise. I mean, we’re going into RPA (robot process automation) a journey I went through in another Enterprise organisation a few years back. Back then you had to consider what go buy, would it be BluePrism, UIPath, automation anywhere, something else? Today there is no competition to Microsoft’s PowerAutomate if you’re already a Microsoft customer. It’s literally $500 a month vs $50k a month… I mean… that’s the future for GPT on Azure.

It’s probably necessary too. Their prices have made a lot of organisations look outside of Azure. Toward places like Hetzner or even self-hosting, but if Azure comes with GPT… well then.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#250

Private and secure? I thought the main issue with privacy and security of (not at all)OpenAI models is that by using their products you agree for them to retain all the data you send and receive from the models forever for whatever they choose to use it for. Or is this just a thing for free use? If you pay, do you get a Ts&Cs that don't contain any wording like this? Still, even if there was no specific "we own every…

I'm pretty sure the point of this version is not to export data hence the name
Post reply on HN