Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

github.com

Post reply on HN
Print view View original

271–280 of 349 posts

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#271

Post by nl »

Maxion wrote: post #241
nl wrote: post #218

Earlier quoted context omitted.

If you read the Llama2 paper it is very clear that small amounts of data (thousands of records) make vast difference at the instruction turning stage. From the Llama2 paper: > Quality Is All You Need. > Third-party SFT data is available from many different sources, but we found that many of these have insufficient diversity and quality — in particular for aligning LLMs towards dialogue-style instructions. As a result…

This quote is quite funny taken out of context like this. Top AI researchers find that garbage in === garbage out.

I'm puzzled. Why do you think it's taken out of context?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#272

Post by vorticalbox »

dr_dshiv wrote: post #247
oceanplexian wrote: post #215

Earlier quoted context omitted.

LLamA 2 at 70B is, let’s say pessimistically 70% as good as GPT3.5. This makes me think that OpenAI is lying about their parameter count, are vastly less efficient than LLaMA, or, the lager model sizes have diminishing returns. Either way, your point is a good one. Something doesn’t add up.

IMO Llama2 really isn’t close to 3.5. It still has regular mode collapse (or whatever you call getting repetitive and nonsensical responses after a while), it has very poor mathematical/logical reasoning and is not good at following multi-part instructions. It just sounds like 3.5/4 because it was trained on it.

This is what presence_penalty and frequency_penalty are for.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#273

Post by nl »

dr_dshiv wrote: post #247
oceanplexian wrote: post #215

Earlier quoted context omitted.

LLamA 2 at 70B is, let’s say pessimistically 70% as good as GPT3.5. This makes me think that OpenAI is lying about their parameter count, are vastly less efficient than LLaMA, or, the lager model sizes have diminishing returns. Either way, your point is a good one. Something doesn’t add up.

IMO Llama2 really isn’t close to 3.5. It still has regular mode collapse (or whatever you call getting repetitive and nonsensical responses after a while), it has very poor mathematical/logical reasoning and is not good at following multi-part instructions. It just sounds like 3.5/4 because it was trained on it.

Llama 2 wasn't trained on ChatGPT/GPT4. I think maybe you are thinking of the Vicuna models?

https://lmsys.org/blog/2023-03-30-vicuna/

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#274

Post by mattlutze »

oceanplexian wrote: post #215
xcdzvyn wrote: post #197

Earlier quoted context omitted.

> The closed source players have the best researchers Is that definitely why? GPT 3.5 and GPT 4 are far larger than 70B, right? So if a 70B, local model like LLaMA can even remotely rival them, would that not suggest that LLaMA is fundamentally a better model? For example, would a LLaMA model with even half of GPT 4's parameters be projected to outperform it? Is that how it works? [I'm not super familiar with LLM tec…

LLamA 2 at 70B is, let’s say pessimistically 70% as good as GPT3.5. This makes me think that OpenAI is lying about their parameter count, are vastly less efficient than LLaMA, or, the lager model sizes have diminishing returns. Either way, your point is a good one. Something doesn’t add up.

We just don't have the information to make judgements, much less leaping to "they must be lying."

There's a few public numbers from a handful of foundation models as to performance vs parameter count vs architecture generation. Not being able to compare in detail the architecture of the various closed models nor being more rigorous on training with progressively sized parameter sets, the conclusion at the moment is a general feeling or conjecture.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#275

Post by longnguyen »

Interesting. One of my most requested feature for my small native apps[0][1] was to support Azure OpenAI service.

Apparently, many organizations have their own Azure OpenAI deployment and won’t let their employees use the public OpenAI service.

My understanding is that Azure makes sure all network traffic is isolated to their network so they have more controls over how their organization use ChatGPT.

I created a super simple step-by-step guide on how to obtain an Azure OpenAI endpoint & key here:

https://pdfpals.com/help/how-to-generate-azure-openai-api-ke...

Hope it would be useful to someone just getting started with Azure.

[0]: https://boltai.com

[1]: https://pdfpals.com

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#276

Post by BoorishBears »

robga wrote: post #262
pietz wrote: post #253

Earlier quoted context omitted.

The Azure SLAs state that neither the chats are stored nor used for training in any way. They are private and protected in the same way all the other sensitive data is stored on Azure. On top, you might argue that Microsoft and Azure are easier to trust than a still rather new AI startup.

I agree with your points. Having said that, Microsoft removed my Azure OpenAI GPT-4 access last week without warning. I was not breaking any TOS. Oh well, pointed back at OpenAi.

Can you expand on this because that's pretty alarming...

What kind of volume were you doing and did you use the API for anything other than your listed use case when applying?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#277

Post by [deleted] »

Roark66 wrote: post #244

Private and secure? I thought the main issue with privacy and security of (not at all)OpenAI models is that by using their products you agree for them to retain all the data you send and receive from the models forever for whatever they choose to use it for. Or is this just a thing for free use? If you pay, do you get a Ts&Cs that don't contain any wording like this? Still, even if there was no specific "we own every…

[deleted]

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#278

Post by rightbyte »

BoorishBears wrote: post #117
shubb wrote: post #85

Earlier quoted context omitted.

This is one of the things that make me uncomfortable about proprietary llm. They get task performance by doing a lot more than just feeding a prompt straight to an llm, and then we performance compare them to raw local options. The problem is, as this secret sauce changes, your use case performance is also going to vary in ways that are impossible for you to fix. What if it can do math this month and next month the h…

I'm not sure you realize how proprietary LLMs are being built on. No one is doing secret math in the backend people are building on. The OpenAI API allows you to call functions now, but even that is just a formalized way of passing tokens into the "raw LLM". All the features in the comment you replied to only apply to the web interface , and here you're being given an open interface you can introspect.

> No one is doing secret math in the backend people are building on.

How do you know that? With SaaS you are at the mercy of the vendor.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#279

Post by chaosbolt »

ttul wrote: post #162
jmorgan wrote: post #152

This appears to be a web frontend with authentication for Azure's OpenAI API, which is a great choice if you can't use Chat GPT or its API at work. If you're looking to try the "open" models like Llama 2 (or it's uncensored version Llama 2 Uncensored), check out https://github.com/jmorganca/ollama or some of the lower level runners like llama.cpp (which powers the aforementioned project I'm working on) or Candle, the…

Llama 2 might by some measures be close to GPT 3.5, but it’s nowhere near GPT 4, nor Anthropic Claude 2 or Cohere’s model. The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. My sense is that the foundation model companies have an edge for now and will probably stay a few steps ahead of the open source realm simply for e…

>but it’s nowhere near GPT 4

It will be if openai keeps dumbing down GPT 4, no proof they're doing it but there is no way it's as good as it was at launch, or maybe I just got used to it and now notice the mistakes more.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#280

Post by wkat4242 »

xcdzvyn wrote: post #197
ttul wrote: post #162

Earlier quoted context omitted.

Llama 2 might by some measures be close to GPT 3.5, but it’s nowhere near GPT 4, nor Anthropic Claude 2 or Cohere’s model. The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. My sense is that the foundation model companies have an edge for now and will probably stay a few steps ahead of the open source realm simply for e…

> The closed source players have the best researchers Is that definitely why? GPT 3.5 and GPT 4 are far larger than 70B, right? So if a 70B, local model like LLaMA can even remotely rival them, would that not suggest that LLaMA is fundamentally a better model? For example, would a LLaMA model with even half of GPT 4's parameters be projected to outperform it? Is that how it works? [I'm not super familiar with LLM tec…

Yeah I've been wondering about this too. Word on the street is that GPT4 is several times the size of GPT3.5. Yet I don't feel it's several times as good for sure.

Apparently there's a diminishing returns effect on ever enlarging the model.

Post reply on HN