Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

github.com

Post reply on HN
Print view View original

301–310 of 349 posts

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#301

Post by geysersam »

dr_dshiv wrote: post #299
geysersam wrote: post #264

Earlier quoted context omitted.

You're mixing up the language model with the chat bot . The llama2 is a language model. I imagine the language model behind chatgpt is not much different (perhaps it's better, but not by many months AI research time). It likely also suffers from "mode collapse" issues etc. But 3.5 also has a lot of systems around it that detects mode collapse and applies some kind of mitigation, forcing the model to give a more reaso…

So this would be testable by showing that chatGPT makes more mistakes than prompting via API? Or would you consider the API a chatbot, too?

I don't think there's any public interface to the LLM underlying ChatGPT, so the only ones able to test this are openAI engineers.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#303

Post by 3abiton »

jmorgan wrote: post #152

This appears to be a web frontend with authentication for Azure's OpenAI API, which is a great choice if you can't use Chat GPT or its API at work. If you're looking to try the "open" models like Llama 2 (or it's uncensored version Llama 2 Uncensored), check out https://github.com/jmorganca/ollama or some of the lower level runners like llama.cpp (which powers the aforementioned project I'm working on) or Candle, the…

Our company is looking into similar solution

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#304

Post by lmeyerov »

gdiamos wrote: post #89
ajhai wrote: post #77

A lot of companies are already using projects like chatbot-ui with Azure's OpenAI for similar local deployments. Given this is as close to local ChatGPT as any other project can get, this is a huge deal for all those enterprises looking to maintain control over their data. Shameless plug: Given the sensitivity of the data involved, we believe most companies prefer locally installed solutions to cloud based ones at le…

I find it interesting to see how competitive this space got so quickly. How do these stacks differentiate?

With Louie.ai, there is a lot of work on specialization for the job, and I expect the same for others. We help with data analysis, so connecting enterprise & common data sources & DBs, hooking up data tools (GPU visuals, integrated code interpreter, ...), security controls, and the like, which is different from say a ChatGPT for lawyers or a straight up ChatGPT UI clone.

Technically, as soon as the goal is to move beyond just text2gpt2screen, like multistep data wrangling & viz in the middle of a conversation, most tools technically struggle. Query quality also comes up, whether quality of the RAG, the fine tune, prompts, etc: each solves different problems.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#305

Post by Ephil012 »

joenot443 wrote: post #292
nl wrote: post #217

Earlier quoted context omitted.

> The closed source players have the best researchers - they are being paid millions a year with tons of upside - and it’s hard to keep pace with that. Llama2 came out of Meta's AI group. Meta pays researcher salaries competitive with any other group, and their NLP team is one of the top groups in the world. For researchers it is increasingly the most attractive industrial lab because they release the research openly…

There are L5 engineers with 3 YOE making 900k+ at OpenAI right now. Tough to say what they're paying their PhDs, but I'd imagine it's similarly nutty. https://www.levels.fyi/companies/openai/salaries/software-en... FAANG pays exceptionally well (I'd know), but what's being offered at OpenAI is eye-popping, even for SWEs. I think they're trying to dig their moat by absorbing the absolute best of the best.

Most of that is in their equity comp which is quite weird in how it works. So those numbers are highly inflated. The equity is valuable only if you sell it or if OpenAI makes a profit. Selling it might be harder given they're not a public company. On top of that, the profit is capped so there is a limit to how much money can be made from it. So while it's 900k on paper, in reality, it might not be as good as that. https://www.levels.fyi/blog/openai-compensation.html

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#306

Post by tsunamifury »

yowlingcat wrote: post #251
tsunamifury wrote: post #246

Earlier quoted context omitted.

That’s exactly what it is. It’s just it turns out you need very good generalized or focused simple reasoning to do accurate compression or else the abstraction and movement to long term memory doesn’t include the most important content. Or worse distracting details. I’ve been working on short and long term memory windows at allofus.ai for about 6 months now and it’s way more complex than I had originally thought it w…

Is it unfair to consider this some kind of correlate to the Nyquist theorem that makes me skeptical of even the theoretical possibility of AGI claims?

I consider GPT4 AGI, so I'm probably not the one to ask this too. It reasons, it understands sophisticated topics, it can be given a purpose and pursue it, it can communicate with humans, and it can perform a reasonable task considering its modalities.

I don't really know what any sort of "big leap" beyond this people are expecting, incremental performance for sure. But what else?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#307

Post by baobabKoodaa »

stavros wrote: post #109
baobabKoodaa wrote: post #102

Earlier quoted context omitted.

> it's the SOTA for chat(prove me wrong) GPT-4 beats ChatGPT on all benchmarks. You can easily google these.

The distinction between GPT-4 and ChatGPT is blurry, as ChatGPT is a chat frontend for a GPT model, and you can use GPT-4 with ChatGPT. The parent probably means ChatGPT with GPT-4.

Typically when people say "ChatGPT" without specifying which specific model they refer to, they refer to gpt-3.5-turbo (in case of API - or in case of the web ui, they mean whatever model is its current web ui equivalent). But now OP says they meant GPT-4, so, sure.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#308

Post by TrapLord_Rhodo »

lmeyerov wrote: post #304
gdiamos wrote: post #89

Earlier quoted context omitted.

I find it interesting to see how competitive this space got so quickly. How do these stacks differentiate?

With Louie.ai, there is a lot of work on specialization for the job, and I expect the same for others. We help with data analysis, so connecting enterprise & common data sources & DBs, hooking up data tools (GPU visuals, integrated code interpreter, ...), security controls, and the like, which is different from say a ChatGPT for lawyers or a straight up ChatGPT UI clone. Technically, as soon as the goal is to move be…

I see this as more of a 'Migration problem'. Why is this offered as a SaaS as opposed to a consulting service?

The code to organize and vectorize the documentation, endpoints and run it through a variety of models and injection prompting like two shots, etc. are going to be highly customized. The 'Base-code' there, is not exactly trivial, but anyone reading all the llama index docs can do it.

Then it's just run of the mil, analyst level integration that you provide to the client on a T&M, or fixed price costs.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#309

Post by robga »

BoorishBears wrote: post #276
robga wrote: post #262

Earlier quoted context omitted.

I agree with your points. Having said that, Microsoft removed my Azure OpenAI GPT-4 access last week without warning. I was not breaking any TOS. Oh well, pointed back at OpenAi.

Can you expand on this because that's pretty alarming... What kind of volume were you doing and did you use the API for anything other than your listed use case when applying?

6 x 1000 token calls per day, for a news bot (listed use case at application).

I think what happened is the azure subscription was converted from a (multi year) promotional subsidy/discount to a full pay as you go subscription. No change to sub id. Payment methods OK. Everything else continued working, but openai gpt-4 access stopped the next day.

I’d rather use the Azure version because they promise 12-month sunsets vs OpenAI 6-month sunsets for model versions.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#310

Post by BoorishBears »

shubb wrote: post #184
BoorishBears wrote: post #117

Earlier quoted context omitted.

I'm not sure you realize how proprietary LLMs are being built on. No one is doing secret math in the backend people are building on. The OpenAI API allows you to call functions now, but even that is just a formalized way of passing tokens into the "raw LLM". All the features in the comment you replied to only apply to the web interface , and here you're being given an open interface you can introspect.

Thank you for pointing that out - I had assumed that things were not how they are. Although performance has varied over time https://arxiv.org/pdf/2307.09009.pdf I also notice that the API allows you to use a frozen version of the model which avoids the worries I mentioned.

That was a pretty deeply flawed paper, one of the largest drops recorded was simple parsing errors in their testing:

https://www.aisnakeoil.com/p/is-gpt-4-getting-worse-over-tim...

Overall evals and pinning against checkpoints are how you avoid those worries, but in general, if you solve a problem robustly, it's going to be rare for changes in the LLM to suddenly break what you're doing. Investing in handling a wide range of inputs gracefully also pays off on handling changes to the underlying model.

Post reply on HN