Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

github.com

Post reply on HN
Print view View original

121–130 of 349 posts

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#121

Post by extr »

BoorishBears wrote: post #119
extr wrote: post #101

Earlier quoted context omitted.

This is effectively my question. I assume there is some magic going on. But how many engineering hours worth of magic, approximately? There is a lot of speculation around GPT-4 being MoE and whatnot. But very little speculation about the magic of the ChatGPT front end specifically that makes it feel so fluid.

That's mostly because there's very little value in deep speculation there. It's not particularly more fluid than anything you couldn't whip up yourself (and the repo linked proves that) but there's also not much value in trying to compete with ChatGPT's frontend. For most products ChatGPT's frontend is the minimal level of acceptable performance that you need to beat, not an maximal one really worth exploring.

What front end is better than ChatGPT? Is the OP implementation doing running summarization or in-convo embedding lookup?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#122

Post by BoorishBears »

alpinemeadow wrote: post #116

We have this at IKEA for a while now. Not impressed, but funny to read the hallucinations.

I'd expect a company like IKEA to have the expertise to create interfaces specific to their workflows so hallucinations aren't an issue.

Imo if you're making an open ended chat interface for a business, you're doing it wrong.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#123

Post by cosbgn »

toomuchtodo wrote: post #84
ajhai wrote: post #77

A lot of companies are already using projects like chatbot-ui with Azure's OpenAI for similar local deployments. Given this is as close to local ChatGPT as any other project can get, this is a huge deal for all those enterprises looking to maintain control over their data. Shameless plug: Given the sensitivity of the data involved, we believe most companies prefer locally installed solutions to cloud based ones at le…

Can you plug this together with tools like api2ai to create natural language defined workflow automations that interact with external APIs?

You can use unfetch.com to make API calls via LLMs and build automations. (I'm building it)

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#124

Post by saliagato »

paxys wrote: post #113
Xenoamorphous wrote: post #75

Earlier quoted context omitted.

The repo contains the UI code, not the model or anything else around ChatGPT, it just uses Azure’s ChatGPT API which doesn’t share data with OpenAI.

So basically – what you really need to do to run Azure ChatGPT is go and click some buttons in the Azure portal. This repo is a sample UI that you could possibly use to talk to that instance, but really you will probably always build your own or embed it directly into your products. So calling the repo "azurechatgpt" is misleading. It should really be "sample-chatgpt-api-frontend" or something of that sort.

Yes exactly

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#127

Post by padolsey »

I'm confused. If this is just a front-end for the OpenAI API then how does it remove the data privacy concern? Your data still ends up with Azure/OpenAI, right? It doesn't stay localized to your instance; it's not your GPU running the transformations. You have no way of knowing whether your data is being used to train models. If customer data is sensitive, I'm pretty sure running a 70B llama (or similar) on a bunch of A100s is the only way?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#128

Post by [deleted] »

Xenoamorphous wrote: post #75
paxys wrote: post #30

Would it be too much to mention somewhere in the README what this repo actually contains? Just docs? Deployment files? Some application (which does..something)? The model itself?

The repo contains the UI code, not the model or anything else around ChatGPT, it just uses Azure’s ChatGPT API which doesn’t share data with OpenAI.

[deleted]

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#129

Post by dbish »

padolsey wrote: post #127

I'm confused. If this is just a front-end for the OpenAI API then how does it remove the data privacy concern? Your data still ends up with Azure/OpenAI, right? It doesn't stay localized to your instance; it's not your GPU running the transformations. You have no way of knowing whether your data is being used to train models. If customer data is sensitive, I'm pretty sure running a 70B llama (or similar) on a bunch o…

Azure is hosting and operating the service themselves rather then for OpenAI, with all the security requirements that come with that. I assume this comes with different data and access restrictions as well and ability to run in secured instances (and nothing sent to OpenAI the company).

Most companies use cloud already for their data, processing, etc. and aren’t running anything major locally, let alone ML models, this is putting trust in the cloud they already use.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#130

Post by laurels-marts »

paxys wrote: post #113
Xenoamorphous wrote: post #75

Earlier quoted context omitted.

The repo contains the UI code, not the model or anything else around ChatGPT, it just uses Azure’s ChatGPT API which doesn’t share data with OpenAI.

So basically – what you really need to do to run Azure ChatGPT is go and click some buttons in the Azure portal. This repo is a sample UI that you could possibly use to talk to that instance, but really you will probably always build your own or embed it directly into your products. So calling the repo "azurechatgpt" is misleading. It should really be "sample-chatgpt-api-frontend" or something of that sort.

Correct. If offers a front-end scaffolding for your enterprise ChatGPT app. Uses Next/NextAuth/Tailwind etc. for deployment on Azure App Service that hooks into Azure Cosmos DB and Azure OpenAI (the actual model).
Post reply on HN