Live data from Hacker News

Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

github.com

71–80 of 349 posts

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#71

This is potentially a huge deal. Companies are concerned using ChatGPT might violate data privacy policies if someone puts in user data or invalidate trade secrets protections if someone uploads sections of code. I suspect many companies have been waiting for an enterprise version.

I imagine most companies serious about this created their own wrappers around the API or contracted it out, likely using private Azure GPUs.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#72

Our company is pushing everyone to use a similar offering. Most of the company is doing low value work … still using excels even though we have a custom ERP. Now seeing people who couldn’t write a coherent email before write 3 page emails. The illusion of being productive by doing more work even though it has zero impact on the bottom line. It’s insane how inefficient organisations are. No doubt we’ll have some KPI s…

[deleted]

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#74
we wrote a blog post about why companies do this here: https://www.lamini.ai/blog/specialize-llms-to-private-data-d...

Here are a few:

Data privacy

Ownership of IP

Control over ops

The table in the blog lists the top 10 reasons why companies do this based on about 50 customer interviews.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#75
post #30

Would it be too much to mention somewhere in the README what this repo actually contains? Just docs? Deployment files? Some application (which does..something)? The model itself?

The repo contains the UI code, not the model or anything else around ChatGPT, it just uses Azure’s ChatGPT API which doesn’t share data with OpenAI.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#76

Our company is pushing everyone to use a similar offering. Most of the company is doing low value work … still using excels even though we have a custom ERP. Now seeing people who couldn’t write a coherent email before write 3 page emails. The illusion of being productive by doing more work even though it has zero impact on the bottom line. It’s insane how inefficient organisations are. No doubt we’ll have some KPI s…

If anything it's less productive because people have to parse all that nonsense. I was gobsmacked to hear a friend say that their work guidance is to use ChatGPT to write letters to external clients for example. I know for sure I'd be insulted if someone sent me paragraphs of text to read created from a sentence long prompt. I'd rather have the prompt, my time is valuable as well.

You'll just have people reversing it into a summary on the other end, kind of like a "text" chat where both sides are using text-to-speech and speech-to-text instead of having a phone call.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#77
A lot of companies are already using projects like chatbot-ui with Azure's OpenAI for similar local deployments. Given this is as close to local ChatGPT as any other project can get, this is a huge deal for all those enterprises looking to maintain control over their data.

Shameless plug: Given the sensitivity of the data involved, we believe most companies prefer locally installed solutions to cloud based ones at least in the initial days. To this end, we just open sourced LLMStack (https://github.com/TryPromptly/LLMStack) that we have been working on for a few months now. LLMStack is a platform to build LLM Apps and chatbots by chaining multiple LLMs and connect to user's data. A quick demo at https://www.youtube.com/watch?v=-JeSavSy7GI. Still early days for the project and there are still a few kinks to iron out but we are very excited for it.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#78

This is potentially a huge deal. Companies are concerned using ChatGPT might violate data privacy policies if someone puts in user data or invalidate trade secrets protections if someone uploads sections of code. I suspect many companies have been waiting for an enterprise version.

I imagine most companies serious about this created their own wrappers around the API or contracted it out, likely using private Azure GPUs.

Most companies are either not tech companies, or do not have the knowledge to manage such a project within reasonable cost bounds.

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#79
One thing I still don't understand is what _is_ the ChatGPT front end exactly? I've used other "conversational" implementations built with the API and they never work quite as well, it's obvious that you run out of context after a few conversation turns. Is ChatGPT doing some embedding lookup inside the conversation thread to make the context feel infinite? I've noticed anecdotally it definitely isn't infinite, but it's pretty good at remembering details from much earlier. Are they using other 1st party tricks to help it as well?

Re: Azure ChatGPT: Private and secure ChatGPT for internal enterprise use

#80

Our company is pushing everyone to use a similar offering. Most of the company is doing low value work … still using excels even though we have a custom ERP. Now seeing people who couldn’t write a coherent email before write 3 page emails. The illusion of being productive by doing more work even though it has zero impact on the bottom line. It’s insane how inefficient organisations are. No doubt we’ll have some KPI s…

I’ve learned that the most important writing skill is to figure out what you’re trying to say — this is a rather important prerequisite to writing well.

Naively asking a chatbot to write for you does not help with this at all.

It would be interesting to try to prompt ChatGPT to ask questions to try to figure out what the user is trying to write and then to write it.

Post reply on HN