Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Major outages across ChatGPT and API

status.openai.com

Post reply on HN
Print view View original

531–540 of 617 posts

Re: Major outages across ChatGPT and API

#531

Post by nomel »

throw555chip wrote: post #429
gwd wrote: post #415

Earlier quoted context omitted.

GPT-4 is quite capable of writing function-length sections of code based only on descriptions. Either in a context where you're not sure what the a good approach is (for myself, when writing Javascript for example), or when you know what needs to be done but it's just somewhat tedious. Here's a session from me working on a side project yesterday: https://chat.openai.com/share/a6928c16-1c18-4c08-ae02-82538d... The mos…

Wow, so, you're not a DBE or DBA but are applying indexes across a database without concern because...a computer model spat out that you should?

I am also not those. Is there something wrong with too many indexes, assuming memory isn't a huge constraint?

Re: Major outages across ChatGPT and API

#532

Post by nomel »

redblacktree wrote: post #411
all2 wrote: post #390

I'd be curious to hear about the workflows people have come up with using ChatGPT. I'm still in the realm of "I don't know how to do this" or "I forgot the exact incantation to that" or "is the an X that does Y in framework Z?"

I like to use it for one-off scripts. For example, I downloaded a bunch of bank statements the other day, and they had a format something like, "Statement-May-1-2023.pdf" and I asked GPT for a powershell script to convert that to "2023-05-01-BankName-Statement.pdf" It saved a bunch of manual work on a throwaway script. In the past, I might have done something in Python, since I'm more familiar with it than powershell…

This is what convinced me to pay. I'm completely fine with spending $1/workday if it saves me 10 to 15 minutes per day of bore.

Re: Major outages across ChatGPT and API

#533

Post by enoch2090 »

frabcus wrote: post #27

Phind is pretty good for coding (LLama 2 trained on billions of extra code tokens) and is still up https://www.phind.com/s

Been playing with Phind for a while and my conclusion is: the Phind model works well on those long existing stuff like C++ libraries, but works generally bad on newer stuff, such as composing LCEL chains.

Re: Major outages across ChatGPT and API

#534

Post by enoch2090 »

rolisz wrote: post #84
enoch2090 wrote: post #74

Earlier quoted context omitted.

This reminds us that, what if our databases are maintained using OpenAI's embeddings, and the API suddenly goes down? How do we find alternatives to match the already generated database?

I don't think you can do that easily. If you already have a list of embeddings from a different model, you might be able to generate an alignment somehow, but in general, I wouldn't recommend it.

That's my point, maybe VectorDBs in production should have a fallback mechanism, for the documents inserted,

1. Generate embeddings using services such as OpenAI, which is usually more powerful;

2. Generate backup embeddings using local, more stable models, such as Llama2 embeddings or simply some BERT-family-model (which is more affordable).

When outages comes up you simply switch from one vector space to another. Though possible, model alignments are much harder and more expensive to achieve.

Re: Major outages across ChatGPT and API

#535

Post by nomel »

boh wrote: post #380
obiefernandez wrote: post #2

Dreaming of the day I run my own models and can be entirely responsible for my own outages.

You can do that today. Oobabooga + Hugging Face models.

Having something that executes, and having something that's genuinely useful, are two different things.

For my hand typed use case's, GPT-4 is the only acceptable model that doesn't leave me frustrated and angry at wasting time. For some automated stuff (converting text to json, etc), the local models are fine.

Re: Major outages across ChatGPT and API

#536

Post by financltravsty »

nomel wrote: post #531
throw555chip wrote: post #429

Earlier quoted context omitted.

Wow, so, you're not a DBE or DBA but are applying indexes across a database without concern because...a computer model spat out that you should?

I am also not those. Is there something wrong with too many indexes, assuming memory isn't a huge constraint?

Biggest one is write overhead: indexes have to be updated every time a record is created or an indexed column is updated. This must be done within the same transaction as the create or update, so you're adding an unnecessary overhead to every single one of those ops. Now, if the data is relatively small or the use-case doesn't warrant it, it doesn't matter.

Lesser issues: additional strain on index rebuilding whenever that happens; messing with execution plans and causing the query planner to be inefficient; primary/secondary memory overhead; or if your DB engine uses locks you can run into a myriad of issues there.

I'm all for SWEs learning about databases, as I'm morally opposed to the proliferation of ORMs and the like, but I don't think ChatGPT is the right way to go about things long-term. It's similar to Googling or using StackOverflow: yes, you will find information that is relevant to what interests you at the moment, but it's soon forgotten and does nothing to help build long-term mental models.

Re: Major outages across ChatGPT and API

#537

Post by mmasu »

cainxinth wrote: post #518
wfhBrian wrote: post #453

Earlier quoted context omitted.

People constantly recommend Bard to me, but it returns false and/or misleading info almost every time

Gpt-4 has the lowest hallucination rates: > OpenAI’s technologies had the lowest rate, around 3 percent. Systems from Meta, which owns Facebook and Instagram, hovered around 5 percent. The Claude 2 system offered by Anthropic, an OpenAI rival also based in San Francisco, topped 8 percent. A Google system, Palm chat, had the highest rate at 27 percent. https://www.nytimes.com/2023/11/06/technology/chatbots-hallu...

i just skimmed through the article - it seems that the numbers are quoted from a company called Vectara. Would be interesting to see how they are getting to this estimate

Re: Major outages across ChatGPT and API

#538

Post by dcchambers »

baq wrote: post #407
grishka wrote: post #366

Earlier quoted context omitted.

What is so special about OpenAI's cloud hardware that one can't build themselves a similar server to run AI models of similar size?

Nothing stopping you from buying an H100 and putting it in your desktop. As for me, I’ve got other uses for $45k.

If you wanted to buy one...are you even able to order one? Aren't they massively backordered by big companies? Is there a black marker for H100 cards?

Re: Major outages across ChatGPT and API

#539

Post by pozol »

rushingcreek wrote: post #404
ttcbj wrote: post #368

Earlier quoted context omitted.

I am curious how people are using Phind. I actually had a discussion with Phind itself recently, in which I said that in order to help me, it seems like it would need to ingest my codebase so that it understands what I am talking about. Without knowing my various models, etc, I don't see how it could write anything but the most trivial functions. It responded that, yes, it would need to ingest my codebase, but it cou…

Founder here. We have a VS Code extension that automatically fetches context from your codebase.

Does the plugin use the codebase code in the model itself for training, or does it stay locally? Just curious about the privacy.

Re: Major outages across ChatGPT and API

#540

Post by rushingcreek »

pozol wrote: post #539
rushingcreek wrote: post #404

Earlier quoted context omitted.

Founder here. We have a VS Code extension that automatically fetches context from your codebase.

Does the plugin use the codebase code in the model itself for training, or does it stay locally? Just curious about the privacy.

We won’t use your codebase for training. And there is an option you can toggle that disables data retention entirely.
Post reply on HN