Live data from Hacker News

Claude 2.1

anthropic.com

271–280 of 339 posts

Re: Claude 2.1

#271
post #48

That 200k context needs some proper testing. GPT-4-Turbo advertises 128k but the quality of output there goes down significantly after ~32k tokens.

Read the article, it's addressed with charts.

I did but I want more independent testing than just QA performance by position.

Re: Claude 2.1

#273

Earlier quoted context omitted.

Howdy, CISO of Anthropic here. I'm not sure what happened in your case but please reach out to support@ and mention my name; we'll respond ASAP.

I am a subscriber, and personally I think it provides results closer to what I am looking for than gpt4.

That’s hard to believe but I’m open to the possibility.

Cam you share a few examples that might demonstrate this?

Re: Claude 2.1

#274
post #255

Earlier quoted context omitted.

The difference is UX: Are you going to have your user work around poor prompting by giving examples with every request? Instead of a UI that's "Describe what you want" you're going to have "Describe what you want and give me some examples because I can't guarantee reliable output otherwise"? Part of LLMs becoming more than toy apps is the former winning out over the latter. Using techniques like chain of thought with…

> Are you going to have your user What fucking user, man? Is it not painfully clear I never spoke in the context of deploying applications? Your issues with this level of prefilling in the context of deployed apps ARE valid but I have no interest in discussing that specific use case and you really should have realized your arguments were context dependent and not actual rebuttals to what I claimed at the start severa…

I thought we were done when I demonstrated GPT 4 can continue a completion contrary to your belief, but here you are throwing a tantrum several comments later.

Re: Claude 2.1

#275

Earlier quoted context omitted.

> The only sensible model of "alignment" is "model is aligned to the user", We have already seen that users can become emotionally attached to chat bots. Now imagine if the ToS is "do whatever you want". Automated cat fishing, fully automated girlfriend scams. How about online chat rooms for gambling where half the "users" chatting are actually AI bots slowly convincing people to spend even more money? Take any onlin…

> chatbots encouraging the humans to spend more money ... LLMs absolutely need some restrictions on their use. No, I can honestly say that I do not lose any sleep over this, and I think it's pretty weird that you do. Humans have been fending off human advertisers and scammers since the dawn of the species. We're better at it than you account for.

In 2022, reported consumer losses to fraud totaled $8.8 billion — a 30 percent increase from 2021, according to the most recent data from the Federal Trade Commission. The biggest losses were to investment scams, including cryptocurrency schemes, which cost people more than $3.8 billion, double the amount in 2021.

https://www.nbcnews.com/business/consumer/people-are-losing-...

The data says we are not that good and getting 30% worse every year.

Re: Claude 2.1

#276
post #19

For coding it is still 10x worse than gpt4. I asked it to write a simple database sync function and it gives me tons of pseudocode like `//sync object with best practices`. When I ask it to give me real code it forgets tons of key aspects.

I find all of them, gpt4 or not, just suck, plain and simple. They are only good for only the most trivial stuff, but any time the complexity rises even a little bit they all start hallucinate wildly and it becomes very clear they're nothing more than just word salad generators.

I have built large scale distributed gpu (96gpus per job) dnn systems and worked on very advanced code bases.

GPT4 massively sped up my ability to create this.

It is a tool and it takes a lot of time to master it. Took me around 3-6 months of every day use to actually figure out how. You need to go back and try to learn it properly, it's easily 3-5x my work output.

Re: Claude 2.1

#277
post #19

For coding it is still 10x worse than gpt4. I asked it to write a simple database sync function and it gives me tons of pseudocode like `//sync object with best practices`. When I ask it to give me real code it forgets tons of key aspects.

Yeah but to be honest been a pain last days to get gpt 4 to write full pieces of code for more the 10-15 lines. Have to re-ask many times and at some point it forgets my initial specifications.

You should read how the infrastructure of gpt works. In peak times you response quality will drop. Microsoft has a few whitepapers on it.

Ideal output is when nobody elese is using the tool.

Re: Claude 2.1

#278

Earlier quoted context omitted.

> chatbots encouraging the humans to spend more money ... LLMs absolutely need some restrictions on their use. No, I can honestly say that I do not lose any sleep over this, and I think it's pretty weird that you do. Humans have been fending off human advertisers and scammers since the dawn of the species. We're better at it than you account for.

In 2022, reported consumer losses to fraud totaled $8.8 billion — a 30 percent increase from 2021, according to the most recent data from the Federal Trade Commission. The biggest losses were to investment scams, including cryptocurrency schemes, which cost people more than $3.8 billion, double the amount in 2021. https://www.nbcnews.com/business/consumer/people-are-losing-... The data says we are not that good and g…

Furthermore "If it were measured as a country, then cybercrime — which is predicted to inflict damages totaling $6 trillion USD globally in 2021 — would be the world’s third-largest economy after the U.S. and China."

https://cybersecurityventures.com/hackerpocalypse-cybercrime...

Re: Claude 2.1

#279

Earlier quoted context omitted.

I hear a lot of complaints about refusals but rarely any examples of said refusals, likely because they are embarrassing. Is it fair to assume that I won't get refusals for code generation and RAG on documentation?

> I hear a lot of complaints about refusals but rarely any examples of said refusals, likely because they are embarrassing. At least circa 8 months ago on ChatGPT (an aeon ago, I recognize), I could readily get it to make gendered jokes about men but would get a refusal when asking for gendered jokes about women. I think things have "improved" in that time, meaning a more equal distribution of verboten topics, but my…

Oh for sure. We're in the golden age of LLMs before the enshittification commences and we don't appreciate that enough

Re: Claude 2.1

#280

Earlier quoted context omitted.

Earlier in the year I had ChatGPT 4 write a large, complicated C program. It did so remarkably well, and most of the code worked without further tweaking. Today I have the same experience. The thing fills in placeholder comments to skip over more difficult regions of the code, and routinely forgets what we were doing. Aside all the recent OpenAI drama, I've been displeased as a paying customer that their products rou…

Could the (perceived) drop in quality be due to ChatGPT switching from GPT-4 to GPT-4-turbo?

Im not really sure what chatgpt+ is serving me. There was a moment it was suddenly blazing fast, that was around the time turbo came out. Off late, it's been either super slow or super fast randomly.
Post reply on HN