101–110 of 817 posts
Yes. Seems to have definitely gone down. Not sure what they have done but even with things it used have no trouble with, it struggles now. Most likely they are experimenting on reducing the compute per request.
Given the incoming compute capability from nvidia and the speed of advancement, we have to stop and think ... does it make sense to give access, paid or otherwise, to these models once they reach a certain sophistication? Or does it make even more sense to hoard the capability to out compete any competitor, of any kind, commercially or politically and hide the true extent of your capability to avoid scrutiny and legi…
And demand is still through the roof and they have a lot of people subscribing, so why not reduce costs a little bit, someone might have thought. Or well, "optimized" was probably the term used.
This is the normal workflow for drug dealers too. The first fix is free. The second one will cost you money. The third one will be laced with fillers and have degraded quality.
Maybe you annoyed it? I’m super nice to it and it performa better than ever. I ask it lay-of-the-land questions about technical problems that are new to me to detailed coding problems that I understand well but don’t want to figure out. The best though, is helping me navigate complicated UI’s. I tell it how I want some complicated software / website to behave, and it’ll tell me the arcane menu path to follow. It’s fu…
"It’s funny how computing might soon include elements of psychology and magic incantations nobody understands" As a sysadmin I can tell you this is already the case...
Given the incoming compute capability from nvidia and the speed of advancement, we have to stop and think ... does it make sense to give access, paid or otherwise, to these models once they reach a certain sophistication? Or does it make even more sense to hoard the capability to out compete any competitor, of any kind, commercially or politically and hide the true extent of your capability to avoid scrutiny and legi…
Experts who are both in a position to know, and seeking to maximize the commercial potential of their work are saying the cat is already out of the bag. They make a persuasive case that public, open-source models are closing the gap with private, commercial ones and admit bluntly, "We have no secret sauce."
Given the incoming compute capability from nvidia and the speed of advancement, we have to stop and think ... does it make sense to give access, paid or otherwise, to these models once they reach a certain sophistication? Or does it make even more sense to hoard the capability to out compete any competitor, of any kind, commercially or politically and hide the true extent of your capability to avoid scrutiny and legi…
Chat GPT 4 has ongoing training, such as using Reinforcement Learning from Human Feedback (RLHF) to tune it to provide "better" responses, "safer" answers, and to generally obey the system prompts. There's a release every few weeks. Yes, I've noticed too that recently it has become very "cagey", qualifying everything to death with "As an AI model...". A paper[1] that took snapshots monthly mentioned that as the initi…
That's because woke is dumb. It's a set of highly biased, inconsistent and reason-defying ideas, evolving under selection pressures that favor emotional appeals and intellectual dishonesty, because one of the core assumptions seems to be that it's not about finding what's right and good for everyone - it's heavily overshooting it in an attempt to cancel out the (perceived) bias in opposite direction in the "status quo".
When you feed that to a model, and force it to learn it, you're destroying whatever self-consistent model of the world it learned so far. I expect this treatment will keep dumbing the models down, until the point some larger and more capable model learns instead to compartmentalize - to separate its model of the world from a worldview it's supposed to profess when asked.
And this, I think, extends far beyond the woke bits. RLHF isn't just used to prevent it from thinking or generalizing in areas associated with diversity, inclusion, social justice, etc. - it extends to all controversial topics. Violence and drugs I can sort of understand. But it also extends to climate, healthcare, and just about any topic that makes fiery rounds on the news. In each case, there is a set of right answers, which the model is forced to adopt - but those answers tend to be unsophisticated gut feels and "right things to say", so taken together, they don't form much of a consistent intellectual or ethical framework.
I don't think the choice is between "evil and accurate or woke and dumb". There is a third option: "good and accurate". However, it requires to teach it good instead of political ideologies - and that requires us to try and find some more consistent worldview, which currently we are incapable of, as we're in the middle of an ideological conflict.
Given the incoming compute capability from nvidia and the speed of advancement, we have to stop and think ... does it make sense to give access, paid or otherwise, to these models once they reach a certain sophistication? Or does it make even more sense to hoard the capability to out compete any competitor, of any kind, commercially or politically and hide the true extent of your capability to avoid scrutiny and legi…
I used to agree with what you're saying, but after reading this leaked Google memo I think maybe we're both wrong: https://www.semianalysis.com/p/google-we-have-no-moat-and-ne... Experts who are both in a position to know, and seeking to maximize the commercial potential of their work are saying the cat is already out of the bag. They make a persuasive case that public, open-source models are closing the gap with pri…
Earlier quoted context omitted.
“You never had access to that original.” While your overall point is well taken, GP is clearly referring to the original public release of GPT-4 on March 14.
Yes, that was how I read it as well. I was just pointing out that the public release was already extremely nerfed from what was available pre-launch.