Inflection-2: the next step up
inflection.ai
Inflection-2: the next step up
1–10 of 39 posts
Re: Inflection-2: the next step up
#2First reaction: okay, should we be impressed?
June: flashy ML-perf demo with CoreWeave using 22k H100s
November: actually trained model using 5k H100s
June: we claim Inflection-1 is the best model in its compute class and are preparing a frontier model
November: we beat PaLM 2, which everyone else forgot about long ago anyway
Inflection got a ton of hype with its $1.3B raise (likely not cash but principally GPU compute credits) earlier this year, but now is starting to look like the next victim of inflated expectations.
Re: Inflection-2: the next step up
#3Regular reminder that most open source LLM benchmarks are not very useful (in the sense that they don't represent day to day ai chatbot usage and what users care about). If you haven't looked through the datasets to see what they actually contain, I'd encourage you to do so. [1] I think we're just in a strange suboptimal schelling point of sorts, where people report their scores on those benchmarks because they think other people care about those sort of benchmarks, and therefore those benchmarks are the ones that people expect and care about.
And to recap their statement about it being second most powerful, it's based on MMLU scores, which IMO is a non-useful comparison. (Also, doesn't test against GPT-4-Turbo or Claude-long-2.1)
What they're saying is that Inflection-2 ranks #2 relative to other models including GPT-4, Claude-2, PaLM 2, Grok-1, and Llama 2 70b, specifically on MMLU scores.
This model could be great, but that'll be determined by "do day to day users, both free and paying, prefer it over Claude 2 and GPT-4-Turbo" - not MMLU scores.
[1]: https://huggingface.co/datasets/lukaemon/mmlu/viewer/abstrac...
Re: Inflection-2: the next step up
#4Their byline is "the second most capable LLM in the world today." Ok thanks for the heads up, I'll go use the first most capable LLM while Inflection catches up... Their press release shows this model well behind gpt-4. There's currently no non-beta API. I'm just not sure who this is for.
Re: Inflection-2: the next step up
#5I realize part of why Sam Altman feels so important in OpenAI is because he invited everyone to come along. You can build something pretty close to chat.openai.com on the APIs, and their APIs continue to grow as their product grows. I think that's closely related to his leadership.
This Inflection announcement feels kind of like your neighbor showing you his cool new Jaguar. Can I even take it for a test drive? Well, no... but he'll take me out in a drive in a while, you betcha. "Yeah, cool model you have there," but I'm eyeing the exit.
This is the other side of the "commercial vs mission" argument. Doing commercial activity is the only way to be inclusive. Except open source... but even there it's not a clear call. And writing papers touting your achievements is... kind of narcissistic?
Re: Inflection-2: the next step up
#6Inflection: the no-drama AI company :D
Pi is great for a "personal" chat. I can't wait to use it with the new model.
Re: Inflection-2: the next step up
#7I quite liked Pi to be honest, it was good to use as a journaling tool and to talk through life and issues when i used it. I just feel sometimes it's slightly generic with it's answers, but then all the LLM's have been when i have used them for this purpose.
It felt like a more personal AI, rather than using it to try and code or solve problems, it worked well as kinda a personal guide to talk through life with.
Maybe ChatGPT could be good at this with the right prompt and creating a GPT for it, but I am currently on the waitlist for ChatGPT plus and Pi is free, so a bump in performance might still be welcome.
Re: Inflection-2: the next step up
#8I quite liked Pi to be honest, it was good to use as a journaling tool and to talk through life and issues when i used it. I just feel sometimes it's slightly generic with it's answers, but then all the LLM's have been when i have used them for this purpose. It felt like a more personal AI, rather than using it to try and code or solve problems, it worked well as kinda a personal guide to talk through life with. Mayb…
[flagged]
Re: Inflection-2: the next step up
#9"Our mission at Inflection is to create a personal AI for everyone."
"By messaging Pi, you are agreeing to our Terms of Service and Privacy Policy."
Yeah, no. Any AI that operates in your cloud where I have to agree to your Terms of Service and Privacy Policy is not "personal AI," no matter how much you want me to believe otherwise.
Modern LLMs can do inferencing on my own personal computer. Some of them can even do it on a Raspberry Pi (no pun intended). That's "personal AI."
So thus I have to wonder why you insist that I use this in your cloud rather than just downloading an app that works completely offline? Especially if you're gonna call it "personal AI."
Re: Inflection-2: the next step up
#10Their byline is "the second most capable LLM in the world today." Ok thanks for the heads up, I'll go use the first most capable LLM while Inflection catches up... Their press release shows this model well behind gpt-4. There's currently no non-beta API. I'm just not sure who this is for.
this is for investors. while some companies actually build useful models, some like inflection are still catching up with palm-2, an old, forgotten, measly model.