This morning I was reviewing some code that a JR engineer submitted. It had this wild logical conditional with twists and turns, negations, and weird property naming.. o1-preview perfectly evaluated the conditional and determined that, hilariously, it would always evaluate to true. o1 untangled the spaghetti, and, verifying that it was correct was quick and easy. It created a perfect truth table for me to visualize.…
It's surprising that the JR engineer who presumably has access to chat jippity submitted this bad code to begin with. Wouldn't they have had the AI review the code first?
My theory would be that the Jr probably generated the code with o1 in the first place. The buggy, nonsensical crap I've seen that thing generate boggles the mind.
I want to be wildly optimistic too, but I still see no evidence LLMs generate new knowledge. They always hew in-distribution. Please correct me if I’m wrong
They are really helpful to answer concrete questions, basically a replacement of manual web search and filtering, getting just exactly answers I was looking for. For example, one of my dialogue with chatgpt 4o was about boosting plant growth in a fish tank - you certainly can find a lot of web sites about it, but I simply described my aquatic environment and asked for a recipe - and the answers sound and well support…
That's the crucial point, though, you wouldn't have trusted it without checking the source after all. My experience with 4o is that 4 out of 5 answers I check are wildly incorrect or entirely made up, while with the rest, if I copy my prompt 1:1 into Google, I get the same correct answer pretty much verbatim in result #1. So I don't understand how so many people still see this as anything else but a waste of time, an intermediary step before going to an actual, credible source -- a step which in the best case is entirely unnecessary, in the worst case dangerously misleading. (From my non-representative survey among friends, the answer may be that barely anyone checks the validity of answers, and are simply unaware that they rely on answers from a system which, except for some lucky cases, gives them wrong ones.
Yes, he's handwaving in this general area, but no, he's not really relying on the UAT. If you talked to most NN people 2 decades ago and asked about this, they might well answer in terms of the UAT. But nowadays, most people, including here Altman, would answer in terms of practical experience of success in learning a surprisingly diverse array of distributions using a single architecture.
I think that while researchers would agree that the empirical success of deep learning has been remarkable, they would still agree that the language used here -- "an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data)" -- is an overly strong characterization, to the point that it is no longer accurate. A hash function is a good ex…
Certainly, it's an exaggeration / simplification. I don't feel it is really a dishonest one, in context [*]. It feels weird for me to "defend" him here, because in general Altman is a stupendous, egregious, world-leading liar.
[*] Ok, we can differ on that. My feeling is partly because the types of distributions that can't be learned - eg hash functions - are generally the kind of functions we don't really want to learn. Underneath this are deeper questions related to no free lunch and how "nice"/"well-behaved" this universe is.
> Is Sam Altman generally considered good at his job? No credit for popularizing the current generation of AI, kicking off $hundreds of billions in CapEx spends, and for more concrete achievements, leading the fastest company to hit 100m users / $1b ARR?
You call some numbers in an ACHS a concrete achievement? I was thinking more like a dam or a school or something...
The question was if he's good at his job. How does building "like a dam or a school or something" have anything to do with that?
Oh wow! Could you please share what processors are exponentially faster than those of 10 years ago? I'm not seeing any here: https://www.cpubenchmark.net
Macbook Airs have 20 billion+ transistors, compared to 50 million on the Pentium 4 in the early 2000s. Moore's law is about transistor density, not processor speed, which is gated by thermal limits.
Oh wow! Could you please share what processors are exponentially faster than those of 10 years ago? I'm not seeing any here: https://www.cpubenchmark.net
Transistor count has consistently been increasing by about 10% a year over the last decade.