Earlier quoted context omitted.
> Is nobody in these very rich guys' spheres pushing back on their thought process? Yes, frequently and loudly. When Altman was collecting the award at Cambridge the other year, protesters dropped in on the after-award public talk/Q&A session, and he actively empathised with the protestors . > So far we are multiple years in with much investment and little return, and no obvious large-scale product-market fit, much l…
> protesters dropped in on the after-award public talk I’m going to guess that GP does not consider random protestors to be in Sam Altman’s ‘sphere’ > The product market fit is fantastic. This is true insomuch that you define “product market fit” as “somebody mentioning it in an Indian restaurant in Berlin” > every definition of "very smart" Every definition you say? https://bsky.app/profile/edzitron.com/post/3lclo77…
I occasionally wonder whether one could get better results by allowing later layers to query against the K/V data from earlier layers. This might make inference, and possibly even training, of models that don’t fit in a single node’s memory rather more complicated.