Sure! Here’s a description: https://www.empirical.health/blog/wearable-foundation-model-...
This is very cool work! I have a quick follow-up: in the biomarker prediction task, what horizon (ie. how far into the future) did you set for the predictions? Prediction is hard beyond an hour, so it'd be impressive if your model handles that.
The prediction task is set up as predicting the next measured biomarkers based on a week of wearable data. So it's not necessarily predicting into the future, but predicting dataset Y given dataset X.
The specific biomarkers being predicted are the ones most relevant to heart health, like cholesterol or HbA1c. These tend to be more stable from hour to hour -- they may vary on a timescale of weeks as you modify your diet or take medications.
Regardless of your opinion of Yann or his views on auto regressive models being "sufficient" for what most would describe as AGI or ASI, this is probably a good thing for Europe. We need more well capitalized labs that aren't US or China centric and while I do like Mistral, they just haven't been keeping up on the frontier of model performance and seem like they've sort of pivoted into being integration specialists a…
LeCun's technical approach with AMI will likely be based on JEPA, which is also a very different approach than most US-based or Chinese AI labs are taking. If you're looking to learn about JEPA, LeCun's vision document "A Path Towards Autonomous Machine Intelligence" is long but sketches out a very comprehensive vision of AI research: https://openreview.net/pdf?id=BZ5a1r-kVsf Training JEPA models within reach, even f…
I've been working to understand the potential uses for JEPA. Outside of video, has anyone made a list of any type (geared towards dummies like me)?
Justifiable. There are a lot more degrees of freedom in world models. LLMs are fundamentally capped because they only learn from static text -- human communications about the world -- rather than from the world itself, which is why they can remix existing ideas but find it all but impossible to produce genuinely novel discoveries or inventions. A well-funded and well-run startup building physical world models (ground…
Honestly, how do people who know so little have this much confidence to post here?
I have a pet peeve with the concept of "a genuinely novel discovery or invention", what do you imagine this to be? Can you point me towards a discovery or invention that was "genuinely novel", ever? I don't think it makes sense conceptually unless you're literally referring to discovering new physical things like elements or something. Humans are remixers of ideas. That's all we do all the time. Our thoughts and acti…
Suno is transformer-based; in a way it's a heavily modified LLM. You can't get Suno to do anything that's not in its training data. It is physically incapable of inventing a new musical genre. No matter how detailed the instructions you give it, and even if you cheat and provide it with actual MP3 examples of what you want it to create, it is impossible. The same goes for LLMs and invention generally, which is why th…
I have a pet peeve with the concept of "a genuinely novel discovery or invention", what do you imagine this to be? Can you point me towards a discovery or invention that was "genuinely novel", ever? I don't think it makes sense conceptually unless you're literally referring to discovering new physical things like elements or something. Humans are remixers of ideas. That's all we do all the time. Our thoughts and acti…
Genuinely novel discovery or invention? Einstein’s theory of relativity springs to mind, which is deeply counter-intuitive and relies on the interaction of forces unknowable to our basic Newtonian senses. There’s an argument that it’s all turtles (someone told him about universes, he read about gravity, etc), but there are novel maths and novel types of math that arise around and for such theories which would indicat…
Regardless of your opinion of Yann or his views on auto regressive models being "sufficient" for what most would describe as AGI or ASI, this is probably a good thing for Europe. We need more well capitalized labs that aren't US or China centric and while I do like Mistral, they just haven't been keeping up on the frontier of model performance and seem like they've sort of pivoted into being integration specialists a…
LeCun's technical approach with AMI will likely be based on JEPA, which is also a very different approach than most US-based or Chinese AI labs are taking. If you're looking to learn about JEPA, LeCun's vision document "A Path Towards Autonomous Machine Intelligence" is long but sketches out a very comprehensive vision of AI research: https://openreview.net/pdf?id=BZ5a1r-kVsf Training JEPA models within reach, even f…
Appreciate your work! Healthcare is a regulated industry. Everything (Research, proposals, FDA submissions, Compliance docs, Accreditation Standards, etc.) is documented and follows a process, which means there's a lot of thesis. You can't sneak in anything unverified or unreliable. Why does healthcare need a JEPA\World model?
If, for even 1s, they get in a position which is threatening, in any way, Big Tech AI (mostly US based if not all), they will be raided by international finance to be dismantled and poached hardcore with some massive US "investment funds" (which looks more and more as "weaponized" international finance!!). Only china is very immune to international finance. Those funds have tens of thousands of billions of $, basical…
I don't see a world where they become threatening and the employees don't become rich from investors flooding in.
> Whenever I see claims about AGI being reachable through large language models, it reminds me of the miasma theory of disease. Whenever I see people think the model architecture matters much, I think they have a magical view of AI. Progress comes from high quality data, the models are good as they are now. Of course you can still improve the models, but you get much more upside from data, or even better - from inter…
> Of course you can still improve the models, but you get much more upside from data, or even better - from interactive environments. I'm on the contrary believe that the hunt for better data is an attempt to climb the local hill and be stuck there without reaching the global maximum. Interactive environments are good, they can help, but it is just one of possible ways to learn about causality. Is it the best way? I…
Are you asking how many books a large language model would need to read to learn a new language if it was only trained on a different language? probably just 1 (the dictionary)
"and we didn't see anything" is not justified at all. Meta absolutely has (or at least had) a word class industry AI lab and has published a ton of great work and open source models (granted their LLM open source stuff failed to keep up with chinese models in 2024/2025 ; their other open source stuff for thins like segmentation don't get enough credit though). Yann's main role was Chief AI Scientist, not any sort of…
I wasn't criticising his scientific contribution at all, that's why I started my comment by appraising what he did. Creating a startup has to be about a product. When you raise 1B, investors are expecting returns, not papers.
they are not expecting returns at 1B+, just for some one to pay more than they paid six months ago
AI is developing backwards. The simplest organisms eat and find food. More complex ones can smell and sense tremors. After several steps in evolution comes vision and complex thought.
AIs that can't smell, can't feel hunger, can't desire -- I do not think it can understand the world the way organic life does.