Liquid Foundation Models: Our First Series of Generative AI Models
1–10 of 159 posts
Re: Liquid Foundation Models: Our First Series of Generative AI Models
#2It's pretty impressive, just note (emphasis added):
> At Liquid AI, we take an open-science approach. We have and will continue to contribute to the advancement of the AI field by openly publishing our findings and methods through scientific and technical reports. As part of this commitment, we will release relevant data and models produced by our research efforts to the wider AI community. We have dedicated a lot of time and resources to developing these architectures, *so we're not open-sourcing our models at the moment*. This allows us to continue building on our progress and maintain our edge in the competitive AI landscape.
Looks like there's no paper (or similar) yet, either. Hopefully they'll release a more detailed writeup soon.
Re: Liquid Foundation Models: Our First Series of Generative AI Models
#3Love the humour here:
What are Language LFMs not good at today: […] Counting r's in the word "Strawberry"!
Re: Liquid Foundation Models: Our First Series of Generative AI Models
#4Give this prompt and watch it burn!
""" Tell me a poem in Polish about dance and love and loss and a spider called stephan """
It goes into a tailspin repeating the following line over and over again till it crashes.
""" W tym kąciku kątny, gdzie tango tańczyli, Była tylko ona, tylko on, tango. W tym kąciku kątny, gdzie tango tańczyli, Była tylko ona, tylko on, tango. """
Re: Liquid Foundation Models: Our First Series of Generative AI Models
#5So a new set of small models that are competitive with and beat phi-3.5 on some benchmarks is extremely impressive.
Context length is the next frontier for models in this range - getting to a true 128-200k token length in a small model would be very very useful. Hallucinations are less of an issue because you can just dump all the source data in, whole codebases can go in for stuff ranging from a refactor to ‘write documentation of the API’.
Claude and gpto-preview are the only games in town for these long context tasks right now, and they are slow. Some of the tasks need the extra intelligence, but a lot don’t. In those cases a lightweight or local model will be greatly appreciated. Not to mention context length that long will more easily enable multimodal parsing.
Re: Liquid Foundation Models: Our First Series of Generative AI Models
#6It's pretty impressive, just note (emphasis added): > At Liquid AI, we take an open-science approach. We have and will continue to contribute to the advancement of the AI field by openly publishing our findings and methods through scientific and technical reports. As part of this commitment, we will release relevant data and models produced by our research efforts to the wider AI community. We have dedicated a lot of…
Missed opportunity. I would argue that the only way they CAN make these smaller models competitive is to make them openly available. As a developer, I'm not going to choose an unknown startup's model over bigger closed models from OpenAI or Anthropic. And if I really need something smaller and faster, I'd prefer to run the model myself for better control and no risk of the model being "upgraded."
Re: Liquid Foundation Models: Our First Series of Generative AI Models
#7So a new set of small models that are competitive with and beat phi-3.5 on some benchmarks is extremely impressive. Context length is the next frontier for models in this range - getting to a true 128-200k token length in a small model would be very very useful. Hallucinations are less of an issue because you can just dump all the source data in, whole codebases can go in for stuff ranging from a refactor to ‘write d…
Gemini has the largest context of any model out there FYI.
Re: Liquid Foundation Models: Our First Series of Generative AI Models
#8Give this prompt and watch it burn! """ Tell me a poem in Polish about dance and love and loss and a spider called stephan """ It goes into a tailspin repeating the following line over and over again till it crashes. """ W tym kąciku kątny, gdzie tango tańczyli, Była tylko ona, tylko on, tango. W tym kąciku kątny, gdzie tango tańczyli, Była tylko ona, tylko on, tango. """
I changed Polish to Romanian and it entered an infinite loop and then crashed. Nice.
Re: Liquid Foundation Models: Our First Series of Generative AI Models
#9Give this prompt and watch it burn! """ Tell me a poem in Polish about dance and love and loss and a spider called stephan """ It goes into a tailspin repeating the following line over and over again till it crashes. """ W tym kąciku kątny, gdzie tango tańczyli, Była tylko ona, tylko on, tango. W tym kąciku kątny, gdzie tango tańczyli, Była tylko ona, tylko on, tango. """
I've tested gpt4o and they've clearly improved since I've tested last year ago when poems were working only in english (like it would apply translation when you asked for other language)
Re: Liquid Foundation Models: Our First Series of Generative AI Models
#10So a new set of small models that are competitive with and beat phi-3.5 on some benchmarks is extremely impressive. Context length is the next frontier for models in this range - getting to a true 128-200k token length in a small model would be very very useful. Hallucinations are less of an issue because you can just dump all the source data in, whole codebases can go in for stuff ranging from a refactor to ‘write d…
It seems like this is a new architecture, alongside Mamba, Hyena, RWKV and Hybrid Transformer.