Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hello OLMo: A truly open LLM

blog.allenai.org

Post reply on HN
Print view View original

71–76 of 76 posts

Re: Hello OLMo: A truly open LLM

#71

Post by espadrine »

simonw wrote: post #69
espadrine wrote: post #58

Earlier quoted context omitted.

> Google's Gemini LLM model was not to blame for the image generation weirdness. That would be like blaming DALL-E weirdness on GPT-4. The way I read the Gemini technical report, it seemed like, unlike GPT-4 vs DALL-E, Gemini was pretrained with multimodal outputs. Is that not the case?

Is that right? I didn't think Gemini was generating images directly, I assumed it was using a separate image generation tool. The paper here https://arxiv.org/pdf/2403.05530.pdf has a model card for Gemini 1.5 Pro that says: Output(s): Generated text in response to the input (e.g., an answer to the question, a summary of multiple documents, comparing documents/videos).

Huh, that is true in both the model cards of Gemini 1.5 Pro and Gemini 1.0.

That feels like it runs counter to this statement from the Gemini 1.0 technical report[0]:

> Gemini models are trained to accommodate textual input interleaved with a wide variety of audio and visual inputs, such as natural images, charts, screenshots, PDFs, and videos, and they can produce text and image outputs

[0]: https://arxiv.org/pdf/2312.11805.pdf

Re: Hello OLMo: A truly open LLM

#72

Post by jrm4 »

vjeux wrote: post #25

If I read the license correctly, it seems that if you want to use the LLM, you need to tell the authors what you are doing with it. Am I reading this correctly? https://allenai.org/licenses/impact-mr “Derivative Impact Reports. AI2 seeks to encourage transparency around Derivatives through the use of Derivative Impact Reports, available here. Before releasing a Model Derivative or Data Derivative, You will share with…

Weird. So even if these things are well intentioned, seems like they don't have any teeth.

Are there any out there that have licenses which are (dare I say) simpler, like the GPL?

Re: Hello OLMo: A truly open LLM

#73

Post by simonw »

espadrine wrote: post #71
simonw wrote: post #69

Earlier quoted context omitted.

Is that right? I didn't think Gemini was generating images directly, I assumed it was using a separate image generation tool. The paper here https://arxiv.org/pdf/2403.05530.pdf has a model card for Gemini 1.5 Pro that says: Output(s): Generated text in response to the input (e.g., an answer to the question, a summary of multiple documents, comparing documents/videos).

Huh, that is true in both the model cards of Gemini 1.5 Pro and Gemini 1.0. That feels like it runs counter to this statement from the Gemini 1.0 technical report[0]: > Gemini models are trained to accommodate textual input interleaved with a wide variety of audio and visual inputs, such as natural images, charts, screenshots, PDFs, and videos, and they can produce text and image outputs [0]: https://arxiv.org/pdf/23…

Yeah what does that bit about "image outputs" mean I wonder?

Re: Hello OLMo: A truly open LLM

#75

Post by Aissen »

If you're interested in truly open source LLMs, I've been playing CroissantLLM, a bilingual French/English LM; it's smaller than OLMo (1.3b vs 7b, so not in the same category), and has checkpoints, training code, logs, datasets, and a new benchmark fully open source:

Paper: https://arxiv.org/abs/2402.00786 Announce: https://huggingface.co/blog/manu/croissant-llm-blog

Post reply on HN