Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

21–30 of 337 posts

Re: MiniGPT-4

#21

Post by causi »

omeid2 wrote: post #9
wsgeorge wrote: post #3

Giving this a quick spin and I like what I'm seeing. I gave it a picture of Paolo Veronese's Lament over the Dead Christ [0] and asked what was going on in the background. > The painting depicts the scene of the crucifixion of Jesus Christ. In the foreground, there is a group of people standing around the cross, including Mary, the mother of Jesus, and the two thieves who were crucified with him. In the background, t…

What did you expect?

I never know what to expect anymore. We live in a world where computers can describe paintings and write sonnets about them but a half-trillion dollar car company can't figure out how to parallel park with eight cameras.

Re: MiniGPT-4

#22

Post by GaggiX »

kramerger wrote: post #15
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Can any of this realistically run on CPU at some point? (Not training obviously)

I've run LLaMa models on my CPU before, ViT-L and the Q-former are two transformer models as well, so I can't see why they wouldn't run on a CPU.

Re: MiniGPT-4

#23

Post by Ambix »

kramerger wrote: post #15
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Can any of this realistically run on CPU at some point? (Not training obviously)

I'm developing framework [1] in Golang with this goal in mind :) It successfully runs relatively big LLM right now, and diffusion models will be the next step

[1] https://github.com/gotzmann/llama.go/

Re: MiniGPT-4

#24

Post by chenzhekl »

152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

It sounds like a BLIP2 with an extra linear layer for finetuning (or aligning the Q-former with a new LLM?). What makes it more powerful than BLIP2?

Re: MiniGPT-4

#25

Post by collaborative »

Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

Adding GPT4 to anything now increases marketing x4. So many AI news coming out lately that not adding it risks drowning in a sea of info.. even in the case of a good project

Re: MiniGPT-4

#26

Post by causi »

Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

Someone needs to write a buyer's guide for GPUs and LLMs. For example, what's the best course of action if don't need to train anything but do want to eventually run whatever model becomes the first local-capable equivalent to ChatGPT? Do you go with Nvidia for the CUDA cores or with AMD for more VRAM? Do you do neither and wait another generation?

Re: MiniGPT-4

#27

Post by [deleted] »

Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

[deleted]

Re: MiniGPT-4

#28

Post by marban »

mjburgess wrote: post #17
sho wrote: post #14

Earlier quoted context omitted.

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

In practice, it's a lot more like web dev than you might imagine. The above means that the approach is web-dev like gluing, almost literally just, from existingliba import someop from existinglibb import anotherop from someaifw import glue a = someop(X) b = glue(a) Y = anotherop(b)

And repeat that ~4 times to make it look like LangChain

Re: MiniGPT-4

#29

Post by p1esk »

HopenHeyHi wrote: post #20
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Thanks for a useful comment. Do you reckon the 4bit quantized Vicuna just won't do here? https://huggingface.co/anon8231489123/vicuna-13b-GPTQ-4bit-1... I think with this everything OpenAI demonstrated ~5 weeks ago has been recreated by actually-open AI. Even if it runs much much slower on prosumer hardware and with less good results at least it is de-magicked.

The magic is in the quality of GPT-4 output. That hasn’t been recreated yet.

Re: MiniGPT-4

#30

Post by m00dy »

Hi CHEN,

Invoker here, I would like to have a chat or send me an email @ community@invoker.network

Post reply on HN