Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

161–170 of 337 posts

Re: MiniGPT-4

#161

Post by Philip-J-Fry »

niemandhier wrote: post #33
causi wrote: post #21

Earlier quoted context omitted.

I never know what to expect anymore. We live in a world where computers can describe paintings and write sonnets about them but a half-trillion dollar car company can't figure out how to parallel park with eight cameras.

That's because the decided they do not need lidar.

It's not lidar they need. BMW, Mercedes, Porsche, etc. All can park themselves almost perfectly every time. Teslas can't, and will take 5x as long to do it when they can, if the computer can even bother to recognise the space.

It's software. Original Teslas with AP1 better than Teslas own in house software on their latest AP.

Re: MiniGPT-4

#162

Post by iandanforth »

Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

Filed an issue: Up or down vote there as you see fit.

https://github.com/Vision-CAIR/MiniGPT-4/issues/9

Re: MiniGPT-4

#163

Post by iandanforth »

twelvechairs wrote: post #42
Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

Its poor form to be calling it 'Open' AI. So I guess its swings and roundabouts. Everyone is leeching where they can.

To be fair they were open when that name was picked and it looks like they may be trying to transition to just 'ai.com'.

Re: MiniGPT-4

#164

Post by Regic »

mewpmewp2 wrote: post #53
redox99 wrote: post #46

Earlier quoted context omitted.

It's pretty simple actually. Get a 3090 or 4090. Forget about AMD.

Should I get a gaming PC with 4090 if I want to get into personal side projects in ML and also games? Do I need dualboot? Or is Windows good?

I'm running models locally on my 3090 and it's fast enough, although for example building a vector database can take a while. I can run LoRa training but I haven't done anything meaningful with it so far. I chose 3090 because of the cable issue of 4090 (also, no nvlink, although I'm not sure that matters) but it's debatable if my fears are justified. I need to leave the gpu running while I'm away and I just don't feel comfortable doing that with a 4090. I rather take the lower performance.

One caveat though, my asus b650e-f is barely supported by the currently used ubuntu kernel (e.g. my microphone doesn't work, before upgrading kernel + bios I didn't have lan connection...) so expect some problems if you want to use a relatively new gaming setup for linux.

Re: MiniGPT-4

#166

Post by craigching »

sho wrote: post #14
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

I was where you're at about ... oh wow, it's been almost ten years since I jumped into machine learning. Mind you, I've been learning on the side most of this time other than a theoretical class at the University of Minnesota. But, that aside, and depending on where you're at in your understanding, this is a great resource for catching up if you're really interested: https://karpathy.ai/zero-to-hero.html it was posted on HN a couple of weeks ago and I have to say it's a really good introduction and Andrej Karpathy is a passionate and excellent teacher. You may want to brush up on some intro Calculus, but it's very understandable.

Re: MiniGPT-4

#167

Post by jcims »

pmoriarty wrote: post #47
wsgeorge wrote: post #37

Earlier quoted context omitted.

A 14-line poem with a consistent rhyme scheme and meter. Perhaps my request should have been more specific.

From my experimentation, LLMs tend to kind of suck at rhyme and meter, and all but the simplest types of poetry, so even if you'd specified it probably wouldn't have been able to deliver. This is definitely something they could be trained to be much better at, but I guess it's hasn't been a priority.

Has anyone tried using phonetic tokens instead of text? I'm curious if that would help with things like rhyming.

Re: MiniGPT-4

#168

Post by m3kw9 »

This is basically Image to Text, then from the text, start a GPT type chat. No need to overthink anything

Re: MiniGPT-4

#169

Post by jokoon »

Are there more university/research centric resource of communities regarding AI, I mean things on a more epistemic and higher level, that disregards current ML techniques and implementations?

I have no problem when there is a lot of money spent to implement things that can be used and tested, but I wish I could read about the next step of AI, something that involves science and not software engineering.

I've yet to read anything related to the analysis of trained data or trained neural networks, the structure of the brains of small insects, psychology on general intelligence, AI and emotions, the real ability of an AI to think and write meaningful code or understand what is a problem/solution etc, I mean anything that is related to AI that can be treated by computer science.

I have no problems with algorithms and implementations, but don't university/professors also try to go beyond, with epistemology, to connect the dots with biology and neurology?

Unless I see science cover several fields of research, it's not real, worthy research in my view, which is why the reason why I'm so stubbornly skeptical of chatGPT.

Re: MiniGPT-4

#170

Post by lhl »

kramerger wrote: post #15
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Can any of this realistically run on CPU at some point? (Not training obviously)

Yes, you can run inference at decent speeds on CPU with llama.cpp. A token is about 0.75/words, so you can see lots of people getting 4-8 words/s on their CPUs: https://github.com/ggerganov/llama.cpp/issues/34

There a lot of optimizations that can be done. Here's one w/ potentially a 15X AVX speedup for example: https://github.com/ggerganov/llama.cpp/pull/996

Post reply on HN