Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

171–180 of 337 posts

Re: MiniGPT-4

#171

Post by idkyall »

f1codz wrote: post #151
ikurei wrote: post #109

Earlier quoted context omitted.

> This ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy. Just like any discussion between advanced web devs would make any humble woodworker feel? And just like any discussion between advanced woodworkers would make a humble web dev feel? "It's really simple, they're just using a No. 7 jointer plane with a high-angle frog and a PM-V11 blade to flatten those curly birch boards, then a…

You make a good point. Except that a number of these concepts and tooling in the ML world have been slingshotted into the forefront in a relatively short time and it has been hard to play catch up. For eg. - someone said "frozen Vicuna" below - what does that mean?

Vicuna is a specific open source AI LLM: https://ai.plainenglish.io/vicuna-the-unparalleled-open-sour...

Re: MiniGPT-4

#172

Post by thewarrior »

jokoon wrote: post #169

Are there more university/research centric resource of communities regarding AI, I mean things on a more epistemic and higher level, that disregards current ML techniques and implementations? I have no problem when there is a lot of money spent to implement things that can be used and tested, but I wish I could read about the next step of AI, something that involves science and not software engineering. I've yet to r…

We’ve tried but we understand even less about how “real” intelligence works. Atleast in a way that we can replicate it.

The people doing so called alchemy are making far better systems that anyone who’s trying to actually understand things. And we’ve been trying for decades now.

Re: MiniGPT-4

#173

Post by craigching »

ttul wrote: post #158
ikurei wrote: post #109

Earlier quoted context omitted.

> This ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy. Just like any discussion between advanced web devs would make any humble woodworker feel? And just like any discussion between advanced woodworkers would make a humble web dev feel? "It's really simple, they're just using a No. 7 jointer plane with a high-angle frog and a PM-V11 blade to flatten those curly birch boards, then a…

The thing is, machine learning sorta requires a few math prerequisites: linear algebra, differential equations, and to some degree vector calculus. Most web developers don’t have this background.

If you want to understand the theory, that's true. If you want to develop an intuitive understanding without having to understand all the nuts and bolts (and I understand that can be a big ask for how some people learn/understand), give this a try: https://karpathy.ai/zero-to-hero.html

Re: MiniGPT-4

#174

Post by ggregoire »

Could you use something like this to summarize the content of a video? Or would you first need to extract the images from the video (like 1 fps, with ffmpeg) and tell the AI to analyze all the images and summarize them like if it was a video?

Re: MiniGPT-4

#175

Post by lhl »

lxe wrote: post #154

Why Vicuna over something like oasst-pythia? Would love to see a table comparing all the new models side by side.

Fabrice Bellard has run a standard set of benchmarks w/ lm-eval on a big chunk of open models here: https://bellard.org/ts_server/ - Flan T5 XXL and GPT-NeoX 20B both outperform Pythia 12B on average (LLaMA 13B+ tops the charts).

Re: MiniGPT-4

#176

Post by minimaxir »

guax wrote: post #48
mjburgess wrote: post #17

Earlier quoted context omitted.

In practice, it's a lot more like web dev than you might imagine. The above means that the approach is web-dev like gluing, almost literally just, from existingliba import someop from existinglibb import anotherop from someaifw import glue a = someop(X) b = glue(a) Y = anotherop(b)

And just like webdev, each of those were done in a different platform and require arcane incantations and 5h of doc perusing to make it work on your system.

This is why the Hugging Face transformer ecosystem is so good, as each of those blocks will roughly have the same unified API.

Re: MiniGPT-4

#177

Post by nobrains »

I think the real news is where this research is coming from. It is from King Abdullah University of Science and Technology in Saudi. The arab world has recently been quite behind in tech research, but this is good news overall for the arab world, which will inspire other universities and researchers in the region to one-up their colleagues, which is overall good for the entire world.

Re: MiniGPT-4

#178

Post by visarga »

cs702 wrote: post #142
AbrahamParangi wrote: post #131

Earlier quoted context omitted.

I think it’s just that affine transforms in high dimensions are surprisingly expressive. Since the functions are sparsely defined they’re much less constrained compared to the low dimensional affine transformations we usually think of.

Good point. Didn't think of that. It's a plausible explanation here, because the dimensionality of the spaces is so different, 5120 vs 768. Not surprisingly, the trained weight matrix has rank 768: it's using every feature in the lower-dimensional space. Still, it's kind of shocking that it works so well! I'd be curious to see if the learned weight matrix ends up being full-rank (or close to full-rank) if both spaces…

They would have full-rank because all the embedding space is used. There are no unused large pockets.

Re: MiniGPT-4

#179

Post by Tepix »

causi wrote: post #26
Tepix wrote: post #12

I think it's poor form that they are taking the GPT-4 name for an unrelated project. After all, the underlying Vicuna is merely a fine-tuned LLaMA. Plus they use the smaller 13B version. The results look interesting, however. Here's hoping that they'll add GTPQ 4bit quantizing so the 65B version of the model can be run on 2x 3090.

Someone needs to write a buyer's guide for GPUs and LLMs. For example, what's the best course of action if don't need to train anything but do want to eventually run whatever model becomes the first local-capable equivalent to ChatGPT? Do you go with Nvidia for the CUDA cores or with AMD for more VRAM? Do you do neither and wait another generation?

Read Tim Dettmer's guide.

Basically, you want nVidia, and you want lots of VRAM. Buy used for much more bang for the buck.

Depending on your budget, get:

- an RTX 3060 with 12GB or

- 1 used RTX 3090 with 24GB (approx twice as expensive as the 3060 but twice the VRAM and much faster) or

- 2 used RTX 3090 cards if you need more than 24GB.

Everything beyond that gets quite a bit more expensive because then you need a platform with more PCIe lanes, you may need more than one PSU and you will have problems fitting and cooling everything.

With two cards and 2x24GB you can run the largest version of the LLaMA model (the 65B variant) and all its descendants with 4-bit quantization inside your GPU's VRAM, i.e. with good performance. Can can also try some low resource fine-tuning variants (LoRa etc).

Oh and while you're at it also get a decent amount of RAM like 64GB or 128GB (it's very cheap right now) and a NVMe SSD. These models are quite large.

Post reply on HN