Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

241–250 of 337 posts

Re: MiniGPT-4

#241

Post by jxy »

152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

This opens a huge possibilities. It's likely we could simply plug in stable diffusion using a linear layer. As well as whisper and some TTS. Getting a back to back mixed image/sound/text engine running on a laptop.

I wonder if there's powerful enough ViT model that does OCR.

Re: MiniGPT-4

#242

Post by gary_0 »

moffkalast wrote: post #69
causi wrote: post #21

Earlier quoted context omitted.

I never know what to expect anymore. We live in a world where computers can describe paintings and write sonnets about them but a half-trillion dollar car company can't figure out how to parallel park with eight cameras.

DriveGPT as it hits 2 parked cars and runs over a dog: "Apologies for the confusion earlier. You are correct that the cars were in the planned local planner path..."

It's like something right out of a Douglas Adams book.

Re: MiniGPT-4

#243

Post by gary_0 »

causi wrote: post #190
astrange wrote: post #87

Earlier quoted context omitted.

That's a well known result from the last AI hype era. https://en.wikipedia.org/wiki/Moravec%27s_paradox Although, I'm not surprised AI can describe paintings, but I'm still pretty surprised it can generate them. How'd we get that to work!?

Yeah, and I'm still completely lost as to why resolution is such a limiting factor. If you know you're drawing a soccer ball why is a 512x512 soccer ball so much easier than a 1024x1024 soccer ball?

There are a few cases where people have used ChatGPT to generate SVG[0], with mostly unimpressive results. I'm sure sooner or later models will be developed specifically for creating vector drawings instead of raster, including with the ability to apply textures and gradients.

Also, the resolution of Stable Diffusion's output isn't much of a limitation if you're willing to use other tools to massage the output into something professional-quality. See [1]

[0] For example: https://praeclarum.org/2023/04/03/chatsvg.html

[1] https://news.ycombinator.com/item?id=35592847

Re: MiniGPT-4

#245

Post by lhl »

BeefWellington wrote: post #230
lhl wrote: post #150

Earlier quoted context omitted.

6800 is RDNA2, not RDNA3. The latter is still waiting for ROCm support 4 months post-launch: https://github.com/RadeonOpenCompute/ROCm/issues/1813

I'm aware that a 6800 is not RDNA3. You stated broadly: > Current AMD consumer cards have terrible software support and IMO isn't really an option. On Windows you might be able to use SHARK or DirectML ports, but nothing will run out of the box. I was merely sharing that I did not have that same experience that current consumer cards have terrible support.

Sure, and I was merely clarifying that only last-gen architectures work. While I'm glad that SD works for you, if we are getting into it, I think that having no support on current-gen flagship models does equal broadly terrible software support and is more worth highlighting, since otherwise someone might assume they could just go pick one of those up and get a 24GB GPU on the cheap, especially in the context of LLMs (which is what the OP was asking about).

For RDNA2, you apparently can get LLMs running, but it requires forking/patching both bitsandbytes and GPTQ: https://rentry.org/eq3hg - and this will be true for any library (eg, can you use accelerate? deepspeed? fastgen? who knows, but certainly no one is testing it and AMD doesn't care if you're not on CDNA). It's important to note again, anything that works atm will still only work with last-gen cards, on Linux-only (ROCm does not work through WSL), w/ limited VRAM (no 30Bq4 models), and since RDNA2 tensor support is awful, if the SD benchmarks are anything to go by, performance will still end up worse than an RTX 3050: https://www.tomshardware.com/news/stable-diffusion-gpu-bench...

Re: MiniGPT-4

#246

Post by cloudking »

It doesn't seem to recognize text from a screenshot very well, I gave detailed error messages from a Windows screenshot: https://filestore.community.support.microsoft.com/api/images...

and prompted "Describe any issues found in this screenshot and steps to resolve them"

while it correctly identified it as a screenshot from a computer, it gave a very generic response and didn't identify the error messages correctly:

"The screenshot shows a computer screen with a warning message that reads, “The system is currently unavailable. Please try again later.” This indicates that there is an issue with the computer system that needs to be resolved.

To resolve this issue, you can try the following steps:

1. Check the system status: Check the system status to see if there are any errors or issues that need to be resolved.

2. Restart the computer: Try restarting the computer to see if that resolves the issue.

3. Check for updates: Check for any updates that may have been released for the computer system and apply them if necessary.

4. Check for viruses: Run a virus scan on the computer to check for any viruses that may be causing the issue.

5. Check for hardware issues: Check for any hardware issues that may be causing the issue. This could include issues with the power supply, hard drive, or other components.

6. Contact technical support: If none of the above steps resolve the issue, contact technical support for further assistance. They may be able to provide additional troubleshooting steps or help you resolve the issue."

Re: MiniGPT-4

#247

Post by raylad »

I gave it a picture of a bottle, which it did a fairly good job of identifying (although it misread some of the text, mistaking a 1 for a 4).

Then I asked it what are the likely ingredients of the product. It still hadn't replied after 2274s so I gave up on it.

Re: MiniGPT-4

#248

Post by pmoriarty »

jcims wrote: post #228
pmoriarty wrote: post #40

Earlier quoted context omitted.

How about asking it to describe a Robert Williams painting? Here's a relatively easy one: [1] [1] - http://www.illustrationartgallery.com/acatalog/WilliamsTimmy...

Used this prompt Could you describe the image as thoroughly as possible? Please include the content, any noteworthy symbology, techniques or styles, what kind of media, guess at the artist or identify any attribution, then attempt to describe the theme or idea it represents? Got this The image is a painting of a child running down a dirt road with a kite in hand. The child is wearing a yellow shirt and blue shorts. T…

It completely missed what makes this painting distinctive and funny/scary, and what's actually going on.

It also missed the right half of the painting.

Try asking it why the painting is called "Timmy's Last Surprise".

Re: MiniGPT-4

#249

Post by dharma1 »

mewpmewp2 wrote: post #53
redox99 wrote: post #46

Earlier quoted context omitted.

It's pretty simple actually. Get a 3090 or 4090. Forget about AMD.

Should I get a gaming PC with 4090 if I want to get into personal side projects in ML and also games? Do I need dualboot? Or is Windows good?

4090 is amazing, but very large card. 3090 is "good enough" for ML - same 24gb vram - and you can pick them up used for half the price of a new 4090. That's what I did.

WSL on windows apparently decent, or native PyTorch, dual boot windows/ubuntu still prob best tho.

Re: MiniGPT-4

#250

Post by artificial »

seydor wrote: post #140
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Then it's an impressive demonstration of how modular neural networks can be. Maybe we don't even need to train monoliths

Maybe a distributed trainer? AI@Home?
Post reply on HN