The word for this is “trademark infringement”. You are specifically not allowed to capitalize on the marketing of another entity’s product to bolster yours by implying through your name that you are somehow related. This is why “DALL-E Mini” had to change their name to craiyon.
It's also just (deliberately) misleading. It's based on the 13B Vicuna/Llama model, not 175B GPT-3 or 1T GPT-4. There is zero justification for calling it MiniGPT-4. A more honest name would be Visual-Vicuna or Son-of-BLIP.
> Son-of-BLIP
maybe even add an "a" for extra spice: Son-of-a-BLIP
I have access to an Nvidia A100. But as a layman, what specs does the rest of the system need to use it for some real work? I would assume there needs to be at least as much ram as vram and maybe a few terabytes of disk space. Does anyone have experience with this?
If you have an A100, which in its 80GB variant costs $23,667 [1], you would not generally quibble over the price of a few terabytes of disk space. [1] https://www.dell.com/en-us/shop/nvidia-ampere-a100-pcie-300w...
A whole different department made those decisions and I don't think they had any idea of what is actually needed. They wanted to buy such a GPU because training your own model is a trendy thing and they wanted to stay within their budget (mostly used up by the GPU).
I'm just trying to scramble together something from the hacked together thing I now have to deal with.
It has more in common with GPT-3 than GPT-4 in terms of size, but in reality it's based on Vicuna/Llama which is 10x smaller than either, so as far as the LLM part of it goes its not mini-anything - it's just straight-up Vicuna 13B. The model as a whole is just BLIP-2 with a larger linear layer, and using Vicuna as the LLM. If you look at their code it's literally using the entire BLIP-2 encoder (Salesforce code). ht…
vicuna was done with sharegpt transcripts, right? did they ever say if those transcripts were while users were using gpt3.5 or gpt4.0?
I haven't read the details of how they created the training data.
>I don't see how the universal approximation theorem is relevant here. The fact that deep models The universal approximation is exactly not about deep models. Deep means many layers. But in the most simple (and proven) case, a single hidden layer perceptron is all it needs according to the UAT. Technically it also needs a nonlinear activation function, but you get all sorts of nonlinearities for free downstream anywa…
You'd need to increase width (dimensionality) if you make these models shallow. My point still stands: The fact that models with sufficient capacity can approximate any function does not imply that two models trained independently of each other on different tasks will learn to approximate functions that relate to each other only by a linear transformation.
The UAT states that depth is fundamentally not important, at least theoretically. It only has immense practical uses. So adding an intermediate linear layer + some nonlinearity already gets you an error scaling like O(1/N) for width N (in theory), regardless of what you are actually mapping. At least as long as it's somewhat continuous.
Should I get a gaming PC with 4090 if I want to get into personal side projects in ML and also games? Do I need dualboot? Or is Windows good?
you can run Windows in VM and passthrough your videocard to it. So it's basically a native gaming (3-5% cost) w/o reboots. Look for VFIO.
Unless you have two graphics cards (well, you can use an integrated GPU) and need to run both OSes at once I think for most people this will be less convenient than a dual boot setup though
You can’t switch which GPU Linux is using without restarting the session