51–60 of 158 posts
How good is the quality of this? BLOOM is a 176B parameter model, but it doesn't seem to compare to GPT-3 (175B parameters) in terms of output quality.
"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…
Can someone tell me what the hell they use ChatGPT for? I tried it a few times and it always confidently gave me wrong results to basic things. What is this thing supposedly “disrupting”? Is it really just marketing cranking out metric tons of spam blogs?
They tried instead putting these RFPs through ChatGPT, and they were blown away by the responses they got. Of course, the responses still need to go through a thorough edit and review process, but that was also true when humans were writing the first draft.
He told me that ChatGPT obviated a couple people's jobs, with the added bonus that the turnaround time between receiving a proposal and sending a response was much faster.
Earlier quoted context omitted.
GPT-3 has been publicly covered in scientific publications. Same as GPT-2, and GPT. Those are all pre-trained models, where GPT is the abbreviation of Generative Pretrained Transformer. Transformers have been invented in 2017 at Google Brain [1]. -> https://medium.com/walmartglobaltech/the-journey-of-open-ai-... GPT-4 is around the corner, and it's allegedly 100x more powerful than it'd predecessor. -> https://medium…
That "100x" figure is extremely poorly sourced. I don't believe that at all.
Earlier quoted context omitted.
I, for one, would like to see an open-source model similar to Stable Diffusion, but for text. It would be a great way to empower general folk without having to pay OpenAI, and not have to worry about the LLM's belief system, which is conservative-biased in the case of ChatGPT[1] (HN discussion[2]). [1] https://davidrozado.substack.com/p/openaicms [2] https://news.ycombinator.com/item?id=34625001
https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_... From the graph (above) linked by the top comment in your [2], I'm wondering whether this demonstrates more anti-conservative bias than liberal bias, or whether the alternative meanings of conventionally conservative versus conventionally liberal words dictate the frequency of a flag. For instance, "Republican" means a variety of things around the world,…
Where are you getting this? The proper term is indeed "Democratic party", and this is almost universal outside of the conservative bubble. You might personally think it's not small-d democratic, but that doesn't make "Democrat party" correct.
"hitting 100 million monthly active users 2 months after its launch". I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions. I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai... "But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two p…
I’m less suspicious. Anecdotally, I’ve compared SimilarWeb on a few low-traffic sites of mine to the results according to an open source analytics tool and SimilarWeb got surprisingly close. They call it their "proprietary dataset".
As a side-note, I suspect that their sources include more than just browser extensions or it wouldn’t be so accurate for small sites. Couldn’t they buy data from autonomous systems or internet exchanges and extrapolate from that while correlating IPs with demographics? They only report rough estimates so SSL wouldn’t be a problem for their analytics.
Where are these numbers coming from? An 80GB A100 GPU is certainly more than capable of hosting a 1.5B GPT. We were running 774M on rinky-dink cards back in 2019 for our inference purposes.
I don’t understand how they went from talking about 175B params across 32 cards to 774M on one card. 175B divided by 32 is 5.4B.
In fact, I’m not sure what they’re saying in general. They seem to be confusing data parallelism with model parallelism with memory fragmentation, while namedropping a bunch of training techniques.
The hard part of ChatGPT isn’t the size. It’s the training process. It took a small army of contractors rating outputs as good or bad. Once that dataset gets replicated, we can start talking about size. Hopefully LAION will deliver.
How good is the quality of this? BLOOM is a 176B parameter model, but it doesn't seem to compare to GPT-3 (175B parameters) in terms of output quality.
It's because BLOOM is undertrained, you can prune a lot of weights in BLOOM and it doesn't impact performance. Look at Chinchilla paper[1], 70B model outperforms 175B GPT-3 model. https://arxiv.org/abs/2203.15556
> On a single multi-GPUs server, even with the highest-end A100 80GB GPU, PyTorch can only launch ChatGPT based on small models like GPT-L (774M), due to the complexity and memory fragmentation of ChatGPT. Hence, multi-GPUs parallel scaling to 4 or 8 GPUs with PyTorch's DistributedDataParallel (DDP) results in limited performance gains. Where are these numbers coming from? An 80GB A100 GPU is certainly more than capa…
> “the generative-AI eruption” I really think we should stick to Nick Bostrom’s (or pls fix attribution) term “intelligence explosion”
Even if it has not so much thing to do with intelligence?
But back to what I’m really saying here: “Generative AI eruption” is a mouthful whereas “intelligence explosion” is concise.