The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

Post by ChatGTP » Mon, Apr 10, 2023, 11:23 AM UTC

Lockal wrote: post #376

tachyon5 wrote: post #218

Earlier quoted context omitted.

GPT-4 fails if we add one more level in this logical problem: Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why? GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on…

Obviously, the doors must open toward the exit, as this is a fire safety requirement in most countries. However, a really smart neural network should be able to figure out that this is not a logic problem. You just have to help the blind person, not shout through the door, just open the door, and guide him through.

Post by bradleyjg » Mon, Apr 10, 2023, 11:36 AM UTC

Robotbeat wrote: post #285

bradleyjg wrote: post #268

Earlier quoted context omitted.

I haven’t seen to much discussion of what’s possible at various sizes for an early stage start up, which is a discussion I’d expect to see on yc. Clearly a company with $5-5MM in the bank can’t train a competitive LLM from scratch but what would it cost to fine tune and/or run a 65B parameter model or a hypothetical future open source 165B parameter model?

> Clearly a company with $5-5MM in the bank can’t train a competitive LLM from scratch... Wait, are we sure? I'm going to make the massive mistake of assuming we're compute bound instead of memory bound, and assume we can train at FP16 (which is a bad assumption because, of course, you're doing calculus where the little pieces you're adding up could get rounded to zero at FP16 pretty easily... although mixed precisio…

Post by uoaei » Mon, Apr 10, 2023, 11:58 AM UTC

jimmySixDOF wrote: post #325

tel wrote: post #204

Earlier quoted context omitted.

I think it's hard to deny that it's doing some level of reasoning. It's quite clear that these models do not merely echo elements of their training data and that they can solve simple and novel puzzles. What that reasoning is, exactly, is hard to know. One can suppose that ideas like "glass", "transparent", "mirror" are all reasonable concepts that show up in the training set and are demonstrated thoroughly

Solving this puzzle is an excellent example of what Ilya Sutskever said recently in the Lunar Society Podcast ..... "It seems predicting the next token well means that you understand the underlying reality that led to the creation of that token" It is the phase shift increases at this meta associative layer (which nobody seems to have seen coming from LLMs or so soon) that are responsible such feats of apparent compr…

Post by sharemywin » Mon, Apr 10, 2023, 12:57 PM UTC

FrostKiwi wrote: post #292

Aeolun wrote: post #277

Earlier quoted context omitted.

All you need is a few thousand dollars lying around to spend solely on your inference fun? I don’t think that many people really qualify as such (though it’s probably true that many of them are on HN).

Not just inference. AFAIK, you are able to fine-tune the models with custom data[1], which does not seem to require anything but a GPU with enough VRAM to fit the model in question. I'm looking to get my hands on an RTX 4090 to ingest all of the repair manuals of a certain company and have a chatbot capable of guiding repairs, or at least try to do so. So far doing inference only as well. [1] https://github.com/tloen…

Post by arthurcolle » Mon, Apr 10, 2023, 1:38 PM UTC

Jcowell wrote: post #409

golergka wrote: post #403

Earlier quoted context omitted.

Only in countries that will cling to welfare programs.

If AI becomes so good that it takes 90% of jobs won’t the majority of the developed world cling to welfare programs?