Live data from Hacker News

Unpredictable abilities emerging from large AI models

quantamagazine.org

181–190 of 326 posts

Re: Unpredictable abilities emerging from large AI models

#181
post #69

Earlier quoted context omitted.

there is no channel for uncertainty. LLMs of this type will just start making up shit when they dont know something. because they simply generate the most probable next token based on previous x tokens. this is not fixable. this alone makes these LLMs practically unusable in vast majority of real-world applications where you would otherwise imagine this tech to be used.

The reason they seem to make things up is because they have no way to verify anything, they can only speak of things in relation to other things, but they have no epistemic framework. This is very much a fixable problem that augmentation with logic engines and a way to prioritise truth-claims could go some ways towards solving.

My memory could be improved by connecting my brain to an external hard drive. Wiring them together, alas, is not just hard; we have absolutely no idea how.

Re: Unpredictable abilities emerging from large AI models

#182
post #23

Earlier quoted context omitted.

I have fun on these HN chats responding to comments like yours . It’s just fancy auto complete to you? You honestly can’t see the capability it has and extend it the future? What’s that saying about “it’s hard to get someone to understand something when their salary depends on their not understanding it”.

I feel very frustrated with these takes because instead of grappling with what we're going to do about it (like having a conversation) it's a flat, dismissive denial, and it isn't even grounded in the science, which says that "memory augmented large language models are computationally universal". So at the very least we're dealing with algorithms that can do anything a hand written program can do, except that they've…

«Computationally universal»? Are you quite sure? Hoo boy pass the nitroglycerin hooooo aaah ohh after 80 years Turing was proven right after all ghasp

Re: Unpredictable abilities emerging from large AI models

#183

Nice write up! I have been using classic back-prop neural networks since the 1980s, and deep learning for the last 8 years. This tech feels like a rocket ship that is accelerating exponentially! I am in my 70s and I don't work much anymore. That said, I find myself spending many hours in a typical day doing what I call "gentleman scientist" activities around Large Language Models. I was walking this morning with a no…

Thanks for the future you helped create for us young people. Hopefully the next generation is more merciful to 'useless' people than the previous one.

This is shortsighted. AI slave labor is the only route to a true overproduction and marxism, a welfare state. Or every body become slaves.

Re: Unpredictable abilities emerging from large AI models

#184
post #182

Earlier quoted context omitted.

I feel very frustrated with these takes because instead of grappling with what we're going to do about it (like having a conversation) it's a flat, dismissive denial, and it isn't even grounded in the science, which says that "memory augmented large language models are computationally universal". So at the very least we're dealing with algorithms that can do anything a hand written program can do, except that they've…

«Computationally universal»? Are you quite sure? Hoo boy pass the nitroglycerin hooooo aaah ohh after 80 years Turing was proven right after all ghasp

https://en.wikipedia.org/wiki/Universal_approximation_theore...

Re: Unpredictable abilities emerging from large AI models

#185
post #80

Earlier quoted context omitted.

Oh man... if it will be as stupid as us humans we are completely doomed. For now lets assume what it most likely is: not emerging anything, just a very good simulation of that emergeance. I mean we already did stupid chat bots in the 90s with convoluted if-else-spaghettis that imitated Eliza. That could already hold for some sentences... GPT-4 now holds much longer, but still you can easily drive it to the point wher…

I have very limited understanding of how these systems are created, but as far as I know they are not programmed to simulate anything. It is just a system that outputs the most likely next word based on a really large dataset of training samples it was given. Yet you are still able to "chat" with these systems. Here's a session I just ran on llama.cpp (7 billion parameter version). Everything above "====" line is the…

User

Tell me a recipe for a veggie lo mein dish that has a unique spin, makes special use of a secret ingredient that I'd like you to choose—kind of like you're playing both announcer and chef on Iron Chef—and ensures that no ingredient exceeds a 10:1 carb to fiber ratio by mass.

GPT-4

Welcome, esteemed viewers, to this special edition of Iron Chef! Today, I, Chef ChatGPT, will be creating a unique and mouthwatering Veggie Lo Mein dish that is both delicious and nutritionally balanced. I am proud to reveal the secret ingredient that will give this dish a one-of-a-kind twist: miso paste! Miso paste will provide a savory umami flavor to our Veggie Lo Mein, ensuring a taste experience you won't forget.

Now, let's begin with the ingredients. Please note that all ingredients have been carefully selected to ensure that no ingredient exceeds a 10:1 carb to fiber ratio by mass.

Ingredients:

8 oz whole wheat spaghetti noodles 2 tbsp vegetable oil 1/2 cup julienned carrots 1/2 cup thinly sliced red bell pepper 1 cup chopped bok choy 1 cup sliced shiitake mushrooms 2 cloves garlic, minced 2 green onions, chopped 1/4 cup miso paste (secret ingredient) 1/4 cup low-sodium soy sauce 1 tbsp rice vinegar 1 tbsp agave syrup or honey 1 tsp grated fresh ginger 1/2 tsp sesame oil Crushed red pepper flakes, to taste Optional garnish: sesame seeds and chopped cilantro Instructions:

Cook the whole wheat spaghetti noodles according to the package instructions until they are al dente. Drain and set aside.

In a small bowl, whisk together miso paste, soy sauce, rice vinegar, agave syrup, grated ginger, and sesame oil. This will be our umami-packed sauce.

Heat vegetable oil in a large skillet or wok over medium-high heat. Add garlic and stir-fry for about 30 seconds, or until fragrant.

Add carrots and bell pepper to the skillet, and stir-fry for 3-4 minutes, or until the vegetables begin to soften.

Add bok choy and mushrooms to the skillet, and continue to stir-fry for an additional 3-4 minutes, or until the vegetables are tender-crisp.

Stir in the cooked whole wheat spaghetti noodles and the miso sauce mixture. Toss everything together until the noodles and vegetables are evenly coated with the sauce.

Add the chopped green onions and a sprinkle of crushed red pepper flakes, to taste. Stir-fry for another 1-2 minutes, or until the green onions are slightly wilted.

Remove from heat and serve immediately. Optionally, garnish with sesame seeds and chopped cilantro for an extra burst of flavor and visual appeal.

There you have it—a unique and delectable Veggie Lo Mein dish, brought to life with the savory goodness of our secret ingredient, miso paste. Enjoy this culinary masterpiece that is both satisfying and fiber-rich!

Re: Unpredictable abilities emerging from large AI models

#186

Earlier quoted context omitted.

I'm sort of glad to be retiring soon. I have a feeling everything I enjoy about programming is going to be going away.

I'm about 10-15 years from retiring, and lately, I've been thinking a lot about how to handle the rest of my career with all these new AI advancements.

This disruption does appear to be different than others prior. As it is not a narrow disruption with limited boundaries for which we can plan and organize our lives for some stabilization period to follow.

Instead, it is constant and accelerating destabilization. Hundreds of AI projects attempting to ride this accelerating wave were essentially just made obsolete yesterday - https://www.youtube.com/watch?v=DH-2BHDYNfk

I feel the excitement is going to very soon turn into frustration of attempting to remain relevant ahead of the accelerating technological curve. Humans need periods of stabilization to plan and reason about their lives.

Re: Unpredictable abilities emerging from large AI models

#187
post #139

Earlier quoted context omitted.

Doing a table lookup for common cases and computing for the less common ones is perfectly valid!

Not 100% sure but I believe this is how we landed the lunar module on the moon the first time...tan/arctan/both were too hard to compute on the processors of those days so they discretized into half angles & stored the tangents in a lookup table.

That's how early calculators worked, and some fast math libraries today too.

Re: Unpredictable abilities emerging from large AI models

#188
post #180
post #157

Earlier quoted context omitted.

>We don't know our own cognition works which makes all arguments along the lines of "LLMs are just .." Sure, but there are very binary tests we can do to understand the first principles of what LLMs are vs. what they are not. Ask an LLM to play tic-tac-toe and it does great. Ask it to play tic-tac-toe on a 100x100 board, it get's confused. This is a very easy test to examine the limits of it's ability to do symbolic…

How does it work regarding queries in natural language? I mean, thinking on translating a natural language question to an SQL query in complex scenarios.

I've been asking GPT-4 to design whole systems for me off of sparse natural language specifications. It gives reasonable designs, I read and critique, it updates and modifies. I regularly run into limitations, sure, but it will likely blow you away with its capability to convert natural language questions to SQL---given adequate specific context about your problem.

Re: Unpredictable abilities emerging from large AI models

#189
post #45

Earlier quoted context omitted.

It's probably going to struggle with things it hasn't seen before?

> It's probably going to struggle with things it hasn't seen before? It wont. It'll just lie through its teeth and produce a very nice, very believable story which will unfortunately shatter when confronted with the real world.

I've asked it to design novel architectures. It has vast experience with existing systems, can be steered toward your goals, and writes simple prototype code more quickly than I can. I run into the current context window pretty quickly and have been working on techniques to ask it to "compress" our conversation to work around that context window.

The whole thing about creativity is that it often begins with lying through your teeth to come up with a starting point and then refining.

Re: Unpredictable abilities emerging from large AI models

#190

Earlier quoted context omitted.

I feel very frustrated with these takes because instead of grappling with what we're going to do about it (like having a conversation) it's a flat, dismissive denial, and it isn't even grounded in the science, which says that "memory augmented large language models are computationally universal". So at the very least we're dealing with algorithms that can do anything a hand written program can do, except that they've…

I agree 100%. I don't understand why we can't look at the potential, or even current, capabilities of these LLMs and have a real conversation about how it might impact things. Yet so many folks here just confidently dismiss it. "It doesn't even think!" -- OK, define thinking? "It doesn't create novel ideas!" OK -- what do most devs do every day? "It is wrong sometimes!" OK -- is it wrong more or less often than an av…

In case we forget the amazing predictive capabilities of HN there’s always https://news.ycombinator.com/item?id=9224
Post reply on HN