Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Creativity fundamentally comes from memorization?

shwin.co

Post reply on HN
Print view View original

311–320 of 372 posts

Re: Creativity fundamentally comes from memorization?

#311

Post by jimmySixDOF »

shw1n wrote: post #9
qingcharles wrote: post #5

I genuinely thought creativity was something else until LLMs hit escape velocity and humbled me hard. After that I realized that creativity wasn't some magical quality that would be hard to reproduce mechanically. And that also made me a little sad.

I'm glad you said this -- I felt the same way after making this discovery through my method outlined in this post It similarly took the magic out of creativity and learning a bit, and made it all seem like work The main way I've found around it is the joy in being creative once basic autonomy is achieved in new skills Consciously discovering the heuristic is another fun part

you guys might be interested in the latest Machine Learning Street Talk podcast [1] which right from the start is all about how LLMs are great for creativity, as in novel combinations of trained data (from memorization) -- but are not capable of the reasoning skill needed to verify if any idea is actually plausible given a set of constraints.

[1] Do you think that ChatGPT can reason? https://www.youtube.com/watch?v=y1WnHpedi2A

Re: Creativity fundamentally comes from memorization?

#312

Post by indigoabstract »

josephg wrote: post #242
zharknado wrote: post #132

OP and others here are stretching the definition of “memorize” to mean “anything that leads to something being retained in memory.” I reject this idea. The trauma of burning your hand on a hot pan creates a memory you won’t soon forget, but almost no one would understand it as an act of memorization. Memorization to me refers to a set of cargo-culty “learning” practices wherein we believe that by using language to dr…

> We do this mainly because experts suck at empathizing with learners and fail to understand that the symbol has meaning for them but not for the learner. I hear you; but teaching deep expertise is really hard. We can use your example of a child learning their first language. They will really understand it. But people are famously, hilariously terrible at teaching their native tongue. We know how to conjugate, and ho…

I quite liked your licking metaphor/symbolism. In fact, if I find out that something tastes good after licking it, I'll go even further. I swallow it. Ha ha.

My intuition is that creativity is a more of a feeling one has, rather than the result of some teaching method or cultural background. It seems very closely related to one's interests and also the novelty effect. Just think about the first time you experienced something, like a great song or your first love. The mind is just bubbling with thoughts. How can that feeling be experienced again with the same object like it was the first time? It can't. It's never the same as the first time. Even if we can mentally forget, the subconscious somehow remembers it's had that experience, so it isn't fresh anymore.

So, yes, you need to make connections between existing things to come up with something new, but once the fascination with a thing dissipates and it becomes familiar, any further creativity in that area quickly dies off. Our interest just moves on.

Perhaps the really creative people are just the ones who have this ability to see familiar things with ever fresh eyes?

Re: Creativity fundamentally comes from memorization?

#313

Post by matwood »

josephg wrote: post #242
zharknado wrote: post #132

OP and others here are stretching the definition of “memorize” to mean “anything that leads to something being retained in memory.” I reject this idea. The trauma of burning your hand on a hot pan creates a memory you won’t soon forget, but almost no one would understand it as an act of memorization. Memorization to me refers to a set of cargo-culty “learning” practices wherein we believe that by using language to dr…

> We do this mainly because experts suck at empathizing with learners and fail to understand that the symbol has meaning for them but not for the learner. I hear you; but teaching deep expertise is really hard. We can use your example of a child learning their first language. They will really understand it. But people are famously, hilariously terrible at teaching their native tongue. We know how to conjugate, and ho…

> There is only so much the best teacher can teach. Sometimes you just have to walk around licking things.

There's being taught and then there's exploring and practice. It's the exploring and practice that is often missing today, IMO because we all want everything to happen so quickly.

In my sport of choice, jiujitsu, there's been a shift happening in teaching. Traditionally the teacher would show a move or two then have the class drill the move. The shift now is more conceptual where the teacher might say the goal is get chest to chest and then have the students operate at 20% figuring it out on their own. While there are some pretty vocal camps on both sides, I think having both methods is the best to learn from. Being taught a solution to a problem a student discovered on their own during discovery is when learning really takes hold IMO.

Finally, the best way to really learn that I have found is to teach something.

Re: Creativity fundamentally comes from memorization?

#314

Post by jamager »

zharknado wrote: post #132

OP and others here are stretching the definition of “memorize” to mean “anything that leads to something being retained in memory.” I reject this idea. The trauma of burning your hand on a hot pan creates a memory you won’t soon forget, but almost no one would understand it as an act of memorization. Memorization to me refers to a set of cargo-culty “learning” practices wherein we believe that by using language to dr…

> They aren’t being “drilled” to learn “mama

They actually are, thou. Small kids are repeated the same things over and over, hundreds of times. They watch the same Bluey episodes dozens of times, get them read the same books dozens of times, listen to the same songs dozens of times.

They drill in their own ways, but they drill.

Re: Creativity fundamentally comes from memorization?

#315

Post by Ekaros »

WalterBright wrote: post #182
zharknado wrote: post #132

OP and others here are stretching the definition of “memorize” to mean “anything that leads to something being retained in memory.” I reject this idea. The trauma of burning your hand on a hot pan creates a memory you won’t soon forget, but almost no one would understand it as an act of memorization. Memorization to me refers to a set of cargo-culty “learning” practices wherein we believe that by using language to dr…

The EEs I have known that carried around a card with: V = I * R I = V / R R = V / I because they couldn't remember it were all bad at EE and bad at math. If you can't remember the pieces making up a concept, how are you going to remember the concept? > It’s disproportionately important for passing tests (And getting hired at tech companies!) I don't remember anyone who couldn't pass tests but was really a great engin…

Saddest part is that you only need first one and and then with memorization of most basic algebra you get the others.. Not even need to do it symbolically.

Re: Creativity fundamentally comes from memorization?

#316

Post by trashtester »

coldtea wrote: post #211
jjk166 wrote: post #209

Earlier quoted context omitted.

> There are questions like max takeoff weight, fuel burn rate, max dive speeds, etc. Stuff a pilot had better know or he's a dead pilot. You don't want a pilot who is creative when it comes to max takeoff weight. Obviously there are good reasons to memorize certain things, creativity just isn't one of them.

> You don't want a pilot who is creative when it comes to max takeoff weight. In certain combat situations, or when smuggling coke across South and Central America, you certainly do.

Not to mention that in combat, you need to understand the limitations not only of your own plane but also of both the allied and enemy planes in your airspace.

But you don't want CIVILIAN pilots to get too creative, though. Running out of runway with a 747 is no joke.

Re: Creativity fundamentally comes from memorization?

#317

Post by dash2 »

rickydroll wrote: post #148
groby_b wrote: post #146

Earlier quoted context omitted.

With the caveat that I haven't looked in a while, the rot seemed very anglo-specific. Coming over from Germany almost three decades ago, it was amazing how much US text books just didn't cover. There were a few developments in Germany that pointed in the same direction, but there a large gap at the time. Meanwhile, even back then, eastern Europe certainly had even higher standards. (I replaced a lot of math textbooks…

The influence of religious and conservative types has weakened US textbooks. Do a deep dive into how Texas can dictate what is presented in schoolbooks nationwide.

That story wouldn’t work in deeply secular England, & it seems unlikely to explain underperformance in maths.

Re: Creativity fundamentally comes from memorization?

#318

Post by Eudaimion »

wokwokwok wrote: post #45

Is there anything substantive here? It’s just a bunch of arbitrary unprovable assertions. Everyone here seems to have, broadly speaking; neither a) the qualifications to knowledgeably comment of the (honestly poorly understood, afaik) function of “creativity” or b) anything more meaningful than “here is my naive personal lived experience and opinion” to contribute on the topic. It’s just armchair psychology. If you w…

> Is there anything substantive here? There is plenty.

If you did not find anything interesting or anything that made you think, in the comments or the post itself. Then is your own failure.

Re: Creativity fundamentally comes from memorization?

#319

Post by bunderbunder »

kqr wrote: post #239
bunderbunder wrote: post #183

Earlier quoted context omitted.

An anecdote I once heard provided in support of this point in a lecture: there are aspects of Spanish grammar that native speakers typically don't grasp until their teenage years. I suspect that, if one were to sit down and count hours of practice so that we could do a better apples-to-apples comparison, we'd find that children learn languages at a glacial pace compared to adults. And the rest is pure selection bias.

The one thing children can learn that adults generally cannot is native pronounciation. If that is included in language proficiency, then adults take an infinite amount of time to learn!

True, but I really dislike the amount of focus people put on that. With the right techniques, it's not particularly difficult for an adult to develop decently good pronunciation that allows them to be comfortably understood. And placing the bar for having learned a language to a high level of proficiency at "able to hide evidence of where they were born from native speakers" smacks of internalized xenophobia.

I also read in a second language acquisition textbook a while back (so, decent chance of being out of date, also decent chance of my memory of what I read being less-than-perfect) that the strongest predictor of how native-like an accent someone develops isn't actually the age at which they started learning the language, it's the lag between when they started learning it and when they started socially integrating into a community of native speakers. And it happens to be the case that, for all sorts of practical reasons, there's a strong correlation between this lag period and age. Which isn't to say that there's nothing to the critical period hypothesis - there is - but when we're talking about children it's difficult-to-impossible to root out selection bias effectively enough to permit even a convincing stab at partial identification of causal effects. Which creates a risky situation for the purposes of drawing firm conclusions, because our prejudices love to helpfully answer the questions that science won't.

In grad school I had a social science textbook that dubbed these kind of questions "Fundamentally Unanswerable Questions" in the first chapter, and in subsequent chapters simply described them as "FUQ'd".

Re: Creativity fundamentally comes from memorization?

#320

Post by darby_nine »

josephg wrote: post #242
zharknado wrote: post #132

OP and others here are stretching the definition of “memorize” to mean “anything that leads to something being retained in memory.” I reject this idea. The trauma of burning your hand on a hot pan creates a memory you won’t soon forget, but almost no one would understand it as an act of memorization. Memorization to me refers to a set of cargo-culty “learning” practices wherein we believe that by using language to dr…

> We do this mainly because experts suck at empathizing with learners and fail to understand that the symbol has meaning for them but not for the learner. I hear you; but teaching deep expertise is really hard. We can use your example of a child learning their first language. They will really understand it. But people are famously, hilariously terrible at teaching their native tongue. We know how to conjugate, and ho…

> I hear you; but teaching deep expertise is really hard.

No it's not; this is the point of apprenticeship. It's just not compatible with the educational institutions people actually pay for.

> I think the learner would just have to go lick a lot of things for themself.

This is not contradictory. If your idea of teaching is just lecturing at people and occasionally verifying specific knowledge, obviously they're not going to learn very much. Your job as a teacher is to facilitate a growth curve of increasingly more difficult and unwieldy problems to solve.

Post reply on HN