Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI’s Instagram Problem

deeplearning.ai

Post reply on HN
Print view View original

81–90 of 99 posts

Re: AI’s Instagram Problem

#81

Post by pantulis »

sigmoid10 wrote: post #80
pantulis wrote: post #75

The article is correct, but it comes from someone that is already on the AI field. LLMs will peak and pass to become mature, that's high-frequency signal. But there's also a low frequency component, a more fundamental signal if you like, that tells us that AI is definitely out of the academia --it's been for some years, it's only reaching general public now. It cannot be ignored.

It has already reached the public a while ago, it just wasn't announced or marketed so heavily. Google for example has been using BERT to improve search results since 2019. If you did slightly more sophisticated english language queries back then, you've already been benefiting from LLMs directly.

Also, transformer-based language models were already being used by Spacy, right?

Re: AI’s Instagram Problem

#82

Post by alfiedotwtf »

bogomipz wrote: post #45
version_five wrote: post #14

This is correct, but there's also some meta comment to be made that research has an "instragram problem". What he's talking about isn't really even AI so much as deep learning: there are lots of other branches that now get virtually no play relative to deep learning. And then there are all sorts of adjacent CS, image processing (when was the last time you saw someone talk about wavelets), language analysis, and other…

>"What he's talking about isn't really even AI so much as deep learning: there are lots of other branches that now get virtually no play relative to deep learning." Can you say what are some of non-Deep Learning branches in AI that aren't getting play relative Machine Learning/Deep Learning?

Expert Systems /s

Re: AI’s Instagram Problem

#83

Post by tysam_and »

kanyethegreat wrote: post #61

I've definitely been feeling deflated. Every angle I've come up with... some project has narrowly shipped before me. And they gain so much popularity so quickly that mindshare per project becomes a power law overnight. It's like the first person to release is at the top of the App Store, and there's no unseating them; the feedback loop is already too powerful. It very much feels like first to release wins. And I swea…

I would not put so much stock in the first mover effect. I can't bring it to mind immediately, but there was an excellent podcast that brought up how the second movers oftentimes do better in the end, at least as a company.

Case in point -- once upon a time, there was a big race called Dawnbench for training CIFAR10 to 94% accuracy in the shortest amount of time a little while ago by Stanford University. During that time, there was a lot of cool movement, and there were a few notable people who really moved the bar (Chen Wang is underrecognized for their contributions, while David Page is relatively well known for his, which indeed truly are excellent).

I remember reading Page's notes on it and thinking that I could never come up with the caliber of ideas that he brought to the training table for these networks, and plus, 24 seconds on a V100?!?! Crazy.

That was years ago that I saw it. I didn't touch it at all -- not anyone really did, transformers were sorta the big thing now, and still are. And the one or two times I did try to do anything with it...anything I tried made it worse, and I really struggled with his code (it's very functionally-written stylistically, very cool but didn't jive with my rapid experimentation style).

In any case, I thought maybe I could do better though if I really and truly took a cool crack at it. And even if I didn't, I sorta needed a good living resume to prove that I could make a good software project. So I reimplemented it in a more hackable (to me, at least) kind of way ala karpathy's nanoGPT (and was almost way too meticulous with writing, organizing, and documenting my code), reorganized and streamlined a few things, and moved it to a more-accessible-to-me GPU, an A100. ~18.1 seconds or so (17.2 with some other open-source code). So that was the line.

Since then, every single time it feels like I've found all that I can find, there's something else (eventually, at least) waiting behind that wall for me. 18.1 seconds turned to 12.7, which I thought was about as far as I could go. Then 12.7 turned to 12.3, which turned to ~9.91 seconds. Then ~9.91 seconds became, incredibly, ~7.7 seconds or so.

Earlier this week I released an update that brought it to roughly ~6.97-6.99 seconds or so. That is unreal, to me. At first, I was numb to how much things could improve, now I'm sorta in denial. The throughput is totally insane, roughly 88,389 training images through the GPU _every second_. This also means that our step time is roughly ~11.35 microseconds per batch, which is...blistering, to stay the least. Hard really for me to wrap my own head around it.

I'd say from the experience that I've had, I've felt similar feelings to what you've talked about here, especially if someone already with a lot of followers from a more hype point of view does something like glue huggingface code together, make a fancy GIF that's well stylized, and gets a ton of adoration from it.

But that said, the market for quality software is small, and the market for hype is large. Not that the above project doesn't have hype, but it's meant to be more valuable as a researcher's workbench than a toy. It did thankfully get a huge boost early on because Karpathy tweeted it out, but even the last release, for example, maybe got 10 likes on Twitter, and an additional 10-20 (or 30) stars on Github from the sum total interactions (including a Reddit post), even if that.

But! The good thing in some senses is that the people that I get to talk to if I'm proactive, that like this software, are often people who are known or are skilled in their field of work. And I honestly don't have too many warm fuzzies about that from lived experience as that is new to me. But I can say that I appreciate the opportunity.

Everytime I've thought about going down the hype/vaporware road just to get eyes on the project(s) I do, I have to ask myself -- "Do I want these eyes on the project? Do I want this kind of attention from this kind of person to make up most of my interactions and what I am building?"

Sure, if you have to feed a family, that sort of make sense. And we have to feed ourselves and our emotional needs too. But maybe we can be okay with being content with the smaller audience, as it is. At least, that's what I'm working towards, though I do fear that I'll stumble and give in to the allure of chasing the hype every now and again. And if I do, I'm sure that particular extreme emptiness (of a sort) will help pull me back towards just working on being content with the little things I have.

I want to close with a video that was made almost exclusively for you, and would like to ask you to watch it in its entirety if you have the time. It talks about content creation (which is what we do, in a sense), but is taught in a way that is very general and I think is the best take I've ever heard on this topic in a condensed/beginner-friendly way, that I can remember at least.

It should not only help alleviate some of your concerns or negative feelings from the shipping arms-race, it'll give you clarity on good next-step solutions that will help hopefully contextualize and give a good 'path forward' to making software that people like. I really cannot recommend this video enough, the wisdom is simple, practical, distilled, and hard-won (and has certainly helped me, I am glad I got to learn this earlier rather than later): https://youtu.be/lNzWsp5UUPA

Happy to discuss or offer any thoughts on any questions. I do recommend the video first, I often enjoy talking about that kind of particular topic.

Re: AI’s Instagram Problem

#84

Post by mcemilg »

raybb wrote: post #77
mcemilg wrote: post #73

Earlier quoted context omitted.

It is not enough to be the first one to ship. I shipped a Mac app that uses whisper to transcribe audio before anyone else, I even implemented a dictation algorithm on top of it. It was my first app, I am not an ios developer. However, after a while, someone with a better track record in app development came up and made a similar stuff. He knows how to build an audience, and he knows how to market an app. As a result…

Did you build macwhisper?

No, but I was talking about it, so you proved that :)

Re: AI’s Instagram Problem

#85

Post by anonzzzies »

Especially as most projects are ‘join the waiting list’ when you go there, which I assume means ‘waiting for the chatgpt api to become available’ or ‘self hosted cheap alternative that I don’t have to charge for tokens’ or, and this is probably actually the reason for most, ‘we have nothing, but figma works’.

I would really like a different category in ‘show HN’ which excludes vapourware.

Re: AI’s Instagram Problem

#86

Post by beepbooptheory »

catach wrote: post #71
beepbooptheory wrote: post #67

Earlier quoted context omitted.

I hear you but: > I have seen many examples of AI-generated art where it's beyond my taste level to see how it could be improved on by human hands This is my overriding point, beyond matters of taste. Everyone who talks about this stuff can't seem to talk about its value without reference to the hypothetical human artist or writer or whoever who didn't make it. The awe of just the fact it was made by AI can't be sepa…

> Everyone who talks about this stuff can't seem to talk about its value without reference to the hypothetical human artist or writer or whoever who didn't make it. Consider that it's because that's the interesting areas of conversation to be had. The overwhelming majority of art I see all day does not warrant comment, and so it goes with AI art that is of a quality to replace that everyday art. The AI art that I do…

Im sorry, but I just have to point out:

> And I agree that that's possibly a novelty that will pass, but that would be due in part to AI gen improving to the level where it's consistently indistinguishable from artist-made art.

Why must it be indistinguishable?? Why not good, but distinct? This is our narrowness in mind right now.

Consider, let's say, 20th century art and music. Abstract expressionism, minimalism. Things like Pollack or Rothko, Reich.

It probably takes much less than a huge array of GPUs and such to recreate something "indistinguishable" from a Pollack, granted you knew you're colors and are really trying. And its, like you say, not art that necessarily "says" anything in the form of statement.

But regardless, the thing is, that while it's not imbued with a statement or meaning that can be told as a story, it can't help but be full of something like "meaning". Post-WWII 20th century art like Pollack's, its turn to deep abstraction or otherwise nonrepresentational things, came from a shared belief about what art should become at that point. That is, now that they had all seen how broadly shared cultural narratives were so easily integrated by facism, the very idea of a work that "speaks" something was suspect.

The fact that Pollack made that kind of art when he did, and made precisely the art he did, is all a part of it. And the content of the work couldn't be any different, or he wouldn't of made it. Unconsciously or not these decisions enter our mind when we view it, they can't help but to, even if you don't have the training I'd argue. But either way its meaning or "worth" as you say can't be tied to maybe any kind of direct communication.

You could make a model do a million Pollack's or Rothko's, but nobody really is going to care, because the moment kind of passed for that kind of art, right? What should we fill our canvases with now? Nobody is going to have the same answer, but the point is that artists want to answer that question, not see how quickly or how perfectly they can fill it up with whatever, but simply, find what should be painted, what is called for. I'll remind you the specific article I have talking about, where the fear is humans will be demoralized by AI in reference to what they are passionate about. Not like, cereal boxes or whatever. Simply pleasing, generative works is somewhat a solved art anyway isnt it?

> I am sympathetic however to what may be your position: what meaning can be learned from a work if no mind was behind it to intend meaning?

I feel like thats not giving the models enough credit honestly! Isn't it rather filled with a million intentions, which can come together in new wholes infinitely. It is in fact, I think, almost like a strange but also beautiful literalization of Plato's dialogue Meno, if that's what you are referencing with the communication between souls bit.

I'm not sure I ever implied you didn't know what you liked. How was I even supposed to know you indeed like that piece that won the state fair? Why, just in general, be so defensive. Am I really saying anything at all that calls for it? I just don't understand the stakes here I guess.

How can so many people both feel that this is a paradigm shifting moment in art, and be utterly sure of how its all going to play out, much less how they will feel about it? Is this not the time for some humility? Wont that all allow us to navigate this soberly and fruitfully?

Re: AI’s Instagram Problem

#87

Post by CapricornNoble »

matwood wrote: post #34
lakomen wrote: post #12

Idk about AI having an Instagram problem, but I know Instagram has an AI problem. So many fake accounts with AI as actors and they send you chat messages trying to pretend to be real people. They even react to comments and can discern good from bad comments. At first it was interesting. Now it's just annoying. Then the Instagram algorithm, if you comment on coffee ads that you drink tea, you'll get tea ads. It will s…

IDK. Seems like usual social media that it’s all about who you follow.

I almost never use instagram, mostly follow military-related fitness[1] and car stuff....and I still get an endless feed of curvaceous women thrust in my face.

[1] https://www.instagram.com/sofletehq/

Re: AI’s Instagram Problem

#88

Post by UncleEntity »

mcemilg wrote: post #84
raybb wrote: post #77

Earlier quoted context omitted.

Did you build macwhisper?

No, but I was talking about it, so you proved that :)

I looked at your webpage and was underwhelmed, uploading the files for transcription is probably not acceptable for most people.

Then I looked at the macwhisper page and, wow… It does all the stuff.

Guessing they are using whisperX for the timecodes.

Re: AI’s Instagram Problem

#89

Post by getoffmycase »

ImaCake wrote: post #26
psyklic wrote: post #20

Earlier quoted context omitted.

Academic research has the same problem. When I studied neuroscience, flashy fMRI studies received substantially more funding than fundamental research. Without understanding how small neural nets work, it's difficult to construct bottom-up theories of the brain.

> flashy fMRI studies received substantially more funding than fundamental research This is particularly infuriating since fMRI studies are not statistically robust and should be held in deep distrust. The same happens everywhere in academia unfortunately. The current big thing is building “Apps” for your project even when no app is needed.

This is the bane of my existence. I work in a lab that uses as close to industry standard coding practices that we can do with our current skeleton crew numbers (unit tests, CI, version control, unit tests, code quality standards and norms). We lost half our senior student programmers in the last two years, replacing them with first years. You know what happened? Nothing. Our development continued as normal. I’m leaving in the next six months, and my contributions to our lab’s projects will continue to be maintained. I will receive no emails at 2 am if somebody finds a bug.

In the average academic lab, the GUI they just published had one maintainer, and it’s a grad student that’s been gone for a year already. Good luck getting an email about support, even though it doesn’t work out of the box and the code is unreadable.

Re: AI’s Instagram Problem

#90

Post by pixl97 »

beepbooptheory wrote: post #56
pixl97 wrote: post #52

Earlier quoted context omitted.

Right, I'm sure you went down to the blacksmith to have some artisan made cutlery for your kitchen when you needed some new forks?

Maybe, if you have the time, do me a favor: read the (very short) piece I am responding to, its the link at the very top of the page. If you still have a thoughtful or even snarkey comment to make to me after that, then please go ahead. But you're not really giving me anything to work with here, and around here we do strive to have real conversations. I just know you have some good points to make to me! Maybe chopsti…

Ok, so lets split the conversation into fine art versus generic commercial art and design we see everywhere.

Because as I see it, this is how the future is splitting out. This goes back to the blacksmith producing unique items. People still buy things like to this day and pay a healthy sum of money for it. But this is not the economy either, the vast majority of consumption is items that are mass produced with as little human interaction as possible. I don't see it in any way controversial that large chunks of this market will be automated away.

And example of this is in game asset design. There is a still a lot of manual human work here, but the workflows are being highly automated by 'smart' tools. I can promise you that my friend in this industry don't want to color in 50 bajillion pieces in an armor set. The tools for transfering things like style across multiple pieces massively reduce the workload. And as time goes on I expect we'll see these tools affect other areas of art. Another good example is music. A single artist can produce and distribute their own digital music easily. Entire portions of the music can be handed off to tools to put in things like drums and such.

Now when you're playing a game, or in the elevator listening to whatever's being piped out of the speaker, do you think about or care about if it's made by a human or not? It won't stop you from buying a human painted mural for your wall. You'll still listen to your favorite human artist. But expect huge swathes of the ambient art/design/possibly music around you to be automated in one form or another.

Post reply on HN