curiously, there is a mild correlation between transformer and UFO. https://hn.curiosity.ai/#/trends?terms=transformer;UFO
Don't give me ideas,
I might have to add a clone of https://tylervigen.com/spurious-correlations for HN posts!
It's not only you, there's an explosion of ChatGPT on HN
91–100 of 148 posts
Re: It's not only you, there's an explosion of ChatGPT on HN
#92Not really an HN problem, but related:
I am very fatigued by the ai "hustle" culture that seems to have bubbled up overnight. ".eth" guys changing their name to ".gpt".
It would be funny, if it wasn't so cynical, how quickly everyone seems to have acquired expert level knowledge in the field of language models. Something that, just a couple years ago, was a fairly obscure field of artificial intelligence.
"LLMs (\w*coin) will revolutionize work (finance). Here's a medium article I just wrote about how the number of attention layers in a transformer (zero knowledge proofs) will change your morning routine."
Re: It's not only you, there's an explosion of ChatGPT on HN
#93Earlier quoted context omitted.
after having studied this more and listened to a talk by one of the OpenAI team I think this will actually go away as a problem in the semi-near term future. Is that talk available online? I’m skeptical that they will ever solve the problem of factuality. I’d love to hear their arguments for why I’m wrong.
Sure, here you go: https://www.youtube.com/watch?v=hhiLw5Q_UFg Summarizing: 1. Obviously you can't completely "solve" truthfulness because people disagree on what is and is not true. But you can go a long way. 2. The models do know what they don't know. Their level of uncertainty is not only expressed in the final token logprobs but also seems to be reified somehow, such that they can express their own level of certa…
This is complete technical hubris.
You're effectively saying that the "truth" problem with AI is solved by letting it guess. Which is probably a fine enough answer, but doesn't actually "solve" the problem at all.
To the points:
3) Training on data sets where the answer is never "I don't know" is effectively the same as raising children to believe they're always right and teaching them to be needlessly confident. No one should trust those folks.
4) "Lack of access to information" is not the issue... These models are trained on the entire corpus of Twitter, Wikipedia, and more information than I've ever seen in my lifetime, and some of them already do this (Bing) and produce little more than summaries of blog posts based on keywords. If anything, the issue is that an LLM lacks the real world knowledge to discern any nuance as to whether something is correct or grounded in reality.
Re: It's not only you, there's an explosion of ChatGPT on HN
#94Earlier quoted context omitted.
I think the difference between those other hype cycles and this one is utility. I found Bitcoin very interesting (Erlang as well, still do) but not very practical in my day-to-day life. Unless I'm going to be hired by a firm that specializes in that tech, they're just not very useful. Nerdy and interesting and fun yes, but I just don't need them in my day-to-day life. On the other hand ChatGPT 3.5 has become immediat…
Yeah, more or less in this boat too. ChatGPT does things . Useful things. "Crypto" has some clever ideas, but always seemed too full of utopians and grifters and people with a solution in search of a problem. I've used Erlang professionally and like the ecosystem very much. I am a bit annoyed with myself for still not having a better understanding of how these LLM's work, because that's important, at the end of the d…
I feel the same way about not understanding how LLMs work. I've found Hugginface's tutorials a good place to start: https://huggingface.co/course/
Re: It's not only you, there's an explosion of ChatGPT on HN
#95I know it's controversial during this incredible AI hype cycle on this site, but as a AI skeptic and increasingly neo-Luddite, I've reduced the amount I spend on this site by 90% since the start of the year. If only I could add my userscript on my iOS Firefox to just hide any post containing the words "GPT", "AI" or "LLM", but alas, I guess I will have to be productive instead of slacking off reading HN. I honestly m…
Have you actually used it?
I can't believe how many people are totally refusing to try it but made judgements on it.
It just seems so slow and time consuming to do things like brainstorm snack or birthday party ideas. ChatGPT can make a list in moments, where with google, you get generic SEO spam.
Re: It's not only you, there's an explosion of ChatGPT on HN
#96Earlier quoted context omitted.
> what people who were overly hyping bitcoin got wrong Did they? It's the best performing asset in the last decade. I think they got it right.
You'd have to substantiate that claim which would probably take significant work.
From 2015 to today it appreciated 8000%. Do I need to elaborate further?
Re: It's not only you, there's an explosion of ChatGPT on HN
#97I was absent from hn pretty much exactly when chatgpt broke out for a few months. the first thing i noticed when i returned is how it now dominates this site. there's pretty much always 2 or 3 chatgpt related articles on the front page. It's interesting to see how swept away users are. Especially as my impressin of HN in the year 1BC (before chatgpt) was general skepticism towards claims of ai.
Re: It's not only you, there's an explosion of ChatGPT on HN
#98Earlier quoted context omitted.
For what it's worth, and at the risk of sounding like I'm nitpicking, traditional Luddism is not dismissal of technology as overhyped or smoke-and-mirrors. Quite the opposite; the technology is recognized as very real and very impactful. The question instead becomes, "what happens to us, the ones it replaces?". And at least in the original incarnation, given an insufficiently satisfying answer, you smash some looms u…
From a historical standpoint, you're spot on. Neo-Luddism is still opposition to a lot of modern technologies, but for a very disparate set of reasons. For example, many oppose the mass adoption of social media due to perceived mental health and social impacts. Others oppose smart phones and "screen addiction." These things can qualify as "neo-luddism" even though the opposition is not rooted in job displacement. I'v…
I thought that this was me ("The industry in which I have based my entire career just doesn't excite me anymore."), but then I realized I actually am mostly just interested in programming the way some people are interested in any other skill or machine. Some people learn guitar because they need to make music and play out. Others just love playing the guitar, even and especially when nobody is watching, for its own pleasure.
To me, there is no greater toy than a programming language. ChatGPT is neat, but it isn't a programming language. Same for blockchain or agile or whatever other trend is happening in the industry. Some of the trends literally are new programming languages! Golang is really fun! So is TypeScript!
Some trends or technologies make programming even more fun (in my opinion anyway) and I embrace those feverishly: distributed version control, CI/CD, pair programming (sometimes it's even more fun with a friend!), configurable linters like Perl::Critic, Intellisense, JUnit-style testing frameworks. All this stuff helps me feel more in control of the computer (or distributed cluster of computers), which I've discovered is the main thing that gets me off about programming.
I'm even still hopeful that LLMs will have some role to play in my having more fun with programming. I've tried CoPilot and so far it hasn't grabbed me, but maybe this will change. In any case, there are clearly other people having fun with it so I guess that's good. Maybe somebody can find joy by debugging GPT-4 prompts the same way I enjoy pouring over stack traces.
Re: It's not only you, there's an explosion of ChatGPT on HN
#99Earlier quoted context omitted.
I see 3 kind of people, all common on HN: - The luddites: it's just hype. - The doomers: AI will kill us all. - The AI indie hackers: dunno about AI, I'm trying to make money with it. Somehow the intersection is right: AI is not really smart, but it will replace a lot of human activity anyhow.
- The realists: what do LLM's actually do, and what are their limits - The luddites: who is going to benefit from LLM's and who will suffer more? - The doomers: AI will kill us all - The children: look at this! look at me! look at this! Wheeeeee! - The money: how many people can we pay less or not at all thanks to AI - The scientists: ok, that's cool, but what happens if we ...
Also:
- The scammers - how many well-meaning people can we use an LLM to build meaningful (to them) relationships with, to the point where they're willing to just send us money?
- The griefers - how much discord can an LLM create, for the lulz?
(to be fair, those might be subsets of "realists", but I think they're important subsets)
Re: It's not only you, there's an explosion of ChatGPT on HN
#100Earlier quoted context omitted.
Sure, here you go: https://www.youtube.com/watch?v=hhiLw5Q_UFg Summarizing: 1. Obviously you can't completely "solve" truthfulness because people disagree on what is and is not true. But you can go a long way. 2. The models do know what they don't know. Their level of uncertainty is not only expressed in the final token logprobs but also seems to be reified somehow, such that they can express their own level of certa…
This is complete technical hubris. You're effectively saying that the "truth" problem with AI is solved by letting it guess. Which is probably a fine enough answer, but doesn't actually "solve" the problem at all. To the points: 3) Training on data sets where the answer is never "I don't know" is effectively the same as raising children to believe they're always right and teaching them to be needlessly confident. No…
I'm summarizing the talk, not giving opinions of my own. It is by an OpenAI researcher who is saying why they think GPT guesses so much and what they are doing about it.
I don't quite follow the rest of your post. Nobody is saying the solution to truth is to let it guess. At most, sometimes something not 100% perfect is preferable to nothing at all, but obviously only sometimes.
The point is to identify what in the training process is accidentally causing it to guess too often instead of admit when it doesn't know or is uncertain. Some of this bias comes from the nature of the data set. On the internet, people don't normally post "I don't know" as an answer to a question because that's useless and would be considered spam, but in conversation it's normal and desirable. In other cases they have QA datasets where the goal is to impart knowledge so every question has an answer, but this accidentally trains the model that questions always have answers. Human raters may accidentally reward guessing. And so on.
The talk goes in to what can be done to correct these biases.
Finally, in many cases where the models hallucinate it's because they can't look anything up. Yes they know a lot but just like a human this knowledge is compressed. So they make up references that sound plausible but don't exist for true facts, for example, because they can't check Google Scholar to find the right reference. This is exactly what you'd expect to see from a human who was forced to come up with everything off the top of their head. Think about how much programmers hate whiteboarding interviews, it's for the same reason. Giving LLMs tooling access does make a noticeably large difference.