21–30 of 144 posts
I understand the process abstractly, but I am unable to grok the details about how it's able to take my vague interpretation of what I want and then write code and actually give me exactly what I wanted.
I posted some comments about ChatGTP in a local FB group and there was a pretty large percent of folks who responded here that think it's just an awful thing that's going to lead to the downfall of civilization. I tried to offer that it is pretty cool, but it's just software that basically presents search engine results in a different manner along with a few other tricks, but it's not "HAL". I live in a very red and…
> I live in a very red and rural area so that probably has something to do with it. They love to have new things to complain about that have no effect on any of us at all. That seems an ungenerous interpretation. I don't doubt that their understanding is full of science-fiction inspired fear, but the implications and dangers of this tech is a hotly debated topic among informed experts. So, even if their specific fear…
I posted some comments about ChatGTP in a local FB group and there was a pretty large percent of folks who responded here that think it's just an awful thing that's going to lead to the downfall of civilization. I tried to offer that it is pretty cool, but it's just software that basically presents search engine results in a different manner along with a few other tricks, but it's not "HAL". I live in a very red and…
This seems like an unnecessary invocation of negative stereotypes. What makes you think people outside outside that demographic don't have similar thoughts? Anecdata and all that.
I get the sense that ChatGPT crosses a complexity threshold where there’s no good way to describe how it works that satisfies everybody, and that’s leading to cyclical stories of the form “everyone else describes it wrong, so here’s my take.” As a heuristic, I see descriptions falling into simple buckets: - stories that talk about tokens - stories that don’t talk about tokens Anything discussing technical details suc…
I think that is due to LLMs being somewhat magical. I think that the wolfram article "What Is ChatGPT Doing … and Why Does It Work?" captures this beautifully.
ChatGPT is probably the first software product that I have no idea how I'd go about implementing. I watched a number of YT videos about it including Andrew Carpathy's 2 hour coding session building a mini GPT. I understand the process abstractly, but I am unable to grok the details about how it's able to take my vague interpretation of what I want and then write code and actually give me exactly what I wanted.
Trial and error in terms of tweaking the system prompt is surprisingly instructive.
I like the token window bit. I don't really like the probability bit, because it kinda alludes that OpenAI just built a huge probability map of all N-grams (N=8000) and called it a day. Which incidentally would also imply that a lot of N-grams just don't exist in the training data, causing the model to completely halt when someone says something unexpected. But that's not the case - instead we convert words into a lo…
The whole point of mentioning relatedness is to show readers that we’re not dealing with finite n-grams.
The biggest drawback of LLM is that it never answers with "I don't know" (unless it is some quote) and it just brings bullshit hallucinations which human has to reject as wrong. Thus it is mostly useless for anything serious. Personally I use it to beautify some text, but still have to do a bit of correction to fix b/s or missed context.
This could end up being another example of knowledge quality going down as people keep going for the easiest was to get answers, like searching for info on topics on Instagram and TikTok.
I posted some comments about ChatGTP in a local FB group and there was a pretty large percent of folks who responded here that think it's just an awful thing that's going to lead to the downfall of civilization. I tried to offer that it is pretty cool, but it's just software that basically presents search engine results in a different manner along with a few other tricks, but it's not "HAL". I live in a very red and…
I feel like as an intro aimed at "normies" it still manages to communicate in a more abstract and overthinky way than necessary. People often find it difficult to intuit examples from abstract descriptions. BUT, people are great at intuiting abstractions from concrete examples. You rarely need to explicitly mention abstractions, in informal talk. People's minds are always abstracting. > If I’m relating the collection…
Same goes for its super brief explanation of a core term like "latent space", and it explain probability distributions by invoking atomic structures rather than a more basic stats example like a bell curve of adult human heights. It's definitely aimed at the sort of "normie" that reads Hacker News rather than actual normal people! I liked it though...
I like the token window bit. I don't really like the probability bit, because it kinda alludes that OpenAI just built a huge probability map of all N-grams (N=8000) and called it a day. Which incidentally would also imply that a lot of N-grams just don't exist in the training data, causing the model to completely halt when someone says something unexpected. But that's not the case - instead we convert words into a lo…
The model says, "I don't have a good enough path forwards here, I'll just make one up given the next best thing I have and serve it back"?
Maybe this is why Bing is working differently, they've changed the model or the working to just say, "I don't know" when there isn't enough confidence in what it's generated based off what it finds in it's database?