Meanwhile, for the use case I recently posted about, no amount of searching answers my inquiry, and ChatGPT can give me a good answer in seconds.
So yeah, moral of the story, depending on context YMMV? Also call me old fashion but I still cross reference ChatGPTs response with other sources…if I care about accuracy and truth, which I basically always do.
You definitely should, hallucination can happen at any time. Luckily for my use case, which is explaining API changes for the purposes of porting a game mod, the compiler tells me if the AI was right or not.
This comes at the price of now both OpenAI and Wolfram getting access to your queries (when you use the plugin, of course). I’m hoping we don’t slide into a future where you either have to give up on confidentiality and privacy or else can’t use any “modern” tools anymore.
How is this different than Google/WA today?
It isn’t, this is literally the Google / Meta business model.
This is a match made in heaven, which makes ChatGPT actually useful for factual data. Or the inverse, it makes Wolfram Alpha even more accessible. That example screenshot of ChatGPT generating three queries to Wolfram Alpha in succession in order to answer the initial question is amazing. It's just how a human might have used Wolfram Alpha to do the same.
OpenAI becomes a really powerful corporation right now. Until reasonable alternatives are developed, in many professions you will be basically handicapped without it. And the cherry on top is that they have all inputs. These provide incomparably more data than e.g. google search queries. Not to mention they have a tool which excels at guessing user intention and interests.
No, Chrome was a revelation when it came out. It was way faster than Firefox and it had many great features we now take for granted, like isolated tabs, popup blocking etc.
isolated tabs and popup blocking (which firefox already had) were not ground breaking in any way. They were implementation details. By that point, Firefox was stable enough to not make a big difference if it was isolated tabs or not.
Last paragraph: I see what’s happening now as a historic moment. For well over half a century the statistical and symbolic approaches to what we might call “AI” evolved largely separately. But now, in ChatGPT + Wolfram they’re being brought together. And while we’re still just at the beginning with this, I think we can reasonably expect tremendous power in the combination—and in a sense a new paradigm for “AI-like co…
Between this and his last post, he continues to dismiss ChatGPT as fancy next word statistics.
That's not the takeaway I got from the "how it works" post. He does describe the word statistics part in detail - which is, of course, pretty fundamental to how it works - but he doesn't claim that it's merely a "stochastic parrot", like many do. On the contrary (highlighting mine):
"... this means that ChatGPT—even with its ultimately straightforward neural net structure—is successfully able to “capture the essence” of human language and the thinking behind it. And moreover, in its training, ChatGPT has somehow “implicitly discovered” whatever regularities in language (and thinking) make this possible."
I cannot imagine a scenario where it doesn’t drop off. The massive recent improvements in GPT’s performance are a result of giving the model enormous numbers of parameters and a wealth of training data. That’s it. Surely this paradigm cannot scale up indefinitely. Moore’s law is moribund. Are we going to build a supercomputer that encircles the globe just for the purpose of trying to make the biggest LLM we can? Also…
Firstly, LLMs are an embarrassingly parallel problem, so yes, you can actually get quite far simply by throwing more hardware at it. The catch is that the gain is not linear - e.g. you need 4x more VRAM for 2x inputs / context window size. But if doing that unlocks more useful emergent properties, it may well be a worthwhile trade-off - and it doesn't have to spontaneously become an AGI for that to happen. So I think we'll be playing this game for quite a long time.
Wolfram himself laid out why this is unlikely to happen, https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... >Or put another way, there’s an ultimate tradeoff between capability and trainability: the more you want a system to make “true use” of its computational capabilities, the more it’s going to show computational irreducibility, and the less it’s going to be trainable. And the more it’s fundamentally…
What is a feed forward network? Is that anything like the current-state and next-state functions for representing finite state machines?
No, a feed forward network is just a neural network which doesn't contain cycles, in contrast to recurrent neural networks, which do. A system that can't perform cyclical behavior is not Turing complete, since we know that it will halt. Large language models are not recurrent networks because they don't contain cycles (the whole point of the transformer architecture is to mix position information with the tokens so that the model is sensitive to short and long range context, which obviates the need for recurrent networks for "memory" ), so they are not Turing complete - at least in a single iteration - and will probably never compute many functions efficiently. Nevertheless, because you can feed the generated text back into the LLM pipeline, they can operate on current-state and next-state as in your example of a finite state machine.
Stephen Wolfram is pretty comfortable being in profitable niches. Even if there was a possible mutually beneficial arrangement it still behooves him to extract value out of it. Especially if upside is disproportionately beneficial to Bing and in time the language model will absorb some of the WolframAlpha capabilities so there is a risk factor involved as well. Plus, people with large egos tend to demand costly tribu…
True but it's quite a danger, another way to see it is that Wolfram is now teaching OpenAI model for free, showing prompt and answers for complex mathematical questions.
That is not a risk. There are already many examples of questions and answers to engineering/physics/mathematical problems in the existing training set. The issue is that GPT is fundamentally not suited for mathematical reasoning, no matter how much training data you feed it.