181–190 of 217 posts
Earlier quoted context omitted.
Are there any TPUs that can be purchased off-the-shelf and then owned, like you can do with a CPU or GPU? Or are you just limited to paying rent to cloud providers and ultimately being at their mercy when it comes to pricing, ToS, etc?
No, but you probably aren't going to buy an A100 either, so it's a moot point.
Earlier quoted context omitted.
That's beside the point, which is that the output copilot produces is useful.
I don't see how that's besides the point. How can it be that useful if the output is a such legal mystery? I'd love to use it but not when there's such a risk of compromising the code base.
Earlier quoted context omitted.
they didn't. that was just alpha beta search with some custom hardware to speed it up. also at this point, both of the strongest chess ai (stockfish and lc0) are using neutral networks and are roughly 1000 elo above where deep blue was (and most of that is from software, not hardware)
> just alpha beta search I will cling to these goal posts every time. Search was and still is AI, unless you think Russell and Norvig should have named the field's foundational textbook something other than "Artificial Intelligence: A Modern Approach"
Earlier quoted context omitted.
>> I would challenge you to name any (non-simple) problem where traditional AI methods are still state of the art. Theorem proving, classical planning, SAT solving, robotics, search, in particular adversarial search, program induction, knowledge representation. Plus all the stuff that used to be considered "AI" but aren't anymore, like rule-based systems (e.g. for fraud detection) etc. Sorry, I know you asked for onl…
Great answers. If we check back in a couple years, I'm sure we'll see good learned approaches to most of these problems. Robotics is already on its way there. Is this program induction https://paperswithcode.com/task/program-induction ? Looks like it's headed towards ML too. I suspect you could stick ML into SAT solving and get yourself a system that worked pretty decently.
These search and sampling algorithms still play key roles in game playing AI (chess, poker, Go) and natural language generation. It is the human knowledge, specification and heuristics, part that tends to be more readily replaceable. A lot of control flow and data-structures that powered old AI approaches can be found in databases, compilers, type inference, computer algebra and even the autodiff libraries neural nets are written in.
Video game AI, constraint solving and business rules engines are probably closest to still using the full symbolic approach rather than merely extracting the control flow and structures portion.
We can therefore make a compact prediction: learned approaches replace human written computer programs (specifications, rules systems or heuristics) whenever human contribution is not valuable or is somehow harmful to robustness/generalization.
I never got why GPT-3 was so closed off, like you needed permission to use it. If it’s so good then why not just make it available?
I think the reasoning was that GPT-3 could easily be used to fill the world with realistic bullshit that would take ages to debunk.
I think other comments are more valid. Sale potential. Buyouts.
Earlier quoted context omitted.
That would only follow if we were trying to optimize code for brevity, and I have no clue why that would be your top priority.
Not at all. Brevity (or verbosity) is largely orthogonal to level of entropy or redundancy. In principle it ought to be possible to code at a higher level of abstraction while still using understandable names and control flow constructs.
I never got why GPT-3 was so closed off, like you needed permission to use it. If it’s so good then why not just make it available?
Earlier quoted context omitted.
>> I would challenge you to name any (non-simple) problem where traditional AI methods are still state of the art. Theorem proving, classical planning, SAT solving, robotics, search, in particular adversarial search, program induction, knowledge representation. Plus all the stuff that used to be considered "AI" but aren't anymore, like rule-based systems (e.g. for fraud detection) etc. Sorry, I know you asked for onl…
Great answers. If we check back in a couple years, I'm sure we'll see good learned approaches to most of these problems. Robotics is already on its way there. Is this program induction https://paperswithcode.com/task/program-induction ? Looks like it's headed towards ML too. I suspect you could stick ML into SAT solving and get yourself a system that worked pretty decently.
When you say "ML" you probably mean the deep neural networks approaches that are currently state of the art for machine vision etc. Deep neural network approaches have been proposed for the task of program induction but they generally lag well behind symbolic machine learning approaches.
The most coherent efforts to tackle program induction by neural networks that I am aware of is the work of Dawn Song's group at Berkeley [1] and of Joshua Tenenbaum's group at MIT. I can't find a handy link to a compilation of the latter group's work but the Dreamcoder paper in the paperswithcode search you linked to was an interesting milestone [2].
There is a lot of work on neuro-symbolic approaches to program induction, for example see the recent (two weeks ago) NeSy workshop [3], part of the first International Joint Conference on Learning and Reasoning for some new work in that burgeoning field. Statistical Relational AI combines symbolic with probabilistic learning; see the STAR-AI workshop [4] also at IJCLR.
If you're interested in recent developments on the front of program induction (again, learning programs from examples) then IJCRL is the conference to keep an eye on.
_______
[1] https://sunblaze-ucb.github.io/program-synthesis/index.html
[2] https://arxiv.org/abs/2006.08381
Earlier quoted context omitted.
>> 1. There's a world of problems (such as "perception-related" e.g. vision and NLP) which we tried to solve for decades with symbolic AI and got worse results than what nowadays first-year students can do as a homework with ML; Perception tasks were traditionally attempted with statistical machine learning approaches rather than symbolic AI, for example the Perceptron was a very early neural network that was used in…
Various NLP tasks used to be very heavy on symbolic approaches (with some of them still being used), I myself worked on them for some years until the statistical approaches started to work better. For computer vision, I would probably consider the work on edge detectors, HOG and SIFT algorithms as the "symbolic" heritage for object detection which has now been replaced with pure ML.
In any case, it seems to me that while real progress has been achieved in language modelling, the same cannot be said for language understanding. That's a bigger conversation but anyway, modelling is still what statistical learning techniques do best, whereas anything to do with semantics, you still need some kind of symbolic approach.
I never thought of HOG and SIFT as "symbolic". If I remember correctly, they were just sets of hand-crafted features? But, features for classifiers, like SVMs and so on.
Earlier quoted context omitted.
>> I would challenge you to name any (non-simple) problem where traditional AI methods are still state of the art. Theorem proving, classical planning, SAT solving, robotics, search, in particular adversarial search, program induction, knowledge representation. Plus all the stuff that used to be considered "AI" but aren't anymore, like rule-based systems (e.g. for fraud detection) etc. Sorry, I know you asked for onl…
Great answers. If we check back in a couple years, I'm sure we'll see good learned approaches to most of these problems. Robotics is already on its way there. Is this program induction https://paperswithcode.com/task/program-induction ? Looks like it's headed towards ML too. I suspect you could stick ML into SAT solving and get yourself a system that worked pretty decently.
Around November 2023? I'll try to remember.