Earlier quoted context omitted.
I have strongly believed the best way of evaluating the state of AI is to look at what people complain it can't do. It's taken a few short years to go from "keep a coherent story over more than a sentence" to "write a multithreaded sudoku solver in one shot". Many programmers would fail to do this. I'm not even sure a randomly selected human would understand the question. And I'm wondering if you've setup any loops l…
> write a multithreaded sudoku solver in one shot I did not say anything about threads but I did hint at the fact that coroutines can be used creatively to solve problems which require backtracking and constraint propagation. The fact that this is still controversial means a lot of people are unaware LLMs do not support backtracking, recursion, and constraint propagation. Sudoku is just an obvious example of a proble…
> Such problems can also be expressed as integer programs but even fewer people know how to do that so I don't usually bring it up but it would be another good test for any software system that is claimed to be intelligent by the corporate marketing department.
It's very telling if the level of testing is "can it do something few people in the field can?".