Ask HN: PhDs/students, how do you come up with viable problems/solutions?

> numerical linear algebra stuffs like iterative solvers

Been there; done some of that:

Once worked on numerical linear algebra, e.g., Gauss-Seidel. Then ran into the M. Newmann numerically exact technique based on (i) multiply by a suitable power of 10 to have only whole numbers, (ii) for a list of prime numbers, solve the system in the integers modulo each prime, (ii) construct the multi-precision rational results using the Chinese remainder theorem.

From a course, a rule: "For numerical calculation, multiply and divide freely, add OK, but avoid subtraction, especially avoid subtracting two numbers whose difference is small, i.e., nearly equal."

One day, talking with Richard Bartels, mentioned that once I wanted a random unitary matrix so generated some random vectors and applied the Gram-Schmidt process, and right away Bartels responded that Gram-Schmidt is "numerically unstable" to which I replied "Wondered about that so applied Gram-Schmidt twice". In general, Bartels has done a lot in numerical methods.

In summary, in my experience, for many decades, a LOT has been done on numerical linear algebra, including iterative methods, for the simplex algorithm, etc. E.g., at one time, used Linpack -- it seemed terrific; on a computer with a 1.8 GHz clock, called Linpack 11,000 times a second.

As I recall, in numerical linear algebra there are some fundamental issues having to do with the eigenvectors of the polar decomposition. E.g., can argue that for these issues, sometimes Gauss-Seidel cannot work well.

Might look at some of the Golub LU decomposition work, e.g., in linear programming.

If you can find some problems where can get new, correct, significant results, okay. Maybe can get some problems from some of the current AI work.

Examples: (1) Took an industrial problem, did some math and computing, and got an engineering style solution. Some other students in the department did much the same but for different practical problems. (2) Had a course in optimization, in the summer went over the notes word by word and rewrote the notes, got deep into the subject, found an unanswered question, got a solution, along the way found a surprising, general result, wrote a paper, got it accepted right away at Mathematical Programming, published later elsewhere. (3) Working in some AI to monitor systems, wanted some results with meager assumptions so used the general result tightness. So, each of these is an example of how to find a problem and get some results.

But broadly for some decades, "numerical linear algebra", including iterative approaches, is a 'well plowed field'.

In maths, start by reading recent papers and see if you can extend their results.

I can tell you the same thing I was told when I started my program: no thesis represents more than 1 year's worth of work. The reason it takes most Ph.D.s 5-10 years (8 in my case) to graduate is that you have to fail, and fail, and fail again for 4-9 years before you find your thesis project.

In my case, I started on two exploratory gene knockout "fishing expeditions", both of which didn't turn up anything interesting after a year. Then I crystalized a protein and submitted it to X-ray diffraction, but the results were not good enough for a "high quality" structure, and besides the structure we did find was not particularly interesting. Then I switched to working on NMR structures, but ended up switching universities (politics...there's going to be lots of politics) before that went anywhere.

At my new university I switched to structure modeling and worked on a project my advisor suggested for about a year to optimize a modeling routine, but even the optimized version didn't turn up anything interesting. Finally, I landed on a very intriguing problem that could have had far reaching implications. I worked hard at it for almost a year, only to realize that even state-of-the-art modeling was at least a decade away from being able to begin to address the problem I needed to solve. Finally, I returned to a question that a professor had asked me in my first year of graduate school, half jokingly, assuming there was no way to answer the question. For about a year I worked hard at it, finally arrived at a very interesting answer, and graduated.

My advisor was into that stuff, and some of my original efforts were to tie those together

The field I researched in was Symbolic Regression / Genetic Programming. My research was able to recover differential equations and systems of equations from data. We worked with GLEON to help scientists better understand lake dynamics through such systems

https://verdverm.com/projects/pge if you want to learn more

I'm no longer in academia, I'm working on ATProto things lately

I believe it depends on your field. In my branch of experimental physics (a long time ago), coming up with a viable problem was your advisor's job! To do a good experiment required collaboration, substantial grant funding, and (hotly competed for) accelerator time. Not a good idea to let a student sink or swim in that environment; the most likely result would have been to sink, which does no one any good.

But more generally, coming up with a good research problem in the sciences can be a sophisticated skill that develops over years of experience. I imagine some grad students might have a precocious talent for this, or they might just get lucky, but it's definitely not something that every beginning grad student has. (Again, at least in some fields.)