Ask HN: What can I do to accelerate scientific research?

Surprised this comment isn't higher.

OP does indeed sound like the discipline they're looking for is Research Software Engineering.

Some reading: https://en.wikipedia.org/wiki/Research_software_engineering "Research software engineering is the use of software engineering practices in research applications. The term started to be used in United Kingdom in 2012, when it was needed to define the type of software development needed in research. This focuses on reproducibility, reusability, and accuracy of data analysis and applications created for research."

https://software.ac.uk/

https://rse.ac.uk/

I just retired from 20 years as a Senior Research Programmer at an ivy league college. It was a very rare job position, there were maybe 5-10 people doing things similar to me. I supported a few specific research projects in computer science. My position was grant funded, which meant if the grant ended, the job ended, and I had a year to year contract, so no job security at all. Much of my job over the years was doing things like technical purchasing, administrative support, proof reading papers, and web research. Mostly stuff that nobody wanted to do because it's more fun to write software or come up with ideas. I also wrote software for the more complicated parts of research projects that students or post docs often didn't have the time or experience to deal with, for example for embedded systems which most CS people have little experience with or understanding of. I think my work made the rest of the team more productive, by leaving them time to work on the key innovations. The work I did is very different than what most software engineers do though. We built many systems as proof-of-concept, leaving out the functionality that would have made it a working system, which can be frustrating for an engineer. Quality assurance, user interface design, reliability, and more were usually ignored in the interest of speed to publication. The most important goal was to publish papers, not to build software that works well. The scientific goals were also set to match the publication goal, not to solve some important problem; we often spent months trying to come up with a problem our solution solved to make it seem more attractive in a publication. So one thing that would help advance science is to encourage more basic research, and more careful research, as well as more research directed at real world problems. Currently the granting agencies and the researchers decide what the important problems to solve are and this often seems to have little relationship to the problems extant in the world. Perhaps going into politics with the goal of changing how science is funded and how that funding is directed for specific purposes would have the most impact. Right now science essentially uses a seniority system, where people who have already made their reputation get to decide who gets funding and for what, and there is a big push for quick results. That arguably means that people who are generally behind the times and short sighted are determining what science focuses on. Another issue is that there is almost no follow up in science. A grad student and professor decide on a project, in part based on whether they think it will get published/funded, they work on it for years, then that's it, nothing more is done. It's up to someone else to take it further if they feel like it. There's little overall coordination of effort, it's a big free for all, except perhaps in medicine, although a lot of that research seems to be somewhat directionless too. There are fads in science too (Internet of Things is a current fad, because there is funding for it). I think people imagine that science is doing Manhattan projects to solve the big problems; it's not like that at all, it's more like a series of pet projects, done on a whim. I'm sure not all science is like that, but a lot of it is. Maybe one thing that would help is a multidimensional index of problems that need solving, science problems but also world problems, with dimensions including cost, estimated complexity, projected benefits, potential risks (cold fusion would be great but could also destabilize economies and shift power balances), time required, etc. Many students have a difficult time finding an idea to work on (they've spent their whole school life doing exercises and now they are asked to innovate.) Research needs to be much more cross-disciplinary too. So look at your skills, identify problem areas, and think about what kind of change you could encourage. At a minimum, you could contribute time to supporting specific projects. Getting paid for that is a problem though, your salary competition is students who work for "free" and underpaid post-docs hoping for a tenure track position. My pay was much lower than current salaries in industry. As I mentioned, the work is much different than what a software engineer usually does too, which can be difficult to adjust to; we made some attempts to hire software engineers and many of them quickly left when they understood the nature of the work. It's kind of like working at a startup, except there are no stock options, little chance of the "company" becoming "successful" (maybe a patent), your job is always at risk (Congress could change NSF funding and eliminate it), work is always short term, and you rarely get to finish anything, plus you have to do a lot of work that usually an administrative assistant would do.

Solve the following problem:

"I want to find all research labs concerned with topic X in this area"

=> Create an (open) crunchbase/patreon for university research groups

Right now, each university has their own website, with their own layout and varying degrees of information. General scraping is impossible. Finding research groups with certain interests without trawling through an endless amount of publications in related journals or randomly meeting people at conferences is difficult. The only comprehensive lists are university ranking sites, but they are not detailed down to the research group level.

- create a crowdsourced, public index of running projects

- help labs find each other and collaborate on work

- help people apply to labs with open positions

- let donors and investors find and support projects more efficiently

- make science journalism easier, by connecting reporters straight to the source

The main difficulty is creating the network effect which is why, I suppose, no one has done it yet.

The ideas published in journals all depend on humans.

But to an outsider, these connections are completely invisible. Conferences cost thousands of dollars to attend. Subject matter experts are hard to find, so many connections are simply not made.

Which is a good reason to support such a platform, because it may make such views possible and existing deficits more visible and quantifiable :)

Small and medium size businesses outside academia could also use it as to improve their technology and connections.