U.S. universities, rich in data, struggle to capture its value, study finds

I have no doubt the problem is worse in academia, but to be quite honest nobody is really doing a good job at curating, governing and unifying data across large organizations (at least without extreme expense.)

Just look at the treadmill of architectures designed to "solve" this problem: integrated databases to "data warehouses" to "linked data" to "data lakes" to "data fabric" to "data mesh."

Organizations that succeed do so because they have the resources to prioritize data management, and do so at significant team size and expense. But any organization that doesn't (or can't) view data curation as first class, top-line budget isn't in a position to capture even a small part of the theoretical value of their data.

Consider the "data mesh" architecture, the latest fad in this space. It basically assumes that not only do you have a dedicated data curation team, each data source also has it's own dedicated team of curators capable of productizing data sources.

That kind of thing is simply out of reach for most organizations.

We need more ad-tech startups for the university/higher education industry NOW

There's a cost/benefit curve here.

Having data can undeniably be useful. The question is, how much ROI does the insights in the data provide, relative to the cost to build the organizational infrastructure capable of extracting these insights?

And there's a bootstrapping problem, because often it's hard to calculate the potential ROI without actually going through the work of building the infrastructure to process the data.

> They have no idea if: Some professors more effective teachers than others.

What are you even talking about? Literally every course at most universities end with an evaluation. Obviously, there are other problematic ways to measure effectiveness (publications). How much that data gets used is another question and varies widely by institution. “People in tech businesses” love to exaggerate the quality of the data they have and its usefulness, hence the Subprime Attention Crisis that is rapidly eroding their cash on hand as the Fed’s cheap money and QE dries up.

I run a centralized data and analytics team at a large, online university that primarily serves adult learners. I’ve been responsible for our data and analysis for 10 years.

I think this article rings true for most universities, but for some, like mine, we’ve had unified systems for the last 20 years. While that’s been helpful in a lot of ways, the primary thing we’re looking for - helping students graduate and get better jobs than they have today - is surprisingly challenging. Adult learners often fail to finish school for a variety of reasons, most of which aren’t academic and often can’t easily be forecast using all available data.

Unifying data sources or mining them further doesn’t solve student health issues, childcare, jugging work responsibilities with school, etc.

What’s more, universities are slow-moving, heavily bureaucratic, and highly regulated. So even when we have good ideas to relieve burden those often come head to head with accreditation rules, financial aid regulations.

In my opinion, for my students at least, data is the least of our problems.