Live data from Hacker News

U.S. universities, rich in data, struggle to capture its value, study finds

newsroom.ucla.edu

51–60 of 97 posts

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#51
post #2

University data is getting outdated very quickly. Large companies have taken over the role of data curators. I personally think this is wrong. Big tech companies should build hardware, while government agencies should use that hardware to curate our data. Like how it was near the beginning of the internet. Companies should be as far away from our data as possible.

I think the exact opposite: government should know as little as is required to provide its services, as should any company. If you imagine a company can do damage with your data, wait til a bad government has it.

[deleted]

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#53

As a data scientist, I think most data is useless, but there is an addictive, video game-like quality to throwing lifeless spreadsheets into a machine and having colorful visualizations come out. It's kind of like a very boring video game for adults that makes them feel like they're working, when they're actually just enjoying colorful abstract shapes and colors. To be honest, this is probably a sizable piece of why…

> It's kind of like a very boring video game for adults that makes them feel like they're working, Absolutely love it. Let's distinguish a few things though. "Data science" seems like a pretty weird name. I mean, it's just "Science" right. Of course there's statistics, mathematics, signal processing, systems analysis, machine learning... all the good things that you and I are into. But how does this get huddled uncom…

The job description for a data scientist isn't necessarily consistent across companies, but I can imagine a good description would be to act like a real scientist in that you collaborate with product teams to figure out what experiments to run, what data to collect, and how to visualize that for both the product and business team to decide on their approach to make the company more money, ie. specializing in a/b testing and experimentation. A problem I see is creating a data science team just to follow the trends and specifically requesting they try to "study user behavior" or something in hopes the data shows some underlying trend that triggers an "eureka!" moment.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#54
post #19

Universities already waste vast sums of money of vice-provosts, middle management, administration, etc. There's no need to encourage them to start treating their mission as one of data mining in order to "capture" more value from the students.

You have a point about the amount of money spent on admin at Universities, but I think you (and most commenters here) don't really realize how bad the data situation is at a lot of major universities. If we want to streamline admin overhead at public universities, there's an incredible amount of time and money wasted due to extremely inefficient use/access to data.

I work as a BI dev at a large Big 10 school (50k students/35k faculty and staff). For all those students and staff, there are a grand total of 3 data engineers, who are responsible for all our central data warehouses, including DBAing our oracle and redshift DBs. Just to get a new (untransformed) table added to the data warehouse from a source takes 6+ months, and that's only if it's from a source with an existing integration.

On top of that, there are at least 50 people I know of whose job is basically to produce one or two reports manually in excel every week, and this isn't even considering people in finance or accounting. I'm talking about simple things like how many active research grants do we have, or how many students have enrolled in certain courses. These are reports (and entire fte positions) that could easily be automated with a single SQL query. Speaking of SQL, outside of the data engineering team, there are only 4 or 5 people who know any SQL out of the 100 I know of in reporting/BI positions. The "advanced" data teams are using MS access as an ETL tool to pull together data for tableau reports.

However, there are a lot of institutional issues that make fixing those problems difficult. For one, while we have a central IT dept, we also have about 10 individual college-level IT teams, which means that data isn't just in different databases, but on a whole separate network. For example, if I want to create a report on student faculty ratio, I need to connect to VPN 1 to export faculty data from redshift, then switch to VPN 2 to export student data from Oracle, then switch to VPN 3 so that I can upload both datasets to our depts SQL server. After all that I can finally write a SQL query to get a student faculty ratio. Oh and when we need to update that ratio in a month, I'll have to go thru the whole manual extract/load process again. Forget automating that, since the network teams have no incentive to allow any tunnelling or bridging from one network to another.

I could rant about this all day, but I think it's fair to say that there are still a ton of low-hanging fruit inefficiency-wise at universities. If we could get universities to value their data more highly, maybe that wod have the additional effect of solving some of these problems and even be a net money saver.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#55
Honestly, down with data science. Down with university administration.

Stop running dragnet data collection on unwilling participants and stop using it to try to “optimize” your interactions with them.

This data shouldn’t exist, it’s value shouldn’t be exploited.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#56
Because it does not "belong" to them. It belongs to all science and all society, whom will through creative destruction find ways to extract the value for everyone.

The same applies to corporations, landholders, monarchs and parliaments and governments.

New year is bringing out my fundamentals :-)

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#57
post #26
post #12

I have no doubt the problem is worse in academia, but to be quite honest nobody is really doing a good job at curating, governing and unifying data across large organizations (at least without extreme expense.) Just look at the treadmill of architectures designed to "solve" this problem: integrated databases to "data warehouses" to "linked data" to "data lakes" to "data fabric" to "data mesh." Organizations that succ…

I lived through an acquisition in the teens that I feel was an illustrative story. An old financial services provider embarked on a massive, whole organization modernization effort. The new CEO went all in and actually made the necessary investment in staff and headcount, which was remarkable in itself. A big part of the effort involved providing customers with big data solutions and all the implied benefits that wou…

Same as it ever was.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#58
The abstract suggests higher ed administrators collect data from “research, administrative operations and other sources” but there is one source not mentioned: donors.

I am absolutely sure that donor data is organized, systematized, useful and generates positive ROI.

It’s surprising that this study would not mention donor data (unless they had a predetermined “struggle with data” point of view in mind)

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#59
post #26
post #12

I have no doubt the problem is worse in academia, but to be quite honest nobody is really doing a good job at curating, governing and unifying data across large organizations (at least without extreme expense.) Just look at the treadmill of architectures designed to "solve" this problem: integrated databases to "data warehouses" to "linked data" to "data lakes" to "data fabric" to "data mesh." Organizations that succ…

I lived through an acquisition in the teens that I feel was an illustrative story. An old financial services provider embarked on a massive, whole organization modernization effort. The new CEO went all in and actually made the necessary investment in staff and headcount, which was remarkable in itself. A big part of the effort involved providing customers with big data solutions and all the implied benefits that wou…

>Turned out the entire modernization and big data effort had been an elaborate bait and switch committed by the board and CEO of the acquired company

Are you saying that rhetorically, or do you actually believe (perhaps with evidence) that the modernization and big data effort was not undertaken in earnest?

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#60

As a data scientist, I think most data is useless, but there is an addictive, video game-like quality to throwing lifeless spreadsheets into a machine and having colorful visualizations come out. It's kind of like a very boring video game for adults that makes them feel like they're working, when they're actually just enjoying colorful abstract shapes and colors. To be honest, this is probably a sizable piece of why…

Only slightly related to your very funny comment here, you might enjoy “The Visual Display of Quantitative Information” by Edward Tufte. Not only does he help you to best make the abstract shapes and colors, but there’s an added aspect of the visuals that tickles the mind. That added aspect is how easily some information is communicated along with the image. It’s along the lines of how sometimes someone can tell a joke just by saying something as concisely as possible (even if the content isn’t humorous).

Like you said, whether or not the visuals provide any value is a completely different line of discussion.

Post reply on HN