Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

U.S. universities, rich in data, struggle to capture its value, study finds

newsroom.ucla.edu

Post reply on HN
Print view View original

81–90 of 97 posts

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#81

Post by gchallen »

analog31 wrote: post #77
gchallen wrote: post #37

Working in a top-tier computer science department, I find our ability to answer basic questions about the health of our degree program fairly troubling. I think non-academics may be surprised by how much we don't know, and how little useful and continuous data analysis is taking place. For example: What is our retention rate? Meaning, what percentage of students who start our degree programs complete it. A fairly sta…

Perhaps one problem is that every college has its own bespoke curriculum and processes, so every data problem is a "little data" problem. Of course there are lots of rationalizations for why every program needs to be unique and special, but does it really benefit the students? A similar problem in medicine: Every clinic system has a unique set of business processes, and a custom build of Epic. Granted the clinics are…

> Of course there are lots of rationalizations for why every program needs to be unique and special, but does it really benefit the students?

Of course not. What would benefit the students would be having a lot more standardization so that we compare ideas and approaches and determine what works. But the problem with standardized evaluation is that half of the programs suddenly discover that they aren't in the top half—as most of them had previously thought. This seems like more or less what happened to standardized testing in K–12 education.

But, in the context of a specific curriculum, having no idea what is happening and therefore no way to improve your bespoke curriculum is even worse than just deciding to do things your own way.

And it's worse than medicine, because at least they have some common metrics for what it means to be healthy. Whereas, faculty get to assign grades however they want! Imagine you ran a diet study where you both controlled the meals and got to reposition the numbers on the scale at will.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#82

Post by gchallen »

nobrains wrote: post #62
gchallen wrote: post #37

Working in a top-tier computer science department, I find our ability to answer basic questions about the health of our degree program fairly troubling. I think non-academics may be surprised by how much we don't know, and how little useful and continuous data analysis is taking place. For example: What is our retention rate? Meaning, what percentage of students who start our degree programs complete it. A fairly sta…

OK, then what? Then you find out why retention is low. Then you brainstorm ideas to increase retention. Then you attempt to apply those ideas. It is at this point that the person responsible for applying those ideas says "been there done that". Point is, most data dashboards are non actionable. The challenge is to create a good actionable dashboard (i.e. if values cross a certain threshold, then the user should take…

> Point is, most data dashboards are non actionable.

Maybe you find out that one specific class or set of classes is responsible for a lot of people leaving the program. So you zero in on that part of the curriculum and improve it. Sounds pretty actionable to me. We've actually done similar things on a smaller scale (week by week) to improve student success in my course.

But of course you don't know anything until you do the analysis.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#83

Post by porknubbins »

civilized wrote: post #28

As a data scientist, I think most data is useless, but there is an addictive, video game-like quality to throwing lifeless spreadsheets into a machine and having colorful visualizations come out. It's kind of like a very boring video game for adults that makes them feel like they're working, when they're actually just enjoying colorful abstract shapes and colors. To be honest, this is probably a sizable piece of why…

You perfectly described why I became disillusioned with any kind of data science job. The feeling that most of the time data is used to support preexisting bias and can be arranged to say anything you want. Interesting that you know this and enjoy it despite that.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#84

Post by wolverine876 »

fullsend wrote: post #74
wolverine876 wrote: post #63

> The study’s authors contend that universities have been slower than organizations in other economic sectors to create senior-level positions focused on data quality, strategy, governance and privacy matters. Universities are not an economic sector; they aren't profit-making enterprises. They are knowledge- and education-making enterprises. Do they need more employees who aren't creating knowledge and educating? My…

That’s exactly what universities are these days. They sell a signal that says “this worker can do the bare minimum you need” called a Degree. We make the worker get certified themselves, but this is the purpose. To act like modern Universities do anything else today is disingenuous in my opinion. It’s like saying people who work in finance are there to allocate capital efficiently. No, they’re there to make money. Th…

It's a popular meme but IMHO does not reflect university-level analysis. That exists, but reductionist arguments merely take the worst of someone or something and blow that up to 100%.

For example, everyone feels self-interest, but the reductionist claim it that it's all we are. A simple look at the evidence shows that it's manifestly untrue; people are much more than self-interested.

Similarly universities do and are much more. I know university students and they are learning a lot - I'm very impressed with the thought and creative effort put into the cirriculum.

The reductionists really hurt themselves and people who listen to them. They wall themselves off from all the good in the world, including education.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#85

Post by roenxi »

nobrains wrote: post #62
gchallen wrote: post #37

Working in a top-tier computer science department, I find our ability to answer basic questions about the health of our degree program fairly troubling. I think non-academics may be surprised by how much we don't know, and how little useful and continuous data analysis is taking place. For example: What is our retention rate? Meaning, what percentage of students who start our degree programs complete it. A fairly sta…

OK, then what? Then you find out why retention is low. Then you brainstorm ideas to increase retention. Then you attempt to apply those ideas. It is at this point that the person responsible for applying those ideas says "been there done that". Point is, most data dashboards are non actionable. The challenge is to create a good actionable dashboard (i.e. if values cross a certain threshold, then the user should take…

In my experience it is generally worse than that. If a program has a retention rate of, say, 30% then it isn't like that is going to come as a surprise. Data generally reveals things that are very obvious. It is easier to read a situation by talking to a few people and asking simple questions. Which raises further questions about why a data-driven approach is needed.

The value in data driven approaches is high, but it takes a rare person to figure out why. Traditionally data has actually been a communication tool for things that are already known. That isn't at all how people expect it to be used, everyone seems to anticipate it is used to make decisions.

An organisation resisting data is bad news because it will struggle to talk about things that everyone knows to be true.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#86

Post by musicale »

Eumenes wrote: post #14

We need more ad-tech startups for the university/higher education industry NOW

> We need more ad-tech startups for the university/higher education industry NOW

I had the same thought - intrusive analytics and a focus on exploiting and monetizing client data seem to have damaged the web, made gaming less fun, and made everything user-hostile and creepy. Do we really want universities to go down that path?

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#87

Post by pcrh »

blthree wrote: post #54
pcrh wrote: post #19

Universities already waste vast sums of money of vice-provosts, middle management, administration, etc. There's no need to encourage them to start treating their mission as one of data mining in order to "capture" more value from the students.

You have a point about the amount of money spent on admin at Universities, but I think you (and most commenters here) don't really realize how bad the data situation is at a lot of major universities. If we want to streamline admin overhead at public universities, there's an incredible amount of time and money wasted due to extremely inefficient use/access to data. I work as a BI dev at a large Big 10 school (50k stu…

If such proposals would result in reduced expenditure on administration, it would obviously be beneficial.

However, I note that your comments largely remark on an insufficient attention given to administration.

The trend in Universities over the past few decades has largely been to increase administration efforts, without that having a notable benefit on student outcomes.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#88

Post by newsclues »

gchallen wrote: post #37

Working in a top-tier computer science department, I find our ability to answer basic questions about the health of our degree program fairly troubling. I think non-academics may be surprised by how much we don't know, and how little useful and continuous data analysis is taking place. For example: What is our retention rate? Meaning, what percentage of students who start our degree programs complete it. A fairly sta…

We don't ask questions we don't want the answers to (be public).

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#89

Post by autoexec »

Jibbedeyeah wrote: post #44
nonrandomstring wrote: post #8

There's a troubling tone behind this article. It's a prophecy that demands to be fulfilled. Starting with the premise "Data is useful", it proceeds to pick at all the ways we've failed to make it useful... and we're damned well going to make it useful if it kills us! Maybe, just maybe (for those that dabble in the sceptical, explanatory game we call science) it might be that "bare data" has little use within certain…

One man's troubling tone is another man's dystopic reality. I read a comment on here months ago about someone who enabled a hotel chain to process the data coming from the motion activated lights in every hotel. The chain was able to track workers with this data and cracked down on people taking too long of breaks. And now a personal anecdote. The business I used to work at had an ID badge door you had to use to ente…

This is the problem. The data could be used for good: to improve the education being provided to the students, and to improve conditions for those working for the university, but history tells that most of the time it will end up being used against/at the expense of those same people and that there will be a much stronger profit incentive to do the wrong thing rather than the right one.

Re: U.S. universities, rich in data, struggle to capture its value, study finds

#90

Post by bluGill »

MaxBarraclough wrote: post #79
trop wrote: post #78

Earlier quoted context omitted.

An anecdote from someone in IT at a major university: The registrar has two employees whose sole job is to write SQL queries. These are to answer basic questions such as, "How many undergraduates are currently enrolled." And it turns out that this is a nearly impossible question to which to give a definitive answer.

Shouldn't the finance folks know that kind of thing pretty definitively?

What does it mean to be enrolled? What if someone takes a semester off? What if they are part time? What if they are on a foreign exchange program this year? What if ...

The above are the type of questions that can result in different definitions of enrolled.

Post reply on HN