Data Science: Reality Doesn't Meet Expectations • Hacker News Board

Re: Data Science: Reality Doesn't Meet Expectations

#131

Post by ehnto » Wed, Apr 08, 2020, 11:17 AM UTC

kqr wrote: post #33

hectormalot wrote: post #5

> Moreover, you may quickly realize much of this work is repetitive and while time-consuming, is “easy”. In fact, most analyses involve a great deal of time to understand the data, clean it and organize it. You may spend a minimal amount of time doing the “fun” parts that data scientists think of: complex statistics, machine learning and experimentation with tangible results. This. Universities and online challenges…

I took a data visualisation class in uni that handled this really cleverly. The second assignment sounded very easy. The teacher provided links to the sources where we could find data. Most people figured that with such a simple assignment (not significantly harder than the first one, which was also easy-ish) they could put off doing it until the last moment. Most people failed. This real world data needed hours upon…

Edit: the comment I replied to has since been edited to show how the teacher understood what he was doing, and how he made it teachable lesson and not a punitive one.

Not to miss the point, but I don't think "All my students failed" is the mark of a good teacher. It sounds like the teacher failed to prepare their students for the nature of the assignment. Perhaps he was surprised as they were when they all failed, as I doubt failing most of his class was his intention.

Re: Data Science: Reality Doesn't Meet Expectations

#132

Post by gcatalfamo » Wed, Apr 08, 2020, 11:38 AM UTC

ehnto wrote: post #131

kqr wrote: post #33

Earlier quoted context omitted.

I took a data visualisation class in uni that handled this really cleverly. The second assignment sounded very easy. The teacher provided links to the sources where we could find data. Most people figured that with such a simple assignment (not significantly harder than the first one, which was also easy-ish) they could put off doing it until the last moment. Most people failed. This real world data needed hours upon…

Edit: the comment I replied to has since been edited to show how the teacher understood what he was doing, and how he made it teachable lesson and not a punitive one. Not to miss the point, but I don't think "All my students failed" is the mark of a good teacher. It sounds like the teacher failed to prepare their students for the nature of the assignment. Perhaps he was surprised as they were when they all failed, as…

You are being downvoted but you are exactly on point. If some fail they may be bad students, but if the majority of my students fail they're not bad students, it is me who is a bad teacher.

Re: Data Science: Reality Doesn't Meet Expectations

#133

Post by rezeroed » Wed, Apr 08, 2020, 11:45 AM UTC

ehnto wrote: post #131

kqr wrote: post #33

Earlier quoted context omitted.

I took a data visualisation class in uni that handled this really cleverly. The second assignment sounded very easy. The teacher provided links to the sources where we could find data. Most people figured that with such a simple assignment (not significantly harder than the first one, which was also easy-ish) they could put off doing it until the last moment. Most people failed. This real world data needed hours upon…

Edit: the comment I replied to has since been edited to show how the teacher understood what he was doing, and how he made it teachable lesson and not a punitive one. Not to miss the point, but I don't think "All my students failed" is the mark of a good teacher. It sounds like the teacher failed to prepare their students for the nature of the assignment. Perhaps he was surprised as they were when they all failed, as…

"Of course, the teacher knew this, gave bonus points to the ones who did start in time, and then extended the deadline as he had expected to from the start."

Re: Data Science: Reality Doesn't Meet Expectations

#134

Post by thablackbull » Wed, Apr 08, 2020, 11:49 AM UTC

gcatalfamo wrote: post #132

ehnto wrote: post #131

Earlier quoted context omitted.

Edit: the comment I replied to has since been edited to show how the teacher understood what he was doing, and how he made it teachable lesson and not a punitive one. Not to miss the point, but I don't think "All my students failed" is the mark of a good teacher. It sounds like the teacher failed to prepare their students for the nature of the assignment. Perhaps he was surprised as they were when they all failed, as…

You are being downvoted but you are exactly on point. If some fail they may be bad students, but if the majority of my students fail they're not bad students, it is me who is a bad teacher.

> Of course, the teacher knew this, gave bonus points to the ones who did start in time, and then extended the deadline as he had expected to from the start.

I think what he meant is they 'failed' to get it completed on time and it was meant as a teaching lesson.

Re: Data Science: Reality Doesn't Meet Expectations

#135

Post by MaxBarraclough » Wed, Apr 08, 2020, 11:55 AM UTC

rezeroed wrote: post #133

ehnto wrote: post #131

Earlier quoted context omitted.

Edit: the comment I replied to has since been edited to show how the teacher understood what he was doing, and how he made it teachable lesson and not a punitive one. Not to miss the point, but I don't think "All my students failed" is the mark of a good teacher. It sounds like the teacher failed to prepare their students for the nature of the assignment. Perhaps he was surprised as they were when they all failed, as…

"Of course, the teacher knew this, gave bonus points to the ones who did start in time, and then extended the deadline as he had expected to from the start."

Similarly, it's not good teaching practice to trivialize deadlines.

Re: Data Science: Reality Doesn't Meet Expectations

#136

Post by Grimm1 » Wed, Apr 08, 2020, 12:12 PM UTC

MaxBarraclough wrote: post #135

rezeroed wrote: post #133

Earlier quoted context omitted.

"Of course, the teacher knew this, gave bonus points to the ones who did start in time, and then extended the deadline as he had expected to from the start."

Similarly, it's not good teaching practice to trivialize deadlines.

Yeah plenty of time for workplaces to do that for you. I can count on my hand the number of times something has been a hard deadline. This teacher taught a valuable lesson usable for the rest of the student's carreers and "the most students shouldn't fail mentality" has led to professors I know personally questioning the caliber of student they are receiving and this is a top 30 program I'm referring to. More people should fail, maybe they'd start treating things seriously and the problem of underqualified technical applicants would resolve itself.

Re: Data Science: Reality Doesn't Meet Expectations

#137

Post by wenc » Wed, Apr 08, 2020, 12:28 PM UTC

wenc wrote: post #51

Data science is correctly valued when you realize how relatively unimportant it is. It is a small cog in a larger machinery (or at least it ought to be). You see, decision-making involves (1) getting data, (2) summarizing and predicting, and (3) taking action. Continuous decision-making -- the kind that leads to impact -- involves doing this repeatedly in a principled fashion, which means creating a system around the…

I should add that programmers have the crucial albeit boring role of creating CRUD front ends (forms) for data input.

That is a akin to a sensor input, and one that is surprisingly important. Without a good CRUD form, data either doesn’t get entered at all, or is entered in crude, unvalidated ways like as loose Excel files with formatting that is all over the place.

Re: Data Science: Reality Doesn't Meet Expectations

#138

Post by zeveb » Wed, Apr 08, 2020, 12:52 PM UTC

> internally, you can make inroads supporting stakeholders with evidence for their decisions!

The problem is that this can all too easily become motivated reasoning: one provides a stakeholder with support for the decision he already made. From his point of view, this is a valuable service, but it does the organisation a disservice: decisions should be made after considering the data, rather than consider only those data which support a decision.

Also, while ethical issues certainly arise, I think that Greyball is not a good example. Uber evading police enforcing the taxi monopoly is no more unethical than the Underground Railroad evading fugitive-slave agents. The taxi monopoly is itself unethical, and evading it increases the common good.

Re: Data Science: Reality Doesn't Meet Expectations

#139

Post by xaedes » Wed, Apr 08, 2020, 12:57 PM UTC

gcatalfamo wrote: post #132

ehnto wrote: post #131

Earlier quoted context omitted.

Edit: the comment I replied to has since been edited to show how the teacher understood what he was doing, and how he made it teachable lesson and not a punitive one. Not to miss the point, but I don't think "All my students failed" is the mark of a good teacher. It sounds like the teacher failed to prepare their students for the nature of the assignment. Perhaps he was surprised as they were when they all failed, as…

You are being downvoted but you are exactly on point. If some fail they may be bad students, but if the majority of my students fail they're not bad students, it is me who is a bad teacher.

Failing is a form of learning. Enabling to fail (preferably in a safe way) is very valuable for learning.

Re: Data Science: Reality Doesn't Meet Expectations

#140

Post by ehnto » Wed, Apr 08, 2020, 1:02 PM UTC

rezeroed wrote: post #133

ehnto wrote: post #131

Earlier quoted context omitted.

Edit: the comment I replied to has since been edited to show how the teacher understood what he was doing, and how he made it teachable lesson and not a punitive one. Not to miss the point, but I don't think "All my students failed" is the mark of a good teacher. It sounds like the teacher failed to prepare their students for the nature of the assignment. Perhaps he was surprised as they were when they all failed, as…

"Of course, the teacher knew this, gave bonus points to the ones who did start in time, and then extended the deadline as he had expected to from the start."

That wasn't in the original comment. It has been edited since I replied, which is fine. I do it all the time, sometimes you miss that someone replied during your editing.