Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yale censored a student's course catalog site. I made an unblockable replacement

haufler.org

Post reply on HN
Print view View original

21–30 of 206 posts

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#21

Post by skywhopper »

EGreg wrote: post #15
ToastyMallows wrote: post #10

Someone please correct me if I'm wrong. The extension: > (3) not scraping or collecting Yale’s data Yet in the code[0], there's a function called getRatingsForCourse() that makes an ajax call to url: " https://ybb.yale.edu/courses/" + id. How is this not scraping? Am I missing something here? [0]: https://github.com/seanhaufler/banned-bluebook/blob/master/e...

Scraping involves storing somethjng on a server. I am not sure a user agent can be cited for copyright infringement. Perhaps caching in violation of a server's policy can be considered that, I am not sure. Is there any precdent for caching to be considered scraping and storing by the developer of the client software? If not, I have an interesting idea for an offline travel app :)

I don't know the full etymology, but I first heard the term "screen scraping" about 15 years ago in the context of writing a web application that interfaced with a mainframe application. The mainframe database was not directly accessible to the webapp platform, so instead a session was opened to the mainframe over telnet, and the web application itself issued commands over that telnet session, parsed the terminal output, and dynamically generated a web UI that displayed the data or form fields or what have you.

Scraping these days may more often be a means of extracting data to an offsite cache/database, but ad-hoc on-demand scraping is consistent with the definition.

Regardless, this doesn't really matter. I don't see how any restrictions on "scraping" can possibly be enforced. Screen reader programs that allow visually impaired individuals to browse the web are "scraping". Mobile browsers that reformat pages on the fly to optimize their display on a small screen are "scraping".

Ad-hoc user-initiated scraping of this sort is really no different than just browsing the web directly. So whether one wants to argue that "scraping" is or isn't happening here, it hardly matters to whether this is ethical, and should not matter to whether it is legal (although on that topic, I'm guessing different judges would see it differently).

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#22

Post by pingswept »

ToastyMallows wrote: post #10

Someone please correct me if I'm wrong. The extension: > (3) not scraping or collecting Yale’s data Yet in the code[0], there's a function called getRatingsForCourse() that makes an ajax call to url: " https://ybb.yale.edu/courses/" + id. How is this not scraping? Am I missing something here? [0]: https://github.com/seanhaufler/banned-bluebook/blob/master/e...

That function just retrieves the rating for one course, based on the course ID that the user has retrieved by searching for some courses. It is certainly requesting data from Yale's server, but I wouldn't call it scraping. To qualify as scraping (at least as I've done it), you'd have to iterate over all possible IDs and store the records in a database.

For example, consider this basketball scraper: https://github.com/andrewgiessel/basketballcrawler This builds a list of all the current NBA players and then pulls stats on all of them, saving them in a giant JSON file, IIRC. That's scraping.

What Sean Haufler is doing is not massively iterated, and it's linked in time and quantity to human-generated queries. Seems different to me.

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#23

Post by throwawaykf03 »

lionhearted wrote: post #12

When I first left America, I disliked and felt disillusioned with the country. I've now spent more of my adult life outside the States than in the U.S., and I've also come to the opinion that the U.S. is one of the most amazing places in the world. Sometimes it's hard to point to exactly why America is so amazing. And the answer, maybe, is on display here. An undergraduate student at an elite university just innovati…

You're obviously delusional because something something NSA war on drugs war on terror civil liberties TSA government.

(Also, it's not just Asian countries, there are some Western countries where your life would be ruined as well.)

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#24

Post by cliveowen »

lionhearted wrote: post #12

When I first left America, I disliked and felt disillusioned with the country. I've now spent more of my adult life outside the States than in the U.S., and I've also come to the opinion that the U.S. is one of the most amazing places in the world. Sometimes it's hard to point to exactly why America is so amazing. And the answer, maybe, is on display here. An undergraduate student at an elite university just innovati…

You could read it that way, or you could say that this miniscule altercation between a student and the dean of an Ivy League university, both part of an elite group that knows little about the real struggles the rest of the world faces on a daily basis, has been elevated to something important while its not and it's been given way too much attention than it deserves. How many college campuses and universities in the world struggle with real problems with real consequences to society like female students getting raped in dark alleys and fraud/bribery/plagiarism and never get the attention they deserve?

The fact that this inconsequential squabble gets discussed so much and gets portrayed as the fight between David and "Goliath The Man", instead of being seen as a public tantrum being thrown by a spoiled kid living a cushioned life inside a cocoon made out of hopes and dreams, makes me sick to my stomach.

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#25

Post by ZoFreX »

lionhearted wrote: post #12

When I first left America, I disliked and felt disillusioned with the country. I've now spent more of my adult life outside the States than in the U.S., and I've also come to the opinion that the U.S. is one of the most amazing places in the world. Sometimes it's hard to point to exactly why America is so amazing. And the answer, maybe, is on display here. An undergraduate student at an elite university just innovati…

I'm British and I did things like this constantly throughout school and university.

Then again, people do keep asking if I am American...

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#26

Post by morgante »

ritchiea wrote: post #16
scep12 wrote: post #11

Earlier quoted context omitted.

He seems smart enough to get a good job regardless. The small win of publicity probably isn't worth the loss of a diploma, especially for a senior with (presumably) only one semester left. Given the time and money he's invested in his degree, I'm not sure he'd agree that it's a "win-win."

What is the meaningful difference between a Yale degree and 7 semesters of a Yale education? If your answer is "some employers wouldn't consider you as an applicant without the proper degree" do you want to work for that sort of employer?

> If your answer is "some employers wouldn't consider you as an applicant without the proper degree" do you want to work for that sort of employer?

Unfortunately yes. A lot of the top technology companies are still saddled with the notion that academics matter.

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#27

Post by higherpurpose »

As long as Google doesn't ban the extension from the Chrome Store and doesn't completely block installing 3rd party extensions to Chrome in schools or something.

I actually don't find that unlikely these days. They've already started to make it hard to install it from other sources. I could see them "sweetening the deal" for ChromeOS/Chrome in schools to not be able to install external extensions at all.

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#28

Post by JohnTHaller »

babs474 wrote: post #19

Very cool, I love this approach as it is very similar to how my (now defunct) unedditreddit app worked and how a clever app called socialfixer for facebook works. Two thoughts. 1. You are not unblockable. My app was removed from the chrome app store after reddit complained, no review process, no appeals, good luck even communicating with google. Installing an app not from the appstore is almost impossible for a non-s…

I believe unedditreddit was removed due to trademark issues, but I could be wrong. As this extension does not have any of Yale's trademarks within and (demonstrably, based on source) does not impact the site, Yale should not have a leg to stand on.

But your point about doing it in Firefox is a very good one. Firefox's extension addons site is much more democratic. And side-loading extensions within Firefox outside of addons.mozilla.org is something even an inexperienced user can accomplish.

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#29

Post by dasil003 »

cliveowen wrote: post #24
lionhearted wrote: post #12

When I first left America, I disliked and felt disillusioned with the country. I've now spent more of my adult life outside the States than in the U.S., and I've also come to the opinion that the U.S. is one of the most amazing places in the world. Sometimes it's hard to point to exactly why America is so amazing. And the answer, maybe, is on display here. An undergraduate student at an elite university just innovati…

You could read it that way, or you could say that this miniscule altercation between a student and the dean of an Ivy League university, both part of an elite group that knows little about the real struggles the rest of the world faces on a daily basis, has been elevated to something important while its not and it's been given way too much attention than it deserves. How many college campuses and universities in the…

Taking that line of reasoning to its conclusion, there can only be one worst thing that happened to someone in the history of the world, and every other person's problems pale in comparison and are therefore inconsequential and should be brushed off as first-world problems.

Re: Yale censored a student's course catalog site. I made an unblockable replacement

#30

Post by blibble »

ZoFreX wrote: post #25
lionhearted wrote: post #12

When I first left America, I disliked and felt disillusioned with the country. I've now spent more of my adult life outside the States than in the U.S., and I've also come to the opinion that the U.S. is one of the most amazing places in the world. Sometimes it's hard to point to exactly why America is so amazing. And the answer, maybe, is on display here. An undergraduate student at an elite university just innovati…

I'm British and I did things like this constantly throughout school and university. Then again, people do keep asking if I am American...

here in the UK the Education Act 1986 makes it a duty of a University to safeguard its members freedom of speech (within the law), and makes it unlawful for a University to do what Yale did here.

http://www.legislation.gov.uk/ukpga/1986/61/section/43

Post reply on HN