Why you should not pirate Google’s geo APIs • Hacker News Board

Re: Why you should not pirate Google’s geo APIs

#101

Post by skyebook » Tue, Sep 10, 2013, 3:33 PM UTC

celoyd wrote: post #97

skyebook wrote: post #70

Earlier quoted context omitted.

About three years ago I got in contact with some very nice people at NASA and/or the USGS (I don't recall which agency ended up being the end-point) for what, at the time, was some of the best data available from the joint ASTER missions they ran with Japan. The process was essentially to fill out a form and send them a new, unopened hard drive. What I got back a few weeks later was about 125GB of high resolution Geo…

I’ve worked with NASA and USGS a fair deal both for fun and professionally, and this is typical. They are extremely competent and sincerely want you to have good data. They are also hampered by the bureaucratic limits of any large organization. So it’s like working with a large, well-run business that’s hired a lot of the best people in its field and is working on good problems, yet is large enough that it can’t move…

With regards to your political argument, its something I've often thought about because it can go either way. There's the loaded language approach as you mention, but there's also the possibility of the hypothetical member of the House or Senate saying "And apparently we’re paying the Geological Survey $N million a year to run a web server that no one is making use of and funding data that no one seems to care about."

This would be a hard argument to defend against as it directly hits the value derived from the service rather than the idea that the service itself exists. In either case, I couldn't agree with you more that the civilian agencies really get a raw deal (and I'm sure its only compounded by the mostly stagnant level of people graduating with STEM degrees[1])

[1] http://www.nsf.gov/statistics/digest12/stem.cfm#3

Re: Why you should not pirate Google’s geo APIs

#102

Post by Doctor_Fegg » Tue, Sep 10, 2013, 3:54 PM UTC

hackula1 wrote: post #100

Doctor_Fegg wrote: post #80

Earlier quoted context omitted.

Building your own elevation API is absolutely trivial; I don't know why anyone would even want to pirate Google's API for large-scale usage. Firstly, the data is freely available. It's NASA's SRTM [1], downloadable from a zillion mirror sites. Then just take some code to calculate a lat/long offset and find the right position in the right tile. Bob Osola has some easily portable PHP for this if you need it [2]. Free…

Building a robust elevation api is definitely not trivial for the average dev using the google api. If someone thought the only way to get elevations was through google then they definitely will not know how to find and compile the data, then write the TIN (triangulated irregular network) algos necessary to appropriately estimate the elevation of an arbitrary point. No, it is not that hard, since tools like Grass can…

Worrying about TIN is the 10% case. Simple interpolation within SRTM raster tiles is all that's needed for 90% of cases (IMHO, obviously).

Re: Why you should not pirate Google’s geo APIs

#103

Post by Sprint » Tue, Sep 10, 2013, 4:05 PM UTC

Doctor_Fegg wrote: post #80

joshma wrote: post #58

Earlier quoted context omitted.

I don't understand this feeling of self-entitlement - who are we to judge whether or not an API has been offered "in a reasonable manner?" I'm guessing it's very resource-consuming to collect, process, and maintain the data necessary for geolocation APIs (especially given the upstream providers). It'd be one thing for Google to be deceptive about it, offer a free service, and try to upsell you every point along the w…

Building your own elevation API is absolutely trivial; I don't know why anyone would even want to pirate Google's API for large-scale usage. Firstly, the data is freely available. It's NASA's SRTM [1], downloadable from a zillion mirror sites. Then just take some code to calculate a lat/long offset and find the right position in the right tile. Bob Osola has some easily portable PHP for this if you need it [2]. Free…

How do you know it is just SRTM?

Re: Why you should not pirate Google’s geo APIs

#104

Post by hackula1 » Tue, Sep 10, 2013, 4:24 PM UTC

Doctor_Fegg wrote: post #102

hackula1 wrote: post #100

Earlier quoted context omitted.

Building a robust elevation api is definitely not trivial for the average dev using the google api. If someone thought the only way to get elevations was through google then they definitely will not know how to find and compile the data, then write the TIN (triangulated irregular network) algos necessary to appropriately estimate the elevation of an arbitrary point. No, it is not that hard, since tools like Grass can…

Worrying about TIN is the 10% case. Simple interpolation within SRTM raster tiles is all that's needed for 90% of cases (IMHO, obviously).

I would not think interpolation would be that much easier than a creating a TIN. I have not looked at the dataset though, so it might be "pre-interpolated" (probably using a TIN if I had to guess), in which case this would be pretty easy with something like pyqgis on a flask server or whatever. The entire api could basically just be:

    @app.route("/elevation")
    def elevation():
      fileName = "/path/to/raster/srtmRaster.tif"
      fileInfo = QFileInfo(fileName)
      baseName = fileInfo.baseName()
      srtmRaster = QgsRasterLayer(fileName, baseName)
      res, ident = srtmRaster.identify(QgsPoint(request.form['lat'], request.form['lon']))
      return ident[0]

I can test this out and throw it on github with some polish if anyone is interested.

Re: Why you should not pirate Google’s geo APIs

#105

Post by Doctor_Fegg » Tue, Sep 10, 2013, 4:33 PM UTC

Sprint wrote: post #103

Doctor_Fegg wrote: post #80

Earlier quoted context omitted.

Building your own elevation API is absolutely trivial; I don't know why anyone would even want to pirate Google's API for large-scale usage. Firstly, the data is freely available. It's NASA's SRTM [1], downloadable from a zillion mirror sites. Then just take some code to calculate a lat/long offset and find the right position in the right tile. Bob Osola has some easily portable PHP for this if you need it [2]. Free…

How do you know it is just SRTM?

Sorry, should have made myself clearer. I'm not saying that Google just uses SRTM - I'm sure they throw lots of other sources and general Google Magick in there. But for most people in most places, SRTM is good enough.

Re: Why you should not pirate Google’s geo APIs

#106

Post by ianstallings » Tue, Sep 10, 2013, 4:50 PM UTC

TallGuyShort wrote: post #2

Here's another reason that I wish was more prevalently discussed: it's not okay to violate Terms of Service anyway. Don't accept the terms? Don't use the service. We'd all have a field day if "big brother" violated the terms of an open-source license, but the freedom we enjoy in the community comes with a responsibility to honor our agreements.

I've said the same in the past and usually it's to the business managers or executives. Too often they are willing to conveniently overlook licensing or service restrictions in order to get what they want or need.

A specific example - I had to convince executives that we needed to pay MPEG LA because we were using FFMPEG to encode and decode using 264 on our servers. I only convinced them after a long debate and contacting our legal department to explain the risk. It had very little to do with my personal views on licensing or patents and everything to do with professionalism. A lot of the other software developers thought this was wasting money and time, and paying off people that were somehow evil because they charge large sums for using their patented software. My stance has always been if yo don't like the terms use something else and avoid the ethical dilemma. It's just plain easier and avoids some risk.

Re: Why you should not pirate Google’s geo APIs

#107

Post by ianstallings » Tue, Sep 10, 2013, 4:52 PM UTC

Killswitch wrote: post #3

TallGuyShort wrote: post #2

Here's another reason that I wish was more prevalently discussed: it's not okay to violate Terms of Service anyway. Don't accept the terms? Don't use the service. We'd all have a field day if "big brother" violated the terms of an open-source license, but the freedom we enjoy in the community comes with a responsibility to honor our agreements.

I just wish people would stop being so self-entitled period.

As my developer friend likes to say - this is why we can't have nice things.

Re: Why you should not pirate Google’s geo APIs

#108

Post by wpietri » Tue, Sep 10, 2013, 6:00 PM UTC

easy_rider wrote: post #89

wpietri wrote: post #62

Earlier quoted context omitted.

I agree in part with your point, but I think Google has pulled a bit of a bait and switch here. For many years, they were conspicuously open and friendly, as you would expect from a company run by engineers. That got them a lot of geek love, and a big boost to their early adoption efforts. I also think it had the effect of stunting the growth of alternatives. (E.g., Google Reader.) Now they are running things more in…

You've missed the point that Google has always been an advertisement company.

I'm aware of that. But they also came out of an academic background, used a lot of open source, published many papers, and built an engineering-focused culture, not a business-focused one.

That is, sadly, changing. But it's not a necessary change.

Re: Why you should not pirate Google’s geo APIs

#109

Post by reginaldjcooper » Tue, Sep 10, 2013, 6:16 PM UTC

mikeash wrote: post #92

reginaldjcooper wrote: post #76

Earlier quoted context omitted.

I've noticed typically I agree with your views, but on this case we diverge. I think it's reasonable to assume that when the service cannot be accessed before the accessor clicks "agree", and the accessor clicks "agree", that constitutes a valid contractual agreement. It's no more silly than mailing in a credit card application and being bound by those terms. The fundamental attributes are present. There is a contrac…

I'm not a lawyer, so take the following with much salt. Furthermore, I'm arguing more from an "ideal legal world" viewpoint rather than the current American legal environment. A contract must consist of: 1. A "meeting of the minds." This means both sides come to a common understanding of what's in the contract. Basically, it means there's an offer by one side and acceptance by the other without changes. 2. Intention…

They probably are fully aware that nobody reads the clickwrap, that's an interesting point. It would be interesting to hear what a lawyer says about that. I don't think it's a truly reasonable defense and I wouldn't want to rely on it. (If we're thinking "ideal legal world", then I'd like clickwraps to be human-readable and people to actually read them.)

But I think there's a very clear distinction between a simple agreement with the original and with altering the DOM. If you are clever enough to alter the DOM you definitely know that the alterations are not going to be sent back to the server. You also know that agreeing will let you use the service because the server will think you agreed to the original. That seems like a clear-cut case of fraud to me. I don't see why an agreement to the original contract is invalidated because you weren't offered the opportunity to make a counter-offer (I agree that's a shitty situation, though). That's why I think it's like a contract in the mail; you get an offer and you can agree to that or not. You can make a counter-offer but it will be binned.

Consideration isn't solely concerned with monetary purchases, is it? I imagine they can construe your incoming data as valuable. It would be interesting to know if sites like Reddit can claim user interaction has inherent value for the site and for the users. I guess the EULA would be granting you license to use the software, but would not give any additional value to the developer so consideration does seem to invalidate that.

You're probably right that a ToS is going to be hard to uphold in civil court.

Anyway, thank you for elaborating, I'm glad to get a better understanding of your perspective and contract law.

Re: Why you should not pirate Google’s geo APIs

#110

Post by chrisbroadfoot » Tue, Sep 10, 2013, 6:47 PM UTC

hackula1 wrote: post #99

crymer11 wrote: post #65

Earlier quoted context omitted.

I would offer that any potential or actual user of the API is an apt judge of whether the API has been offered "in a reasonable manner." As it currently stands, one has the option of 2,500 requests per day for free or a minimum of $10,000 [1] and a limit of 100,000 requests per day. Is it unreasonable to wish for at least one option between the two current choices of $0 and >=$10,000? The parent isn't being self-enti…

I actually talked to Google yesterday. Minimum tier is 100k per day at $17.5k per year. Very fair price (If you think that is expensive, then you clearly do not work in commercial GIS, where licensing of software and data is insanely expensive). At the same time, they force you to use the data on a Google Map component (yes, even for the paid accounts), and while their api is acceptable, their js map control is sever…

    their js map control is severely lacking in features compared to something like leaflet, openlayers, or esri's js component.

I work on the Google Maps API. I'd love to hear more about what features you're missing.