Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google launches Public Datasets program

cloud.google.com

Post reply on HN
Print view View original

51–60 of 98 posts

Re: Google launches Public Datasets program

#51

Post by [deleted] »

philovivero wrote: post #45
fhoffa wrote: post #24

Earlier quoted context omitted.

2008. I remember those days. GMail was still in beta and ' a big area of concern was "Why won't Google just shut this down when the going gets rough?" ' 8 years later GMail has a billion users, and not going anywhere but up. I haven't been to the future (yet), but I'll happily take any bet you have against BigQuery. Disclaimer: I'm Felipe Hoffa, and I work at Google. ( https://twitter.com/felipehoffa )

Can you give me an update on Wave and Orkut? I never use this Reader thing, but I hear others do.

[deleted]

Re: Google launches Public Datasets program

#53

Post by vgt »

philovivero wrote: post #45
fhoffa wrote: post #24

Earlier quoted context omitted.

2008. I remember those days. GMail was still in beta and ' a big area of concern was "Why won't Google just shut this down when the going gets rough?" ' 8 years later GMail has a billion users, and not going anywhere but up. I haven't been to the future (yet), but I'll happily take any bet you have against BigQuery. Disclaimer: I'm Felipe Hoffa, and I work at Google. ( https://twitter.com/felipehoffa )

Can you give me an update on Wave and Orkut? I never use this Reader thing, but I hear others do.

I'm happy to respond to every single person that makes this argument in perpetuity :)

There's a difference between experimental free services and Enterprise-grade SLA'd SLO'd fully-supported paid services with a very clear and recorded deprecation policy and an army of customers with contracts and full support from Google CEO and chairman.

Re: Google launches Public Datasets program

#54

Post by awqrre »

vgt wrote: post #48
awqrre wrote: post #40

Earlier quoted context omitted.

Google employees using something internally doesn't mean that it won't go away for everyone else... Also, they could decide to honor existing contracts and stop issuing new ones...

BigQuery is a truly fully-managed service. This means that SREs are on call 24/7. It's not software that one downloads. Hence it really doesn't make a lot of sense to have it available to just a few folks, incremental overhead is low. I'd love to hear industry standards around deprecation of paid services. This is really a way of doing business in the entire world, and I'm not sure why Google is being held to an unre…

I was not blaming anybody, it's possible that a service would be more valuable to a company if it's kept internally.

Re: Google launches Public Datasets program

#55

Post by Veratyr »

vgt wrote: post #5

One large difference between this program and alternative programs is that data already resides in Google BigQuery: - You do not need to spin up a database to work with BigQuery - You can simply start writing SQL on top of BigQuery - You may leverage Dataflow and MapReduce connectors to work with this data directly in Hadoop, Spark, or Dataflow - BigQuery has a free tier - one Terabyte of data processed per month Fin…

> Finally, for folks who would like to share their datasets, BigQuery offers free hosting and credits to help get a pipeline going.

Is this available in an automated fashion or is it something one has to contact Google for?

Re: Google launches Public Datasets program

#57

Post by kyrra »

awqrre wrote: post #40
vgt wrote: post #21

Earlier quoted context omitted.

Google BigQuery is a paid service, Dremel under the hood, used internally at Google by ~80% of Googlers every week. It's simply not going away because Google relies on it heavily. Externally, companies like Spotify and Kabam Games rely heavily on BigQuery as well: https://www.youtube.com/watch?v=LTVFg6YOjWo&list=PLIivdWyY5s... https://www.youtube.com/watch?v=6Nv18xmJirs&list=PLIivdWyY5s... So the story of Google Clou…

Google employees using something internally doesn't mean that it won't go away for everyone else... Also, they could decide to honor existing contracts and stop issuing new ones...

BigQuery is cover by the Google Cloud deprecation policy: https://cloud.google.com/terms/deprecation

Google is required to give at least a 1 year warning on shutting down released Cloud products.

Re: Google launches Public Datasets program

#58

Post by jrockway »

ortusdux wrote: post #2

"HACKER NEWS - A dataset that contains all stories and comments from Hacker News since its launch in 2006." I know what I'm doing this weekend.

It's mildly amusing:

> SELECT sum(length(text)) FROM [bigquery-public-data:hacker_news.comments] where author="jrockway"

4830955

That's almost 5 MB of comments I've written.

Re: Google launches Public Datasets program

#59

Post by toomuchtodo »

jrockway wrote: post #58
ortusdux wrote: post #2

"HACKER NEWS - A dataset that contains all stories and comments from Hacker News since its launch in 2006." I know what I'm doing this weekend.

It's mildly amusing: > SELECT sum(length(text)) FROM [bigquery-public-data:hacker_news.comments] where author="jrockway" 4830955 That's almost 5 MB of comments I've written.

Get outside more :)

Re: Google launches Public Datasets program

#60

Post by jrockway »

minimaxir wrote: post #47
panarky wrote: post #42

Earlier quoted context omitted.

I'm dying to get comment scores, not just submission scores. There's a lot you could do with that to find the best comments, which is really why HN is so awesome.

Blame HN for that. It was removed from public access right after I made a blog post about it, although it was coincidental I think.

Though the data could be assembled; I can see my comment scores, you can see your comment scores...
Post reply on HN