Anyone know if they'll be offering Spanner as a publicly available service?
Probable in the future.. They never offer the latest technology...
Cloud Platform at Google I/O – new Big Data, Mobile and Monitoring products
11–20 of 33 posts
Re: Cloud Platform at Google I/O – new Big Data, Mobile and Monitoring products
#12This is one space where Google really excels. We're in the AWS ecosystem, and the database offerings are really subpar. DynamoDB, which I originally expected to be somewhat comparable to MongoDB, is an incredibly frustrating (and expensive) product to use. AWS Data Pipeline is extremely confusing and very expensive as well. AWS's offerings really lag behind Google's offerings (like BigQuery) in this space. Hopefully…
We've had good luck with DynamoDB, but it could be that it just fits our use case very well. What sort of frustration were you running into? (Honestly interested to avoid trouble down the line)
Most recently: hot hash key. DynamoDB uses the object to be persisted's hash key to route it to the right data cluster.
We're a SaaS company with lots of tiny customers and a few very large customers. We need to keep an index to show a specific customer only their data. That means the index for our largest customers gets hit a lot. The problem with this structure is we have to pay as if all of our customers were as popular as our biggest customers, or we get throttled. And even though the DynamoDB interface shows that you have provisioned 10x above your current usage, you still get throttled, because you're being throttled only in a single cluster.
So, let's say you solve that problem, but now you need to drop the troublesome index on a billion+ row table. With DynamoDB you can't change a table's indexes, so you have to migrate your table to a new table. Doing that without downtime is an incredible challenge.
Which reminds me of when they announced indexes. We were so excited only to find out we couldn't add indexes to our tables, but instead had to recreate them all.
The whole point of SaaS is to make our lives easier, but with DynamoDB our lives were much more difficult than just using Mongo.
Anyway, I need to do a blog post on this -- it's a bit too complicated for a HN comment. :-)
Re: Cloud Platform at Google I/O – new Big Data, Mobile and Monitoring products
#13Earlier quoted context omitted.
We've had good luck with DynamoDB, but it could be that it just fits our use case very well. What sort of frustration were you running into? (Honestly interested to avoid trouble down the line)
Most recently: hot hash key. DynamoDB uses the object to be persisted's hash key to route it to the right data cluster. We're a SaaS company with lots of tiny customers and a few very large customers. We need to keep an index to show a specific customer only their data. That means the index for our largest customers gets hit a lot. The problem with this structure is we have to pay as if all of our customers were as p…
Please do. It sounds like a very interesting adventure.
Re: Cloud Platform at Google I/O – new Big Data, Mobile and Monitoring products
#14Google adding cloud monitoring has me sorely tempted to abandon a side project of mine. I'm sure Google could do it better. Bah. :P
What type of monitoring? We use Stackdriver, which Google just recently bought.
Re: Cloud Platform at Google I/O – new Big Data, Mobile and Monitoring products
#15Google adding cloud monitoring has me sorely tempted to abandon a side project of mine. I'm sure Google could do it better. Bah. :P
What type of monitoring? We use Stackdriver, which Google just recently bought.
I probably will end up building the bare minimum to meet my needs and moving on tbh.
It was basically a monitoring/metrics system to merge how I handle the monitoring of crons, work queue, system metrics, analytics, etc. into a single service. Right now, I'm stuck using 3.
Sure, I could just build something to merge it together ... but at that point, I'm halfway to building my own.
Re: Cloud Platform at Google I/O – new Big Data, Mobile and Monitoring products
#16Anybody have experience moving from AWS to Google Cloud? If so, did you have any surprises in terms of difficulty or cost?
Re: Cloud Platform at Google I/O – new Big Data, Mobile and Monitoring products
#17Earlier quoted context omitted.
What do you think is the analogous product to Cloud Dataflow in the AWS ecosystem? SWF? http://aws.amazon.com/swf/
I presume AWS Data Pipeline. But they have their differences, so perhaps there is no true analog. http://aws.amazon.com/datapipeline/ Simple Workflow is new to me, so thanks for putting it on my radar!
I've been using Simple Workflow (in particular, the Flow framework: http://aws.amazon.com/swf/details/flow/) a lot recently to manage complex asynchronous distributed workflows, and it's a revelation. Some people laugh about the "Simple" part in the name, but it's kind of true - once you get over the initial learning curve. It's a bit like Git that way (First a lot of banging your head, then a lot of bang for buck)
Re: Cloud Platform at Google I/O – new Big Data, Mobile and Monitoring products
#18This is one space where Google really excels. We're in the AWS ecosystem, and the database offerings are really subpar. DynamoDB, which I originally expected to be somewhat comparable to MongoDB, is an incredibly frustrating (and expensive) product to use. AWS Data Pipeline is extremely confusing and very expensive as well. AWS's offerings really lag behind Google's offerings (like BigQuery) in this space. Hopefully…
Have you looked at AWS Redshift? It is somewhat analogous to BigQuery.
Re: Cloud Platform at Google I/O – new Big Data, Mobile and Monitoring products
#19This is one space where Google really excels. We're in the AWS ecosystem, and the database offerings are really subpar. DynamoDB, which I originally expected to be somewhat comparable to MongoDB, is an incredibly frustrating (and expensive) product to use. AWS Data Pipeline is extremely confusing and very expensive as well. AWS's offerings really lag behind Google's offerings (like BigQuery) in this space. Hopefully…
If you are in AWS there is also the four RDS (Oracle, MySQL, PostresSQL, SQL Server) options as well as RedShift. Also the best thing about AWS is that there are so many third party choices e.g. MongoLab, MongoHQ, Instaclustr, Cloudant.
Databases is not the area I would be choosing Google for.