11–20 of 144 posts
See https://www.eofs.eu/_media/events/lad16/07_thiell_cheap_n_de... (slides) and https://www.youtube.com/watch?v=WbE0nl5V8WE
So a large part of choosing GCE is possibly because CEO was a Googler and he was very familiar with how Google works and his contact at Google helped him scored a better negotiation and possibly even better support. Although I won't deny GCE team would snatch Evernote regardless because it's an important client to show off in GCE's client portfolio. > The decision to go with Google over another provider like AWS or A…
Sadly the article never describes HOW they transfer the data. Simply sending 3PB on the wire will cost a fortune. Maybe Google does have some non-public service like AWS Snowball?
"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."
Moving into the cloud at this stage doesn't make any sense otherwise.
Earlier quoted context omitted.
Nothing against the author, but it sounds like another story that Google PR pitched and influenced. It is normal these days. I am sure AWS does the same. That is why the technical details may be light. The submitter is likely a Social Media Manager for Google (nothing wrong with that - kudos for not obscuring it!) I, too, have wondered about the technical details when I see stories like this. If the data was original…
The most straightforward is running aws s3 command but as you noted it's single threaded, slow as hell. Imagine you have 100TB of data to migrate from one bucket to another. If the command encounter an error (which is very likely to happen), you have to start again. I think the only other option left is use Spot Fleet EMR. I believe the S3 API takes a marker for pagination so you can probably keep paginate to list th…
I suspect Evernote, if using S3, would have a similar data profile of many small objects.
Now it may be that even if more expensive, it is a better use of your capital (renting instead of owning so that you can invest your capital in higher return investments like hiring better people). But is it cheaper, at an equivalent level of service / redundancy?
Evernote’s engagement with Google engineers was a pleasant surprise to McCormack. The team was available 24/7 to handle Evernote’s concerns remotely, and Google also sent a team of its engineers over to Evernote’s facilities to help with the migration. Those Google employees were around to help troubleshoot any technical challenges Evernote was having with the move. That sort of engineer-to-engineer engagement is som…
I fear there is no journalism here. No attempt to uncover real motivations, nothing remotely negative, no dissenting comment from current/past engineers, nothing about who paid what to whom, the incestuous relationships of ex-Googlers (Evernote CEO is ex-Google), how much Google is losing to take on high profile customers, who is placing these stories or anything other than pushing the narrative that "everybody must move to Google Cloud".
Sadly the article never describes HOW they transfer the data. Simply sending 3PB on the wire will cost a fortune. Maybe Google does have some non-public service like AWS Snowball?
TLDR: Shard at a time was replicated over the network, then live traffic was cut over to the Google Cloud replica.
[1] https://blog.evernote.com/tech/2017/02/08/part-1-evernote-se...
Earlier quoted context omitted.
"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."
Yeah, this is actually AWS Snowball. But the Google Cloud does no have a competing offering. It has none publicly, so it would be of real interest if they only have it for big customers handing over their first born son or for everyone on request but not advertising it like AWS.
The Stanford HPC team reported last year having sent more than 2PB to Google drive using Lustre/HSM. This was using the public API and the fact that gdrive is free for *.edu sites. See https://www.eofs.eu/_media/events/lad16/07_thiell_cheap_n_de... (slides) and https://www.youtube.com/watch?v=WbE0nl5V8WE