Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Unlikely Database Migration

tailscale.com

Post reply on HN
Print view View original

11–20 of 190 posts

Re: An Unlikely Database Migration

#11

Post by eatonphil »

crawshaw wrote: post #6
jamescun wrote: post #3

This post touches on "innovation tokens". While I agree with the premise of "choose boring technology", it feels like a process smell, particularly of a startup whose goal is to innovate a techology. Feels demotivating as an engineer if management says our team can only innovate an arbitrary N times.

It is a tricky tradeoff for startups. On the one hand, a startup has very limited resources and so has to focus on the business. On the other hand, a startup has to experiment to find the business. I don't think there's an easy answer. In our case, the control plane data store really should be as boring as possible. It was real stretch using anything other than MySQL. We tried to lay out the arguments in the post, bu…

One solution is to use a standard subset of sql so you can use sqlite in unit tests and mysql/postgres in prod. Many languages also have in-memory SQL implementations that are also a more convenient substitute for sqlite.

Of course the benefit of what you did, even if I wouldn't have done it, is that you're _not_ using a different system in dev vs prod.

Re: An Unlikely Database Migration

#12

Post by bradfitz »

eatonphil wrote: post #11
crawshaw wrote: post #6

Earlier quoted context omitted.

It is a tricky tradeoff for startups. On the one hand, a startup has very limited resources and so has to focus on the business. On the other hand, a startup has to experiment to find the business. I don't think there's an easy answer. In our case, the control plane data store really should be as boring as possible. It was real stretch using anything other than MySQL. We tried to lay out the arguments in the post, bu…

One solution is to use a standard subset of sql so you can use sqlite in unit tests and mysql/postgres in prod. Many languages also have in-memory SQL implementations that are also a more convenient substitute for sqlite. Of course the benefit of what you did, even if I wouldn't have done it, is that you're _not_ using a different system in dev vs prod.

Exactly. That's why I wrote: "No Docker, no mocks, testing what we’d actually use in production."

Re: An Unlikely Database Migration

#13

Post by gk1 »

npiit wrote: post #2

Unfortunately this company is known for its shady agressive marketing on hacker news. The upvotes count is really suspicious and this is not the first time. EDIT: Why is the downvoting? the post was given like 9 upvotes in the first 5 minutes. I frequently go to "new" and this is a highly suspicious behaviur.

What is suspicious about the number 36? (The upvote count at this moment.)

Just because you dislike the product (and it's clear you do) does not prevent others from liking it, or at least finding their articles interesting.

Re: An Unlikely Database Migration

#14

Post by npiit »

gk1 wrote: post #13
npiit wrote: post #2

Unfortunately this company is known for its shady agressive marketing on hacker news. The upvotes count is really suspicious and this is not the first time. EDIT: Why is the downvoting? the post was given like 9 upvotes in the first 5 minutes. I frequently go to "new" and this is a highly suspicious behaviur.

What is suspicious about the number 36? (The upvote count at this moment.) Just because you dislike the product (and it's clear you do) does not prevent others from liking it, or at least finding their articles interesting.

The post was given like 9 upvotes in the first 5 minutes. I frequently go to "new" and this is a highly suspicious behavior.

Re: An Unlikely Database Migration

#15

Post by rubenv »

crawshaw wrote: post #6
jamescun wrote: post #3

This post touches on "innovation tokens". While I agree with the premise of "choose boring technology", it feels like a process smell, particularly of a startup whose goal is to innovate a techology. Feels demotivating as an engineer if management says our team can only innovate an arbitrary N times.

It is a tricky tradeoff for startups. On the one hand, a startup has very limited resources and so has to focus on the business. On the other hand, a startup has to experiment to find the business. I don't think there's an easy answer. In our case, the control plane data store really should be as boring as possible. It was real stretch using anything other than MySQL. We tried to lay out the arguments in the post, bu…

For PostgreSQL and Go, here's a package to spin up a temp in-mem PostgreSQL: https://github.com/rubenv/pgtest/

Re: An Unlikely Database Migration

#16

Post by Foxboron »

npiit wrote: post #2

Unfortunately this company is known for its shady agressive marketing on hacker news. The upvotes count is really suspicious and this is not the first time. EDIT: Why is the downvoting? the post was given like 9 upvotes in the first 5 minutes. I frequently go to "new" and this is a highly suspicious behaviur.

>EDIT: Why is the downvoting? the post was give like 9 upvotes in the first 5 minutes. I frequently go to "new" and this is a highly suspicious behaviour.

They are popular people so people submit the link. Once duplicate links are submitted there is an upvote on the first submission. No duplicates.

Source: I was the second upvote.

Re: An Unlikely Database Migration

#17

Post by ignoramous »

hfern wrote: post #4
npiit wrote: post #2

Unfortunately this company is known for its shady agressive marketing on hacker news. The upvotes count is really suspicious and this is not the first time. EDIT: Why is the downvoting? the post was given like 9 upvotes in the first 5 minutes. I frequently go to "new" and this is a highly suspicious behaviur.

I don't think it's shady marketing. The company is composed of several prominent software engineers (eg Brad Fitzpatrick of memcached fame). I think many folks on HN (myself included) are interested to see what they are working on. Especially as Brad left Google to work on it.

Brad Fitzpatrick aside (he's done a lot more than memcached), David Crawshaw, co-author of TFA, lead go-mobile (native golang apps for Android and iOS) and NetStack (userspace TCP/IP implementation used by gVisor and in-anger by Fuschia) while at Google.

Re: An Unlikely Database Migration

#18

Post by jrockway »

eatonphil wrote: post #11
crawshaw wrote: post #6

Earlier quoted context omitted.

It is a tricky tradeoff for startups. On the one hand, a startup has very limited resources and so has to focus on the business. On the other hand, a startup has to experiment to find the business. I don't think there's an easy answer. In our case, the control plane data store really should be as boring as possible. It was real stretch using anything other than MySQL. We tried to lay out the arguments in the post, bu…

One solution is to use a standard subset of sql so you can use sqlite in unit tests and mysql/postgres in prod. Many languages also have in-memory SQL implementations that are also a more convenient substitute for sqlite. Of course the benefit of what you did, even if I wouldn't have done it, is that you're _not_ using a different system in dev vs prod.

I would not go down the road of figuring out what subset of SQL various database understand. You will always be surprised, and you'll be surprised in production because "the thing" you developed against sqlite doesn't work in postgres.

I used to do this and stopped when I noticed that sqlite and postgres treat booleans differently; postgres accepts 't' as true, but SQLite stores a literal 't' in the boolean-typed field. This means you get different results when you read things back out. All in all, not a rabbit hole you want to go down.

Personally, I just create a new database for each test against a postgres server on localhost. The startup time is nearly zero, and the accuracy compared to production is nearly 100%.

Re: An Unlikely Database Migration

#19

Post by marcinzm »

crawshaw wrote: post #6
jamescun wrote: post #3

This post touches on "innovation tokens". While I agree with the premise of "choose boring technology", it feels like a process smell, particularly of a startup whose goal is to innovate a techology. Feels demotivating as an engineer if management says our team can only innovate an arbitrary N times.

It is a tricky tradeoff for startups. On the one hand, a startup has very limited resources and so has to focus on the business. On the other hand, a startup has to experiment to find the business. I don't think there's an easy answer. In our case, the control plane data store really should be as boring as possible. It was real stretch using anything other than MySQL. We tried to lay out the arguments in the post, bu…

I'm used to Scala for these things and it seemed fairly easy to do. Docker DB container would be spun up at the start of testing (using a package such as testcontainers-scala). Then the DB would be setup using the migration scripts (using Flyway) so it has a known state. Every test (or set of tests if they're linked) would have a pre-hook which nuked the DB to a clean state. Then the docker container would be shut down at the end of testing. I'm guessing there's even some way to have this run concurrently with multiple DBs (one per thread) but we never did that. Is Java's ecosystem for this type of tooling just that much better?

Re: An Unlikely Database Migration

#20

Post by npiit »

ignoramous wrote: post #17
hfern wrote: post #4

Earlier quoted context omitted.

I don't think it's shady marketing. The company is composed of several prominent software engineers (eg Brad Fitzpatrick of memcached fame). I think many folks on HN (myself included) are interested to see what they are working on. Especially as Brad left Google to work on it.

Brad Fitzpatrick aside (he's done a lot more than memcached), David Crawshaw, co-author of TFA, lead go-mobile (native golang apps for Android and iOS) and NetStack (userspace TCP/IP implementation used by gVisor and in-anger by Fuschia) while at Google.

Of course I know of Brad Fitzpatrick. I am just questioning the product and its "innovation" compared to the rest of the industry in order to be promoted here that much and I think I know enough about the industry to see that there is nothing to see here for all that hype assuming it's genuine. I can get that for FOSS projects, but not for companies.
Post reply on HN