Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We saved $50k/year with a Go microservice coded in a hackathon

movio.co

Post reply on HN
Print view View original

171–180 of 264 posts

Re: We saved $50k/year with a Go microservice coded in a hackathon

#171

Post by pisky »

zimpenfish wrote: post #165
jjeaff wrote: post #149

Earlier quoted context omitted.

Well, you should school those fools at Google, Facebook, Twitter, Pinterest, Amazon... and tell them how they are wasting time with their toy MySQL databases.

> those fools at Google, Facebook, Twitter, Pinterest, Amazon ... have dedicated hundreds of engineers and millions of dollars to nothing more than keeping MySQL up, running, and not crapping the bed every time someone looks at it funny. If you can afford that resource expenditure, by all means go nuts with MySQL. Most companies can't and would be far better served by something which doesn't need that amount of handh…

There is one thing that bugs me about all the talk of postgres' superiority: why haven't these companies switched to postgresql? Surely they weren't all too far invested in mysql before a "more knowledgeable" DBA came along saying postgresql is better.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#172

Post by manigandham »

This is a great example of how to do things wrong. I'm surprised they couldn't find any other columnstore database to take the place of InfiniDB in 2018.

The numbers they quote (5M members, 100M transactions) are tiny for any modern data warehouse. Many solutions would run these in sub-second speeds without changing the SQL at all, and it would be far better than building a quasi-SQL engine in Go.

Actually for the occasional querying + caching that they have, something like BigQuery or Snowflake data would be even cheaper with basically 0 operational effort.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#173

Post by rypskar »

orf wrote: post #8

I don't quite get this. How fast was running this query: Select loyaltyMemberID from table WHERE gender = x AND (age = y OR censor = z) Why the random complexity with individual unions and a group? Of course that's going to be dog slow. Sure, the filters can be arbitrary but with an ORM it's really really simple to build them up from your app code. The Django ORM with Q objects is particularly great at this. Obviousl…

I blame ORMs', if you don't understand SQL and how databases work you should not be allowed to use an ORM. If you know how databases work you, in many cases, will not use an ORM except for the most simple CRUD operations.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#174

Post by orf »

rypskar wrote: post #173
orf wrote: post #8

I don't quite get this. How fast was running this query: Select loyaltyMemberID from table WHERE gender = x AND (age = y OR censor = z) Why the random complexity with individual unions and a group? Of course that's going to be dog slow. Sure, the filters can be arbitrary but with an ORM it's really really simple to build them up from your app code. The Django ORM with Q objects is particularly great at this. Obviousl…

I blame ORMs', if you don't understand SQL and how databases work you should not be allowed to use an ORM. If you know how databases work you, in many cases, will not use an ORM except for the most simple CRUD operations.

I know how databases work and I use a (good) ORM almost exclusively for app work (SQL for exploration usually though). The benefits are huge.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#175

Post by zimpenfish »

thorin wrote: post #145
any626 wrote: post #29

Earlier quoted context omitted.

The user data is most likely in rows instead of columns. Instead of having id, name, age, gender 1213, fake, 60, female they would have property_id, user_id, value 1 (assume age), 1213, 60 2 (gender), 1213, female This gives them the freedom to add more properties to the user without always having to add a column to the users table. When querying the database you'll have to do unions or joins.

Entity attribute value anti pattern - this has been well known for at least 20 years. It can be tempting when you want to design a "flexible" system but really needs to be used sparingly. I was BI team lead on a product where the architect insisted that it be used on every entity (>300) as you never knew when you might want to add some bit of data. It led to some interesting (multi-page) sqls and the project ultimate…

We have this as a "meta" field (because MySQL is balls at adding new columns to big tables without multi-hour downtime) with some arcane nonsense format. Totally unqueryable with any efficiency.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#176

Post by stplsd »

attaboyjon wrote: post #78

I've come to the conclusion that the problem in tech is that all the people doing the work are in their early twenties and have no idea what they are doing. Once they get some experience they are quickly promoted to the CTO position. Rinse and repeat. What we have here is a classic dbms problem and no one at Movio seems to know how to deal with that. Instead of migrating from Mysql to something serious (Postgres) the…

>Instead of migrating from Mysql to something serious (Postgres)

This is just trolling

Re: We saved $50k/year with a Go microservice coded in a hackathon

#177

Post by zimpenfish »

pisky wrote: post #171
zimpenfish wrote: post #165

Earlier quoted context omitted.

> those fools at Google, Facebook, Twitter, Pinterest, Amazon ... have dedicated hundreds of engineers and millions of dollars to nothing more than keeping MySQL up, running, and not crapping the bed every time someone looks at it funny. If you can afford that resource expenditure, by all means go nuts with MySQL. Most companies can't and would be far better served by something which doesn't need that amount of handh…

There is one thing that bugs me about all the talk of postgres' superiority: why haven't these companies switched to postgresql? Surely they weren't all too far invested in mysql before a "more knowledgeable" DBA came along saying postgresql is better.

> Surely they weren't all too far invested in mysql

I think by the time Postgres sorted itself out into a more user/admin-friendly system (which is still fairly recently, really), MySQL had pretty much conquered the "quick and easy" mindshare and was deeply embedded almost everywhere.

And if you've spent millions of dollars architecting your systems such that MySQL's flaws aren't killer issues, there's very little financial benefit to switching, I guess.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#179

Post by dugmartin »

Regardsyjc wrote: post #47
owenmarshall wrote: post #13

Earlier quoted context omitted.

> Stuff like this is bread and butter SQL. Ten or fifteen years ago, sure - a DBA would look at a query plan and figure out how to do it properly. Worse case you'd slap a materialized view in and query that. But this is 2018! Programmers don't want to treat the database as anything but one big key value store ;)

Do you have any recommendations for resources to learn best database practices? I'm currently designing my first database and I'm not sure what information is worth storing (like calculations) and how to choose which data to group in tables.

I’d suggest learning the fundamental normal forms - once you understand them database design is very straightforward.

https://en.wikipedia.org/wiki/Database_normalization

Re: We saved $50k/year with a Go microservice coded in a hackathon

#180

Post by dijit »

jjeaff wrote: post #146
dijit wrote: post #122

Earlier quoted context omitted.

I disagree with your assertion that MySQL /is/ a serious database. The questions I usually ask myself when evaluating database solutions is: * Does it accept invalid data? * Does it change data on error? * Does the query planner change drastically between minor versions? * How strong is transaction isolation? can I create constraints, columns or tables in a transaction? * Does it scale vertically above 40~ CPU thread…

Considering many, many of the world's largest tech companies use MySQL or MySQL compatible databases, it's rather absurd to say that MySQL isn't a serious database. Regardless of whether it matches yours or someone else's personal list of capabilities.

To be perfectly fair with you, you can make bad choices and still get something useful done.

Most companies are not alive "because they chose mysql over something else" they're alive because they have "good enough" tech to get the job done. The job that they're trying to accomplish is the thing that makes them successful.

Uber isn't super huge because it used a specific database technology. It's huge because it's good at marketing, it's providing some value to people.

Post reply on HN