Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We saved $50k/year with a Go microservice coded in a hackathon

movio.co

Post reply on HN
Print view View original

191–200 of 264 posts

Re: We saved $50k/year with a Go microservice coded in a hackathon

#191

Post by toddBarkus »

cookiecaper wrote: post #37
toddBarkus wrote: post #26

Earlier quoted context omitted.

I think this is a case for the return of the traditional "sysadmin" as "devops"/"SRE" is now the role of unblocking deploying a solution instead of questioning it's complexity/fitness.

While I agree, I also think that overt gating and approval processes create a high tension dynamic that frequently breaks down, whether it's ops v. dev, security v. dev, or others. It's easy for people to get their pride wounded, and they end up encouraged to find workarounds to the process. The simple answers to this are pretty much imaginary, unfortunately.

Sure, of course. I was hoping to point out that there's an increasingly overlooked value in having someone question complexity. The "no, you don't need React" of the frontend dev or the "our data is actually relational" of the back end dev.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#192

Post by foepys »

slx26 wrote: post #184
attaboyjon wrote: post #78

I've come to the conclusion that the problem in tech is that all the people doing the work are in their early twenties and have no idea what they are doing. Once they get some experience they are quickly promoted to the CTO position. Rinse and repeat. What we have here is a classic dbms problem and no one at Movio seems to know how to deal with that. Instead of migrating from Mysql to something serious (Postgres) the…

To be fair, we used to die at 40. Now you can get a CS degree at 21 or 22, and you still "have no idea what are you doing". Maybe it's just that technology is really complicated, the world is really complicated, and everything is changing fast. I don't mean it to be conformist, but it's easy to forget some things are actually hard when you are very clever or old enough to forget how it was like when you were still le…

But at the same time it's easier than ever to find information on nearly everything. Asking experts also is easier than ever before. I don't think it's feasible to dismiss GP's claim with yours.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#193

Post by rypskar »

orf wrote: post #174
rypskar wrote: post #173

Earlier quoted context omitted.

I blame ORMs', if you don't understand SQL and how databases work you should not be allowed to use an ORM. If you know how databases work you, in many cases, will not use an ORM except for the most simple CRUD operations.

I know how databases work and I use a (good) ORM almost exclusively for app work (SQL for exploration usually though). The benefits are huge.

OK, the part about not using ORM if you know SQL is a bit of an exaggeration. At least when you know SQL you know when to use an ORM and when to not use it. If all you know is ORM then you will always use it, and ORM seems to lead to many developers not learning SQL

Re: We saved $50k/year with a Go microservice coded in a hackathon

#194

Post by dijit »

BartBoch wrote: post #154
dijit wrote: post #122

Earlier quoted context omitted.

I disagree with your assertion that MySQL /is/ a serious database. The questions I usually ask myself when evaluating database solutions is: * Does it accept invalid data? * Does it change data on error? * Does the query planner change drastically between minor versions? * How strong is transaction isolation? can I create constraints, columns or tables in a transaction? * Does it scale vertically above 40~ CPU thread…

I think a lot of what have you written can be solved software-side. Good Database should be no excuse for bad code. I do not think MySQL is a technological debt as in 80% startups moving to the different solution is cheap and non-problematic. The LAMP is good enough and quickest/cheapest for the majority of tech companies.

I'm being perfectly fair in being critical of a software which claims to be doing those things.

You can solve issues in your application if you know there will be issues like these, knowing the pitfalls and drawbacks of a technology is certainly noble- but if you do then why not choose something that follows principle of least surprise. (There might be reasons).

I would never claim that you should move everything from MySQL if you use it. However if you care about data consistency ensure that you change the defaults, engage strict mode, ensure that your application has no bugs in handling data.

This is actually hard to do correctly, it's overhead in development that you shouldn't be caring about. Just choose something that has sane error conditions and the problem vanishes.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#195

Post by z3t4 »

amq wrote: post #143
z3t4 wrote: post #139

mySQL is very slow when it comes to joins, groups, etc. You are always better off with simple select's. If you are using for example PHP the only viable solution is to have the db crunch it. But when using Go, NodeJS et al. you can pull the data out as a stream/array-like and apply filter/map/reduce and the logic would probably be easier to manage, rather then generating a complex SQL query. And would also allow you…

In my experience, if you have a proper schema, even very complex, but thought-out, queries are instant on virtually any database size.

Stuff like the group builder in the article is hard to reason about as there are so many combinations. MySQL is bad at optimizing O(n2), no matter how you design the schema it will be slow. The solution, like they probably did in the article, is to break it down into separate more simple O(n) queries.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#196

Post by wlll »

pisky wrote: post #171
zimpenfish wrote: post #165

Earlier quoted context omitted.

> those fools at Google, Facebook, Twitter, Pinterest, Amazon ... have dedicated hundreds of engineers and millions of dollars to nothing more than keeping MySQL up, running, and not crapping the bed every time someone looks at it funny. If you can afford that resource expenditure, by all means go nuts with MySQL. Most companies can't and would be far better served by something which doesn't need that amount of handh…

There is one thing that bugs me about all the talk of postgres' superiority: why haven't these companies switched to postgresql? Surely they weren't all too far invested in mysql before a "more knowledgeable" DBA came along saying postgresql is better.

Because MySQL is the PHP of databases. Much like PHP is designed to just carry on truckin' whatever problem it encounters (https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/), MySQL was too. It's easy to slam some data into it and get it back without having to think too hard about what you're actually storing (https://grimoire.ca/mysql/choose-something-else). It's ease of use made it pervasive, much like PHP.

Following on from that, I suspect a lot of large companies use MySQL because they always have, not because it's actually any good. For example, Basecamp used MySQL while I was there, but I never met a single Sysadmin there who would use it over Postgres if they were to start a new project.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#197

Post by kevin_nisbet »

attaboyjon wrote: post #78

I've come to the conclusion that the problem in tech is that all the people doing the work are in their early twenties and have no idea what they are doing. Once they get some experience they are quickly promoted to the CTO position. Rinse and repeat. What we have here is a classic dbms problem and no one at Movio seems to know how to deal with that. Instead of migrating from Mysql to something serious (Postgres) the…

> I've come to the conclusion that the problem in tech is that all the people doing the work are in their early twenties and have no idea what they are doing.

Not that I disagree, but to be fair, I've seen plenty of tech ignorance with experienced and older engineers as well that has been pretty crippling.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#198

Post by bpicolo »

thorin wrote: post #145
any626 wrote: post #29

Earlier quoted context omitted.

The user data is most likely in rows instead of columns. Instead of having id, name, age, gender 1213, fake, 60, female they would have property_id, user_id, value 1 (assume age), 1213, 60 2 (gender), 1213, female This gives them the freedom to add more properties to the user without always having to add a column to the users table. When querying the database you'll have to do unions or joins.

Entity attribute value anti pattern - this has been well known for at least 20 years. It can be tempting when you want to design a "flexible" system but really needs to be used sparingly. I was BI team lead on a product where the architect insisted that it be used on every entity (>300) as you never knew when you might want to add some bit of data. It led to some interesting (multi-page) sqls and the project ultimate…

You could argue that there are cases for it. Datomic[0] is basically EAV on steroids.

https://www.datomic.com/

Re: We saved $50k/year with a Go microservice coded in a hackathon

#199

Post by bpicolo »

nstart wrote: post #69
any626 wrote: post #29

Earlier quoted context omitted.

The user data is most likely in rows instead of columns. Instead of having id, name, age, gender 1213, fake, 60, female they would have property_id, user_id, value 1 (assume age), 1213, 60 2 (gender), 1213, female This gives them the freedom to add more properties to the user without always having to add a column to the users table. When querying the database you'll have to do unions or joins.

Oh gosh this pattern. The first time I encountered it was in my first job where we used Magento. Super flexible. Also super slow. Does anyone have any advice how to make a db design like this work faster? Generally I thought when data is arranged like this it might be a prime candidate for document based storage. But I'm no dba so I have no idea if that would be correct.

If what you actually want is data arranged like such, Datomic is probably a prime candidate

https://www.datomic.com/

Re: We saved $50k/year with a Go microservice coded in a hackathon

#200

Post by bpicolo »

dbrgn wrote: post #117
DCoder wrote: post #97

Earlier quoted context omitted.

This is a pretty popular pattern known as Entity-Attribute-Value [0]. It's used by many products where a) data model needs to be very flexible and allow new attributes without schema changes, or b) a typical entity has a large number of possible attributes that may or may not be set for all entities ("sparse" attributes). WordPress uses this to store post metadata, Magento uses this to store product attributes and mo…

Sounds like a great case for Postgres hstore (like OpenStreetMap does it)?

hstore querying is quite slow (and GIN indexes on hstores are pretty massive). I'd always go jsonb over hstores these days, but jsonb has the same indexing problem. JSON has a well-optimized/spec compliant serializer/deserializer in every language you can imagine as a baseline, whereas hstore does not.
Post reply on HN