Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We saved $50k/year with a Go microservice coded in a hackathon

movio.co

Post reply on HN
Print view View original

141–150 of 264 posts

Re: We saved $50k/year with a Go microservice coded in a hackathon

#141

Post by colde »

indigochill wrote: post #125
attaboyjon wrote: post #78

I've come to the conclusion that the problem in tech is that all the people doing the work are in their early twenties and have no idea what they are doing. Once they get some experience they are quickly promoted to the CTO position. Rinse and repeat. What we have here is a classic dbms problem and no one at Movio seems to know how to deal with that. Instead of migrating from Mysql to something serious (Postgres) the…

Well, a problem in tech certainly. An alternative possibility (and another problem in tech) could be some mid-level developer could have figured out the problem at the start but because of artificial time pressure to deliver they didn't have the time to, so went with the first bad idea that popped into their head without taking the necessary time to evaluate it. The fact this had to happen in a hackathon suggests a t…

It is worth noting that on their website, their management team doesn't include a CTO, even though their main product is basically a software solution. They have a few sales people represented though, so management might not be great techwise.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#142

Post by amq »

voltagex_ wrote: post #45
hippich wrote: post #14

What else it shows - how expensive AWS hardware vs hosting own hardware. I guess you have to consider how often you have to scale, but hetzner offers dedicated servers with 64Gb and NVMe drives starting from 54 euros per month - https://www.hetzner.com/dedicated-rootserver?country=us - compare that to $580 per month these guys were paying for i3.2xlarge instance..

What if you need boxes closer to where your customers are? Latency is a big deal, especially in Australia / New Zealand.

Another one is Scaleway with baremetal servers starting from €3/month and offering snapshots, volumes, security groups and floating IPs.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#143

Post by amq »

z3t4 wrote: post #139

mySQL is very slow when it comes to joins, groups, etc. You are always better off with simple select's. If you are using for example PHP the only viable solution is to have the db crunch it. But when using Go, NodeJS et al. you can pull the data out as a stream/array-like and apply filter/map/reduce and the logic would probably be easier to manage, rather then generating a complex SQL query. And would also allow you…

In my experience, if you have a proper schema, even very complex, but thought-out, queries are instant on virtually any database size.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#144

Post by shoo »

ksec wrote: post #116
attaboyjon wrote: post #78

I've come to the conclusion that the problem in tech is that all the people doing the work are in their early twenties and have no idea what they are doing. Once they get some experience they are quickly promoted to the CTO position. Rinse and repeat. What we have here is a classic dbms problem and no one at Movio seems to know how to deal with that. Instead of migrating from Mysql to something serious (Postgres) the…

If I ever be a CEO of the company / Startup, that one criteria I made is either I decide on all the technologies we use, or there is no CTO so i make those decision. And that criteria of technologies could be summed into one sentence. Use something boring . No Hyped programming languages / DB / tools allowed. Of course some would argue you would be doing it wrong even if it was using old tech / programming / tools. W…

Quoting Dan McKinley's "choose boring technology" [cbt]:

> Embrace Boredom.

> Let's say every company gets about three innovation tokens. You can spend these however you want, but the supply is fixed for a long while. You might get a few more after you achieve a certain level of stability and maturity, but the general tendency is to overestimate the contents of your wallet. Clearly this model is approximate, but I think it helps.

> If you choose to write your website in NodeJS, you just spent one of your innovation tokens. If you choose to use MongoDB, you just spent one of your innovation tokens. If you choose to use service discovery tech that's existed for a year or less, you just spent one of your innovation tokens. If you choose to write your own database, oh god, you're in trouble.

> Any of those choices might be sensible if you're a javascript consultancy, or a database company. But you're probably not. You're probably working for a company that is at least ostensibly rethinking global commerce or reinventing payments on the web or pursuing some other suitably epic mission. In that context, devoting any of your limited attention to innovating ssh is an excellent way to fail. Or at best, delay success.

[CBT] http://mcfunley.com/choose-boring-technology

Re: We saved $50k/year with a Go microservice coded in a hackathon

#145

Post by thorin »

any626 wrote: post #29
orf wrote: post #8

I don't quite get this. How fast was running this query: Select loyaltyMemberID from table WHERE gender = x AND (age = y OR censor = z) Why the random complexity with individual unions and a group? Of course that's going to be dog slow. Sure, the filters can be arbitrary but with an ORM it's really really simple to build them up from your app code. The Django ORM with Q objects is particularly great at this. Obviousl…

The user data is most likely in rows instead of columns. Instead of having id, name, age, gender 1213, fake, 60, female they would have property_id, user_id, value 1 (assume age), 1213, 60 2 (gender), 1213, female This gives them the freedom to add more properties to the user without always having to add a column to the users table. When querying the database you'll have to do unions or joins.

Entity attribute value anti pattern - this has been well known for at least 20 years. It can be tempting when you want to design a "flexible" system but really needs to be used sparingly. I was BI team lead on a product where the architect insisted that it be used on every entity (>300) as you never knew when you might want to add some bit of data. It led to some interesting (multi-page) sqls and the project ultimately failed. This was one of the reasons. Slow performance and often runtime errors when expected data wasn't present and the application layer couldn't cope. It was a good learning experience. https://mikesmithers.wordpress.com/2013/12/22/the-anti-patte...

Re: We saved $50k/year with a Go microservice coded in a hackathon

#146

Post by jjeaff »

dijit wrote: post #122
peburrows wrote: post #89

Earlier quoted context omitted.

Overall, I agree with the sentiment of your comment, but the assertion that MySQL isn’t a serious database is just flat out wrong.

I disagree with your assertion that MySQL /is/ a serious database. The questions I usually ask myself when evaluating database solutions is: * Does it accept invalid data? * Does it change data on error? * Does the query planner change drastically between minor versions? * How strong is transaction isolation? can I create constraints, columns or tables in a transaction? * Does it scale vertically above 40~ CPU thread…

Considering many, many of the world's largest tech companies use MySQL or MySQL compatible databases, it's rather absurd to say that MySQL isn't a serious database. Regardless of whether it matches yours or someone else's personal list of capabilities.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#147

Post by twic »

Okay, before i've even read the article, i'm going to guess that they were doing something egregously expensive in the cloud, and the microservice helped them do it more efficiently - but still much more expensively than doing it in a simple, old-fashioned way.

Now i'll read the article ...

EDIT: I would say i'm no more than 30% right. They were doing heavyweight data crunching in the cloud, and so paying more for it than if they were doing it on rented hardware. But that's a constant-factor thing; it's not like they were downloading gigabytes of CSVs from S3 on every request or some such. Their query looks suspect to me: couldn't it be written to do one big scan, rather than unioning a load of things? Or is this the right way to write queries on column stores? Still, there is no glaring obvious (to me) old-school fix for this.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#149

Post by jjeaff »

davidp wrote: post #91
attaboyjon wrote: post #78

I've come to the conclusion that the problem in tech is that all the people doing the work are in their early twenties and have no idea what they are doing. Once they get some experience they are quickly promoted to the CTO position. Rinse and repeat. What we have here is a classic dbms problem and no one at Movio seems to know how to deal with that. Instead of migrating from Mysql to something serious (Postgres) the…

I like to trot out this old gem[1] when people wonder why there's so much hate for MySQL. Nontransactional DDL alone is sufficient to classify it as a toy DB for me. Yes, I've been personally bitten by it. [1]: https://grimoire.ca/mysql/choose-something-else

Well, you should school those fools at Google, Facebook, Twitter, Pinterest, Amazon... and tell them how they are wasting time with their toy MySQL databases.

Re: We saved $50k/year with a Go microservice coded in a hackathon

#150

Post by jjeaff »

Reedx wrote: post #90
matte_black wrote: post #81

Earlier quoted context omitted.

Agreed, I’m sure if this is the kind of problems they are having I could probably be saving them even more than $50k a year if they were on Postgres.

The difference between mySQL and Postgres is that significant? What makes up the difference?

Hardly. And in general, MySQL outperforms Postgres. But you will have more options to tweak Postgres to work better for you in edge cases.
Post reply on HN