Live data from Hacker News

Stop Designing Your Web Application for Millions of Users When You Dont Have 100

darrenhorrocks.co.uk

71–80 of 103 posts

Re: Stop Designing Your Web Application for Millions of Users When You Dont Have 100

#72

[caveat: not read the text because if you click the partners link onthe nag box, it looks like I need to object to each “legitimate interest” separately and I've not got time for that – congratulations darrenhorrocks.co.uk on keeping your user count lower, so you don't have to design for higher!] The problem often comes from people solving the problem that they want to have, not the ones that they currently have. The…

>It looks like I need to object to each “legitimate interest” separately and I've not got time for that I absolutely don't understand why some websites do this. Either don't show them or don't make them annoying to disable. Let me explain: Legitimate interest is one of the lawful reasons for processing personal data. They don't have to ask for your permission. Usually adspam cookies are not in your legitimate interes…

> Legitimate interest is one of the lawful reasons for processing personal data.

Only if it is genuinely legitimate interest, in which case as you say they shouldn't even need to ask.

The reason they make them annoying to disable is the hope that people will stop bothering. In fact, they are usually hidden inside nested concertina UI elements so people don't even see the option to object. Dark pattern through and through.

The reason they ask even when only collecting/tracking strictly necessary data, is to try turn the public against the regulations by adding an unnecessary irritation. They want us to blame the legislators for causing us an inconvenience, when in fact the companies are doing so very deliberately because they are annoyed at the inconvenience they are caused by having to be transparent about the stalking that they do (or try to do).

> I assume this is just some trick abused by advertisers to work around GDPR

Sort of. And other similar regulations. Though it isn't actually GDPR-compliant by my reading, not by a long shot, the way most implement it.

Re: Stop Designing Your Web Application for Millions of Users When You Dont Have 100

#73

Earlier quoted context omitted.

Nowadays there are a lot of tools to setup this infra in a standard way very quickly though - in terms of CI/CD, tests, e2e tests etc.

There is no "standard way", only rough principles, and it all depends on the unique DNA of the company (cloud, stack, app deployment targets, etc). Yes there are a lot of tools, and experienced infrastructure engineers spend a lot of time integrating them. Often times they won't work without enormous effort, because of early "move fast & break things" design decisions made by the org. My experience has been that a st…

There are quick ways to setup a satisfying dev exp for small teams for full stack apps in terms of what will get you going for a while:

1. Git based CI/CD pipelines.

2. Multiple and arbitrarily configured amount of instances.

3. Easy rollbacks, no off time deployments.

4. Different set of environments. Branch/PR based preview links.

5. Out of the box E2E frameworks, etc.

I do agree however that you need someone in the first place to know that those things exist, but in a start up that would just be one experienced eng who is aware that they exist.

But they are standard in terms of that these are things that 99% of SaaS based Start Ups would definitely want.

Re: Stop Designing Your Web Application for Millions of Users When You Dont Have 100

#74

At a previous job, there was an argument over a code review where I had done some SQL queries that fixed a problem but were not optimal. The other side were very much "this won't work for 1000 devices! we will not approve it!" whereas my stance was "we have a maximum of 25 devices deployed by our only customer who is going to leave us next week unless we fix this problem today". One of the most disheartening weeks of…

In my last company that was always the golden rule - manage the present challenge with an eye on the future - but let's get things done.

Re: Stop Designing Your Web Application for Millions of Users When You Dont Have 100

#75

At a previous job, there was an argument over a code review where I had done some SQL queries that fixed a problem but were not optimal. The other side were very much "this won't work for 1000 devices! we will not approve it!" whereas my stance was "we have a maximum of 25 devices deployed by our only customer who is going to leave us next week unless we fix this problem today". One of the most disheartening weeks of…

I'm stunned to realize that most developers just blindly follow whatever are the newest hottest "good practices" and completely ignore the actual goals that code should try to achieve. Literally yesterday I had an argument with a coworker because he wouldn't buy the argument "High-availability costs us $2000 per year extra, downtime costs us $100 per day, so if the database breaks once per year, there's no point havi…

On the other side of the argument, does $2000 a year seem a lot for high-availability?

Does downtime really cost only $100 per day? How was it calculated? How much does your business make? It would seem it should make more than 365 * $100 = $36,500 to be able to be in a position to hire people in the first place.

Database downtime would potentially:

1) Break trust with customers.

2) Take away focus from your engs for a day or more which is also a cost. If an eng costs $200 per day for example, and you have 4 engs involved with, it's already a cost of $800, not to mention increase of odds of stress and burn out. And in US engs would cost of course much more than that, I was just picking a more favourable sum.

3) If it's a database that breaks, it's likely to cause further damages due to issues with data which might take indefinite time to fix.

Overall in most businesses it would seem a no brainer to pay $2000 a year to have high availability and avoid any odds of a database breaking. It's very little compared to what you pay your engineers.

Re: Stop Designing Your Web Application for Millions of Users When You Dont Have 100

#76

Earlier quoted context omitted.

That's where SQLite really shines - used it in a massively scaled facial recognition system, with several hundred simultaneous cameras, a very large FR database with over 10K people, each having dozens of face entries, and all the events that generates from live video viewing large public venues. SQLite was never a bottleneck, not at all.

Single sqlite db or did you shard it ?

Single SQLite db embedded within the FR server, with some crazy huge database size of 75+ million faces too. This system is now deployed in a huge percentage of the western world's airports.

Re: Stop Designing Your Web Application for Millions of Users When You Dont Have 100

#77
Counter argument is that, you do not know whether your system will go big next week. If it indeed does, now you do not have the Google/Meta/Amazon engineer legion to immediately scale it up without losing users[1]. Also, a gambit of scalable system initially helps to solve the later death-march, when management wants to ship the MVP in global production and then come howling when users are bouncing because the system was not designed for scale.

[1] Unlike early days, users are very quick to dismiss or leave immediately if the thing is breaking down and of course they will go rant at all social media outlets about the bad experience further making life hell for "build things that don't scale".

Re: Stop Designing Your Web Application for Millions of Users When You Dont Have 100

#78
post #77

Counter argument is that, you do not know whether your system will go big next week. If it indeed does, now you do not have the Google/Meta/Amazon engineer legion to immediately scale it up without losing users[1]. Also, a gambit of scalable system initially helps to solve the later death-march, when management wants to ship the MVP in global production and then come howling when users are bouncing because the system…

”I better buy a smaller clothes now in case I suddenly lose weight next week”

Re: Stop Designing Your Web Application for Millions of Users When You Dont Have 100

#79
The main thing is choosing the right technology that will allow scaling if it's part of the plan. It's not that you need to build Twitter-level traffic scale from Day 1, but you should of course choose a tech stack and platform that you can scale if you need.

(Twitter famously learned this the hard way)

If I loved Ruby, why would I choose Rails for a high-traffic social media app? If I loved Python, why choose Django for an API-first service that doesn't have an Admin dashboard? Yet I see it all the time - the developer only knows PHP/Laravel so everything gets built in Laravel. Point is... why lock yourself into a monolithic setup when other options are available even in those languages? You can definitely choose the "wrong stuff" and for bad reasons, even if it works today. "Doing things that don't scale" seems frankly stupid whenever scaling is a big part of the plan for the company, especially when there are plenty of options available to build things in a smart, prepared way.

But go ahead, install your fav monolith and deploy to Heroku for now, it will just be more work later when it has to be dismantled in order to scale parts independent from others (an API gateway that routes high traffic, a serverless PDF renderer, job scripts like notifications, horizontal scaling of instances, etc.).

It's smarter though to just choose a more future-proof language and framework setup. The Node setup on my rpi4 has been on GCP, AWS, Heroku, Render in various forms (as a monolith, as microservices, in between) - repo-wise, it's a "mono server" of 15+ APIs, apps, and websites that separate businesses in town rely on, yet I can work in it as one piece, even easily move it around providers if I want, horizontally scale as needed with no code changes, and because of essentially the folder structure of the app (I copied Vercel) and how Node require works, any export can be either imported to another file or deployed as a serverless function itself.

There's nothing about the codebase that forces me to choose "not scale" or "scale". I even have platform-agnostic code (libraries) that can run in either a browser or a server, talk about flexibility! No other language is as fast and scalable while also being this flexible.

Re: Stop Designing Your Web Application for Millions of Users When You Dont Have 100

#80

At a previous job, there was an argument over a code review where I had done some SQL queries that fixed a problem but were not optimal. The other side were very much "this won't work for 1000 devices! we will not approve it!" whereas my stance was "we have a maximum of 25 devices deployed by our only customer who is going to leave us next week unless we fix this problem today". One of the most disheartening weeks of…

Ugggggggggggh. My favorite part of the job is scaling/optimization, and my least favorite part of the job is... this.

There are a LOT of "engineers" who understand that one thing might be faster than another thing, but lack the chops/wisdom to understand when it actually matters.

Post reply on HN