Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I build software quickly

evanhahn.com

Post reply on HN
Print view View original

181–190 of 215 posts

Re: How I build software quickly

#181

Post by kukkeliskuu »

benterix wrote: post #47
kukkeliskuu wrote: post #22

In recent years, I have learned how to build sufficiently robust systems fast. Here are some things I have learned: * Learn one tool well. It is often better to use a tool that you know really well than something that on the surface seems to be more appropriate for the problem. For extremely large number of real-life problems, Django hits the sweet spot. Several times I have started a project thinking that maybe Djan…

> Forget Kubernetes, Redis While I agree with you, these two are the boring tech of 2025 for me. They work extremely reliably, they have well-defined use cases where they work perfectly and we know very well where they shouldn't be used, we know their gotchas, the interest around them seems to slowly wane. Personally, I'm a huge fan of these, just because they're very stable and they do what they are supposed to do.

I am not going to argue with you about specific technology or use cases. But these are complex pieces of software, and the fact that you consider them to be "boring technology" is the reason why I was talking about "extremely boring technology".

When you are building software really fast, you need to take the boringness to the next level and cut all non-essential components.

At fly.io, hosted Redis costs over $100/month, when one server + Postgres is less than $10/month. This tells you about the complexity.

Many people use Redis with Django, because that is "obvious choice" for Celery. But either Django built-in tasking (or backport with django-tasks) allows you to handle most of the use cases (long-running tasks, timed events) when combined with a cron such as Superchronic.

Many people consider ElasticSearch as an "obvious choice" to handle full-text search, but Postgres can handle it in most cases.

Mostly, if you have problems that require "webscale solutions", then you will have a team building a dedicated solution anyway.

Re: How I build software quickly

#182

Post by kukkeliskuu »

brabel wrote: post #172
dingi wrote: post #145

Earlier quoted context omitted.

I haven't tried Dropwizard. Is it a batteries included as Spring Boot? Eg: How is authentication support? Do we need a lot of boilerplate to implement let's say an OAuth2 resource server?

I used it a long time ago. As far as I know, it's an abandoned project now, very few people use it in the Java community. Go for one of the more popular ones: Spring, Micronaut, Helidon or Quarkus.

There are new frameworks all the time, but only very few establish a community that allows the framework to survive on the longer term. When you choose something that has been around for a long time, the probability for this is much higher, than for something that looks promising. Another benefit of the established frameworks is the amount of experience from various corner cases that has been coded in them. New frameworks often appear to handle simple use cases more conveniently, but that is only because they ignore all the complexity that the established solution has painfully learned throughout the years. You get that "for free" when you choose such a framework.

Re: How I build software quickly

#183

Post by kukkeliskuu »

jimbokun wrote: post #98
physicsguy wrote: post #34

Earlier quoted context omitted.

There's an almost pathological resistance to using anything that might be described as a 'framework' in the Go community in the name of 'simplicity'. I find such a blanket opinion to be unhelpful, what's fine for writing microservices is less good for bootstrapping a whole SaaS app and I think that people get in a bit too much of an ideological tizz about it all.

In my experience frameworks (in contrast to "libraries") add complexity making your application harder to understand and debug once you get past the initial prototype. Also, by their very nature they almost require you to write a program of a minimum size even when a simpler program would do to solve your problem.

This has been my experience as well with almost all frameworks.

It goes like this: the framework helps with the initial progress, because framework works with the more typical use cases. This allows you to "prototype" the solution and show progress to potential users etc.

But then you have a specific use case that does not fit the model of the framework, and you will end up going around the limitions of the framework. You often end up spending more time that you saved with the initial implementation.

My experience with core Django is that it has so many hooks and ways to inherit functionality, that it is extremely rare to end up with such problems. But it still means you need to learn when to use what feature, and how to structure your solution. I had to learn this through painful experience. I am still making mistakes, such as the "single-file status page app", a mistake which cost me several hours of productive coding. That might not sound like much, but when working full-time and running several side projects, few productive hours is extremely costly.

Most third party Django apps, even the popular ones, suffer from the framework problem, however. For example, I have have had to rewrite django-ads and django-distill, because their structure was limiting me from implementing features I needed. I just ditched django-formset as well.

I believe the reason for the limitation is that these third party apps have not been exposed to enough use cases and users to overcome the framework problem.

Re: How I build software quickly

#184

Post by vbezhenar »

aatd86 wrote: post #146
vbezhenar wrote: post #138

Earlier quoted context omitted.

> No one in their right mind would say: just use the standard library but I've seen it online. That discourse is not helping. I would say that. The most important thing about standard library is its stability. You won't ever need to touch code that works with standard library. It's finished code. Other than bug fixes, of course. Third-party libraries is a very different thing. They gets abandoned all the time, so now…

I don't think that everyone is capable of or should be implementing csrf protection or cors handling. While the standard library is an awesome starting point, telling people that it is sufficient is not going to convince them.

I wrote CORS handling in one project, it's like 10-20 lines of code. Very simple and well documented feature.

Re: How I build software quickly

#185

Post by kukkeliskuu »

creshal wrote: post #46
kukkeliskuu wrote: post #22

In recent years, I have learned how to build sufficiently robust systems fast. Here are some things I have learned: * Learn one tool well. It is often better to use a tool that you know really well than something that on the surface seems to be more appropriate for the problem. For extremely large number of real-life problems, Django hits the sweet spot. Several times I have started a project thinking that maybe Djan…

> Always choose extremely boring technology. Just use python/Django/Postgres for everything. Hell, think twice before you consider postgres. Sqlite scales further than most people would expect it to, especially for local development / spinning up isolated CI instances. And for small apps it tends to be good enough for production too.

I have worked on a database engine core team, and I really like the simplicity of SQLite and would love to use SQLite more. I am fully aware of benefits of the simplicity it brings to operations and I also know that the scalability of SQLite is more than sufficient for most of my use cases.

But I am running two major side projects I while still working full time. One of my side projects is a site with 2M page views per month. Additionally, depending on how you count (whether apps are included in the major projects or not), I have 3-4 smaller side projects, two of which use SQLite database.

To be able to run all of this, I have an extreme cut-throat approach to technology. SQLite is almost there, but ultimately Postgres is still more mature all-around solution. As an example, I believe (have not tested this though) it is quicker to set up full-text search for Postgres using online resources. The developer experience is better, because Postgres full-text search has been around for longer and has wider adoption.

Re: How I build software quickly

#186

Post by kukkeliskuu »

stavros wrote: post #148
t312227 wrote: post #117

Earlier quoted context omitted.

hello, as always: imho. (!) while i personally really love SQLite for a lot of use-cases, i wouldn't recommend / use it "in serious production" for a django-application which does more than a simple "hello world". why!? concurrency ... especially if your application attracts user or you just want to scale your deployment horizontally etc. ;)) so in my opinion: * why not use sqlite for development and functionality te…

Yeah, I've also found that foregoing Postgres is one step too far. It's just too useful, especially with Listen/Notify making it a good task queue broker. SQLite is great, but Postgres is definitely worth the extra dependency.

You distilled the point well. SQLite is great, but Postgres still hits the sweet spot.

Re: How I build software quickly

#187

Post by kukkeliskuu »

lmm wrote: post #83
kukkeliskuu wrote: post #22

In recent years, I have learned how to build sufficiently robust systems fast. Here are some things I have learned: * Learn one tool well. It is often better to use a tool that you know really well than something that on the surface seems to be more appropriate for the problem. For extremely large number of real-life problems, Django hits the sweet spot. Several times I have started a project thinking that maybe Djan…

> Most applications don't need to be single-page apps nor require heavy frontend frameworks. Even for those that can benefit from it, traditional Django views is just fine for 80% of the pages. For the rest, consider AlpineHJS/HTMX Doesn't that contradict "learn one tool well"? I write every webpage in React, not because I think everything needs to be an SPA, but because enough things end up needing client-side state…

Yes, I contradicted myself in that sense. I am not going to argument against your point.

But let me elaborate my approach which explains this apparent contradiction. Backend is needed in any case, and for me that is almost always Django. It is extremely fast to build traditional (non-SPA) CRUD apps with Django if you have only one model / one page. I can make a new one less than an hour.

If I need more interactivity or more data on the page, I use the same approach (models, views, forms with validation) but I create partial forms and return HTML fragments, instead of full pages. AlpineJS/HTMX allows me to implement 80% with Django/python, and I can avoid implementing a REST API and JSON serialization/deserialization, all the frontend data management, and there is usually almost no need for Javascript.

Re: How I build software quickly

#188

Post by kukkeliskuu »

fhd2 wrote: post #53
kukkeliskuu wrote: post #22

In recent years, I have learned how to build sufficiently robust systems fast. Here are some things I have learned: * Learn one tool well. It is often better to use a tool that you know really well than something that on the surface seems to be more appropriate for the problem. For extremely large number of real-life problems, Django hits the sweet spot. Several times I have started a project thinking that maybe Djan…

Agree with almost everything, but Celery is pretty common in my Django projects. I don't like the complexity cost, but especially when using some PaaS for hosting, it's usually the least painful option. I kinda always start out thinking this time I'll manage without, and then I have a bunch of jobs triggered via HTTP calls running into timeouts. At that point it's either threads, cron jobs (tricky with PaaS) or Celer…

I was using Redis and Celery but was not happy with the complexity.

But then I found out that background workers are being implemented to Django.

https://github.com/django/deps/blob/main/accepted/0014-backg...

This functionality has been backported as django-tasks library. You can use the database (i.e. Postgres) as the permanent storage.

For periodic jobs and running the django-tasks, I use Superchronic, which is a Docker-compatible cron. It is compatible with cron, with the added benefit that you can run jobs with sub-minute granularity. I have had no problems with running Superchronic inside my fly.io app.

I run Superchronic and Django with honcho and Procfiles.

Re: How I build software quickly

#189

Post by kukkeliskuu »

jimbokun wrote: post #94
kukkeliskuu wrote: post #22

In recent years, I have learned how to build sufficiently robust systems fast. Here are some things I have learned: * Learn one tool well. It is often better to use a tool that you know really well than something that on the surface seems to be more appropriate for the problem. For extremely large number of real-life problems, Django hits the sweet spot. Several times I have started a project thinking that maybe Djan…

This is great, but it's for a very narrow set of programming problems. You clearly work on web applications of moderate scale. For example, Kubernetes and Redis can suddenly become almost necessities for a back end service once it reaches a certain scale.

In my experience it works for a large variety of use cases.

As one of my side projects, I am running a website with 2M page views per month that has been implemented on Django, deployed to fly.io. Another side-project built-in tenant support, and several independent deployments as well.

Yes, those are moderate scale (for modern hardware), but in my experience very few sites really require "webscale". It is far more common to build prematurely for scalability than the opposite. If you have "webscale" problems, then you also have have a custom platform and team working on those.

The context of this discussion is how to build robust systems as fast as possible. My approach fits surprisingly large number of use cases.

Re: How I build software quickly

#190

Post by kukkeliskuu »

nakovet wrote: post #88
kukkeliskuu wrote: post #22

In recent years, I have learned how to build sufficiently robust systems fast. Here are some things I have learned: * Learn one tool well. It is often better to use a tool that you know really well than something that on the surface seems to be more appropriate for the problem. For extremely large number of real-life problems, Django hits the sweet spot. Several times I have started a project thinking that maybe Djan…

> Forget Celery How do you do background job processing without Celery? Every app/website I ever developed required some sort of background job processing, for Python I use Celery, Rails I use Sidekiq, Node.js I use Faktory. One of the biggest drawbacks (at least until a while ago) was that setting this up had to be a hacky webhook call to your own app that allowed up to N seconds request/response.

Django background workers, backported as django-tasks library, the tasks being stored in my database (Postgres). Everything is started by honcho, which starts Superchronic and Django, and Superchronic runs django-tasks. All this (except the database) is running inside single fly.io app.
Post reply on HN