Ask HN: Those with money-making side projects,how did you come up with the idea?

We started with a simple web scraping solution for real estate market that was up and running in just a couple of days. We used it to track prices of apartments in our area aggregated across multiple websites.

Then, as we saw value in this, we expanded data scraping to other cities and types of properties and released the product to external users. We had a few paying customers after a couple of months.

As we wanted to include more websites to collect data from, we run into significant problems of being blocked. In result, we started investigating how to overcome different mechanisms that websites use to prevent automated traffic from web scrapers.

It turns out that one of the the most important factors is to use good quality proxy which provides IP addresses shared with other real users and change them frequently. So, we started building our own proxy infrastructure powered by 4G proxies and implemented an API on top of it. And this is how we created Scraping Fish API for web scraping.

Now, we can offer a reliable solution for scraping even the most demanding websites like Instagram or Facebook.

Here is the full story of our product on IndieHackers: https://www.indiehackers.com/product/scraping-fish

A lot of people will answer "solve a real-world problem and alleviate your customers' pain points", but I've seen so many people interpret this in the most bizarre ways possible, go down ridiculously dumb paths and fall flat on their arse.

To be brutally honest, if you're asking how to find a product idea on HN, I don't think you'll be successful. Good business ideas jump out at you screaming, and you'll just stumble across them from time to time. Trying to artificially/rationally force this process pretty much guarantees failure.

There are people that glorify failure too, as some kind of "learning exercise", but I think for most it's genuinely painful, can waste years of your life, make you bitter and destroy relationships.

I was asking myself why the benefits of version control was not used more outside of the software industry, and decided to work on a MVP to prove/disprove it's usefulness in another hobby - photography

I like games but hate almost all mobile games, IAP and ads monetization, and how brainless the industry is. I looked for single player chess variant apps and found the offerings really spotty. The chess app market is oversaturated so I knew nobody was coming to save me. I started during 3 months of unemployment, then got a job with a brutal public transit commute. So I made ChessCraft for myself, mostly. So pretty standard advice: make something you know and want.

The reason is, bringing a product to market involves so much drudgery and success is so unlikely — you might as well have a good time.

Enjoyment will also keep you at it for longer, which can make a huge difference.

I won’t pretend my project [0] is sustaining me financially. But i’m still going after 3-4 years and I still find it “worth it”.

0. https://tatatap.com