If you need to build a product that relies heavily on real time updates, I would look into using Elixir and Phoenix.[0] They nailed the channel abstraction which is the main entry point for realtime communication over websockets. It takes me hours to make scalable realtime applications in what would normally take me days using other systems. The language may take some time to get used to, and the ecosystem isn't as mature as other languages, but what is there is incredibly impressive.
Firebase outages and misleading status reporting
41–50 of 52 posts
Re: Firebase outages and misleading status reporting
#42AppEngine had the same problems - seemingly every week some component of the service would be down for some non-negligible amount of time (laughably it was often search -- we're talking about Google here). I've generally found AWS more reliable than GCP - even when GCP isn't having downtime, you'll occasionally get 503's from their APIs, so you need to wrap all your calls to them in retries. AWS has had multiple inst…
AppEngine had some big failures early on, but I (and some friends) built a $$$$$$$$$$$$ company on AppEngine (and GCP) and couldn't have done it without it. The stability the last few years has been extremely good. Our base logic was that we trust Google to hire and train talented DevOps more than we can do it and it sure sucks carrying a pager.
Snapchat?
If your app is that big, someone is always carrying a pager for when there are problems. The difference is on PaaS, you can't do a damn thing about it if it's a problem with the platform.
I've helped multiple companies get off of app engine because even for companies losing money (startups), it's too unreliable -- and actually very slow (datastore) if your app is relational. Also, it's very very expensive if you hit the datastore hard.
Re: Firebase outages and misleading status reporting
#43If you need to build a product that relies heavily on real time updates, I would look into using Elixir and Phoenix.[0] They nailed the channel abstraction which is the main entry point for realtime communication over websockets. It takes me hours to make scalable realtime applications in what would normally take me days using other systems. The language may take some time to get used to, and the ecosystem isn't as m…
Firebase does a lot more, including a slew of Auth options that make life much easier.
Add to that the ability to resolve connections dropping out (common on mobile) and that their libraries have been ported all over the place, and Firebase is a defacto answer for mobile developers. It can be up and running from in less than 30 minutes for someone who has 0 experience in cloud development.
It is hard to replicate that.
Re: Firebase outages and misleading status reporting
#44Earlier quoted context omitted.
It's different. If a business has the option to do something themselves and doing so would cost them less in the long run and give them more flexibility, then doing it themselves is a competitive advantage. If having solar panels becomes consistently cheaper than buying electricity from the grid (per megawatt), then individuals and businesses will all switch to solar panels... Especially if the business uses a lot of…
That is interesting, because the advice I always hear for businesses is to keep in house their core business and contract/outsource everything else. That 'bring everything in house' strategy only works for the biggest companies that have enough scale. For almost everyone else, the cost of providing a profit margin to the contracting company is dwarfed by the savings you get from the economy of scale the contractor is…
>> keep in house their core business and contract/outsource everything else
That makes sense, except we'd need some kind of working definition for "core business", since that's not necessarily self-evident.
Some businesses seem to think that the management of their brand is their only core business, and everything else is fair game for outsourcing.
Re: Firebase outages and misleading status reporting
#45If you need to build a product that relies heavily on real time updates, I would look into using Elixir and Phoenix.[0] They nailed the channel abstraction which is the main entry point for realtime communication over websockets. It takes me hours to make scalable realtime applications in what would normally take me days using other systems. The language may take some time to get used to, and the ecosystem isn't as m…
Firebase does a lot more, including a slew of Auth options that make life much easier. Add to that the ability to resolve connections dropping out (common on mobile) and that their libraries have been ported all over the place, and Firebase is a defacto answer for mobile developers. It can be up and running from in less than 30 minutes for someone who has 0 experience in cloud development. It is hard to replicate tha…
The common use cases for firebase can be easily reproduced with Phoenix. Phoenix also comes with a handy presence feature that allows you to track whether someone is currently using the product. (Think which present users in a chat room)
I understand the skepticism, but I would highly suggest taking a look and playing around. It's really, really good plus you get to fully own everything you build ;)
Re: Firebase outages and misleading status reporting
#46AppEngine had the same problems - seemingly every week some component of the service would be down for some non-negligible amount of time (laughably it was often search -- we're talking about Google here). I've generally found AWS more reliable than GCP - even when GCP isn't having downtime, you'll occasionally get 503's from their APIs, so you need to wrap all your calls to them in retries. AWS has had multiple inst…
AWS suffers from the same status page gaslighting though.
Re: Firebase outages and misleading status reporting
#47Earlier quoted context omitted.
AppEngine had some big failures early on, but I (and some friends) built a $$$$$$$$$$$$ company on AppEngine (and GCP) and couldn't have done it without it. The stability the last few years has been extremely good. Our base logic was that we trust Google to hire and train talented DevOps more than we can do it and it sure sucks carrying a pager.
Snapchat? If your app is that big, someone is always carrying a pager for when there are problems. The difference is on PaaS, you can't do a damn thing about it if it's a problem with the platform. I've helped multiple companies get off of app engine because even for companies losing money (startups), it's too unreliable -- and actually very slow (datastore) if your app is relational. Also, it's very very expensive i…
Not quite, but first MVP in 3 months and $80m gross revenue in the first year. Selling t-shirts. We did it with 3 engineers, no devops or qa teams and definitely no pagers. We had zero downtime and the very rare bugs were fixed on the next push to master (CI/CD) and real testing.
I'm not saying the datastore is perfect, but using the datastore has well known and predictable limitations that need to be engineered for. It is definitely not something you can RTFM later on. Just like any database to be honest. It is not a relational database. It doesn't do aggregations. It is for storing data (using Objectify [0]) and memcache is for caching that data.
Re: Firebase outages and misleading status reporting
#48Firebase is really awesome. However there kinds of reliability issues and the lack of integrity and communication with which Google handles such things are major reasons I would avoid committing to it. On top of that, Google's history of overlapping products (Firebase or Firestore?) and discontinuing or foot dragging support make decisions confusing and commitment harrowing. Amazon on the other hand has a history of…
The Amplify and AppSync models are also architecturally more scalable as you don't have one big opaque DB and endpoint in a single region.
Re: Firebase outages and misleading status reporting
#49I think that now, Firebase is build on Google Cloud Datastore. I have used Datastore in production since 2015, and have had no outages, but if I had to do it again I think I'd go normal RDB, just because query support is extremely limited (no full text search) and "schema change == data rebuild" issues.
Do you mean that you’d use something like a managed Postgres AND build and run a backend service that interfaces a web client to that database?
yeah exactly.
Re: Firebase outages and misleading status reporting
#50Earlier quoted context omitted.
It's different. If a business has the option to do something themselves and doing so would cost them less in the long run and give them more flexibility, then doing it themselves is a competitive advantage. If having solar panels becomes consistently cheaper than buying electricity from the grid (per megawatt), then individuals and businesses will all switch to solar panels... Especially if the business uses a lot of…
That is interesting, because the advice I always hear for businesses is to keep in house their core business and contract/outsource everything else. That 'bring everything in house' strategy only works for the biggest companies that have enough scale. For almost everyone else, the cost of providing a profit margin to the contracting company is dwarfed by the savings you get from the economy of scale the contractor is…
I think that any business which hires developers in-house should consider their software systems as being their core business. That's were the competitive edge comes from.
Most companies who use BaaS or PaaS these days already hire developers and sometimes even DevOps engineers; for them it doesn't make sense to outsource huge parts of their software. Some open source systems just work really well.
For example, I tried several times to launch a business around my current popular open source project but it hasn't worked so far; the problem I have is that companies only need consulting for a short while at the beginning to adapt the system to their use case and then it just works perfectly so they dont need me anymore.
When I reach out to a previous customer who didn't contact me in months, it's very common to hear that the system has been running perfectly without any issues at all. Even a couple of companies which have millions of users. Most of them never needed any consulting at all. So yes, it's much cheaper for a company in the long run to self-host in many cases.