Live data from Hacker News

Write libraries instead of services, where possible

catern.com

281–290 of 328 posts

Re: Write libraries instead of services, where possible

#281
post #131

Earlier quoted context omitted.

You're right. The fact that people can make outsize rewards from subscription services are the reason that even small single use tools are now 'services'. I have a piece of software that removes the background from my video feed. It's a subscription, and it regularly phones home. I have a piece of software that lets me visually combine, rotate and reorder pdfs. It's a subscription. Even the simplest things, like a 't…

> It's miserable fighting off a million people who want to help me for 'less than the daily price of your coffee'. Is it also miserable "fighting off" a million companies who want to sell you stuff for the daily price of a coffee? Stuff like T-Shirts, books, newspapers, candy, etc? You seem to have a patronizing attitude towards software products/services. You seem to think that they are so simple, that they should b…

> You seem to think that they are so simple, that they should be free.

I don't think anyone is claiming that; the opposite of subscription SaaS is not freeware. At some point companies realized that they could have more predictable, long-term, higher revenue streams if instead of selling you a bit of client-side software a single time (with uncertain future business from upgrades), they sold you a subscription to their software, which is often (unnecessarily?) cloud-based. This all feels weird to those of us who have been around for a while and got used to buying software the "old way".

Building a cloud/web-based app means it's (mostly) effortlessly multi-platform. You don't have to build separate apps for Windows, Mac, and (occasionally) Linux. (But there's also the e.g. Electron option.) And meanwhile, you can push bugfixes and new features out to your customers near-instantly. Your release cycles are small, and are often measured in days or weeks, not quarters or years. You can justify charging on a subscription model because you are constantly working for your customers. On the flip side, if a customer wants to cancel their subscription, what happens to any data generated with your app? If it's in a proprietary format, IMO it's unethical to hold a user's data hostage like that.

Selling software by the download is a hard business to be in, and the incentives are not always aligned well. You're expected to fix bugs and release patch versions for "free". But usually you can charge for new major version upgrades. So the incentive is to skimp on bugfixes and instead work on new, big features. Beyond that, there's an incentive to make big, sweeping changes to your app so you can justify calling it a new major release, which triggers an upgrade fee, even if those changes don't actually benefit users. (Then again, this phenomenon, for some reason, exists with subscription apps too.)

But many people just see it as a money grab, especially for products that used to not require a subscription. In 1998 I could go and buy a copy of MS Office in a box from my local store, and it was then mine. I could use it as long as I could find an OS that would run it. Likely I could still run it today under wine or something if I still had a copy. But now we have Office 365. I have to sign up for a subscription. If at any point I want to cancel, I can't use the software anymore. The data I've created with it is still mine, but I have to deal with imperfect format conversions done by other office apps. You could perhaps draw a similar parallel with Adobe's creative software.

Regarding money, I think there's also an aversion to having to pay indefinitely. Sure, MS Office was expensive to buy, at several hundred dollars or whatever it was. But when I forked over that cash, I knew I was done paying for that version. Even if the SaaS version is priced at a few dollars a month, and I'm unlikely to ever subscribe long enough to pay the old "full price", there's still an irrational feeling of getting a raw deal. I think most people are more comfortable with known, one-time costs than with recurring costs, which may change if the company later decides to charge more.

Re: Write libraries instead of services, where possible

#282

Phoenix is practically built with this idea in mind. My startup is built out of a single phoenix monolith. Only external dependency is postgres. Despite this, I've managed to completely avoid all the typical scaling issues of a monolith. Need a service or background worker? I juts make a Genserver or Oban worker and mount it in the supervision tree. The Beam takes care of maintaining the process, pubsub to said servi…

In my experience, this is also a major benefit of running Akka on Scala or Java. I had the realization that it's basically a single-language Kubernetes, with some really nice abstractions built on top.

> it's basically a single-language Kubernetes

Well, yes and no.

By using Kubernetes, you get a scalable infrastructure. By using OTP/Akka, you get a scalable application.

While there is common problems to both domains, they are still 2 different domains.

For example, using only Kubernetes, you won't have the ability to react to a Pod restart within your application (unless your application is aware of Kubernetes).

Using only OTP/Akka, you still need a workflow for deployment and infrastructure management, and you still need to implement node discover for clustering.

NB: For Elixir, you have libcluster[1] that can use different strategies for node discovery, including using a Kubernetes Service to discover the nodes (as Pods).

EDIT: Using Kubernetes, libcluster, and Horde[2], you get the best of both worlds IMHO.

[1] - https://hexdocs.pm/libcluster/2.5.0/readme.html

[2] - https://hexdocs.pm/horde/0.1.4/api-reference.html

Re: Write libraries instead of services, where possible

#283

Earlier quoted context omitted.

BEAM will automatically distribute processes across a cluster, right?

not automatically but its pretty easy to configure in your supervision tree file. I don't know the details because whatever happens by default has taken care of our needs so far. IT does automatically distribute processes across all the cores on a cpu though.

What I like about the Erlang platform is that it seems like it has the most sensible “microservice” story: deploy your language runtime to all the nodes of a cluster and then configure distribution in code. Lambdas, containers, etc. all push this stuff outside your code into deployment tooling that is, inevitably, less pleasant to manage than your codebase.

Re: Write libraries instead of services, where possible

#284

Earlier quoted context omitted.

It’s not hard to find a T-shirt I can wear over and over again without having to keep paying for it. Nowadays it’s nearly impossible to find software that lets you pay once at a reasonable price. So yeah, it is very frustrating to fight off subscriptions.

So write your own software for your needs and sell it for a one off price? But you presumably won't because the incentive isn't great enough. Which is exactly why people milk subs. And if you're not willing to do it for the incentive of a one off price, why should anyone else be?

Fells like the reason one off pricing is less common is because the model has felt broken since the time of Kazaa (file sharing app from way back), Napster etc. For some reason software is almost exclusively sold through app stores. I don't remember the last time I bought software that wasn't subscription based for a non mobile device. Ok, maybe one app.

Re: Write libraries instead of services, where possible

#285
post #60

I don't hear people talk about the design side of services vs libraries enough, but I think it's a huge part of the picture. Many people reach for services too fast because they feel more comfortable thinking about APIs through the lens of HTTP verbs and resource URLs than they do in the comparatively infinite garden of options inside a program. The REST paradigm, despite most people not understanding, needing, or ut…

I think the flow architecture (i.e. Redux) is sufficiently descriptive for how to organize a project, at least compared to REST. There is one giant state object which is essentially a struct/trivially serializable to JSON. This state can only be changed via actions/events which are themselves only structs, not full objects. Reducer function(s) transform the old state into a new state using these events (in practice people use multiple reducer functions which are chained together).

This isn't even message passing really since you do have global state, it's just that this global state is changed via messages/events.

Re: Write libraries instead of services, where possible

#286

Earlier quoted context omitted.

> So why can't you just write your language support library in whatever language you like, wrap that in something that supports the C ABI if it doesn't already, then call that from your editor? You can! This is called "defining an API" and this is basically what an LSP is. The downsides of using the C ABI like I said is not all programming languages use the C ABI. To work around this you have suggested writing a wrap…

This still doesn't seem superior to library calls from a complexity point of view. On one hand you have a library to secure. On the other hand you have the same library (or at least the same logic and methods) now with a JSON-RPC server wrapping it, and both need to be secured. I would feel a lot better if it weren't JSON, I guess. Binary protocols are just SO MUCH FASTER and require so much less memory. Parsing JSON…

I'd also be much happier if LSPs were in Thrift or gRPC (something more generic with existing client/server generators).

Would make documenting, updating, and understanding the protocol much easier.

Re: Write libraries instead of services, where possible

#287
post #282

Earlier quoted context omitted.

In my experience, this is also a major benefit of running Akka on Scala or Java. I had the realization that it's basically a single-language Kubernetes, with some really nice abstractions built on top.

> it's basically a single-language Kubernetes Well, yes and no. By using Kubernetes, you get a scalable infrastructure. By using OTP/Akka, you get a scalable application. While there is common problems to both domains, they are still 2 different domains. For example, using only Kubernetes, you won't have the ability to react to a Pod restart within your application (unless your application is aware of Kubernetes). Us…

Thanks! I was just going to point out that Elixir/Phoenix + Libcluster + K8s is like a match made in heaven. I haven't tried Horde yet but I'm quite intrigued now.

Re: Write libraries instead of services, where possible

#288

Earlier quoted context omitted.

BEAM will automatically distribute processes across a cluster, right?

No, typically you register a node and instruct it on what processes to run. But there are libraries to help instrument this kind of behavior. For elixir: - https://github.com/derekkraan/horde - https://github.com/bitwalker/swarm

A zero-cost alternative that has worked well for me so far is to use a front-end load balancer to distribute requests to multiple Phoenix instances (in k8s), and then just let those requests' background tasks run on the node that starts them.

The whole app is approximately a websocket-based chat app (with some other stuff), and the beauty of OTP + libcluster is that the websocket processes can communicate with each other, whether or not they're running on the same OTP node.

Re: Write libraries instead of services, where possible

#289
post #137
post #75

Earlier quoted context omitted.

SOAP doesn't have anything like the restrictions rest has, does it?

"REST" these days is a defacto code name for "(JSON?) calls over HTTP", with loosey-goosey adherence to anything close to what the original REST author meant. At this point, we might as well call it RPC-style organized calls over HTTP using JSON representations and defined by "Swagger" files. Likewise, most of SOAP was "RPC-style organized calls over HTTP (or TCP) using mostly XML serialization and defined by XSDs. Y…

I think JSON is an improvement over XML for RPC, though they are similar. Generic JSON serialization/deserialization is trivial for any language with arrays and string maps. With XML, are fields serialized as attributes or as tags? Unless you are doing something really weird, JSON tells you just to use a JSON object. And for lists: do you use a special like take for the items or do you skip the intermediate step and just assume each child item is a list?

JSON is definitely not perfect and has similar problems with things like serializing maps with objects as keys, but it does have more opinionated ways for serializing things than XML does.

Re: Write libraries instead of services, where possible

#290

Phoenix is practically built with this idea in mind. My startup is built out of a single phoenix monolith. Only external dependency is postgres. Despite this, I've managed to completely avoid all the typical scaling issues of a monolith. Need a service or background worker? I juts make a Genserver or Oban worker and mount it in the supervision tree. The Beam takes care of maintaining the process, pubsub to said servi…

How do you keep track of background jobs? Is the queue persisted on disk somewhere? If it's not and a background worker crashes for some reason, then is the job lost?

Oban uses Postgres for persistence https://hexdocs.pm/oban/Oban.html
Post reply on HN