Live data from Hacker News

High-Performance server for NATS.io, the cloud and edge native messaging system

github.com

51–60 of 75 posts

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#51
post #21

I got into it again about 2 weeks ago. Been doing a small project with the microservice api. Compared to HTTP/REST I miss the debugabilty of it. Since all messages are sent over websocket and received, in binary form, since there is no support from Chrome or Firefox, one has to tediously manually extract the json payload and try to make sense of it. Jetstream is still quite the mystery. It just doesn't want to work t…

Nats provides a nice CLI that can help with debugging: https://github.com/nats-io/natscli

Besides that I'm also working on a UI solution, that will help to get better overview of your cluster: https://qaze.app/

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#52
post #48

We have had pretty bad experience with NATS. Apart from core NATS which is just in memory, you need their sub modules to build any kind of serious business on top of it. We initially built our code around NATSStreaming, they then went ahead and deprecated that. But since we saw so many big companies using NATS, they thought it might be a good idea to stick with it and we did a year long migration to their shiny new N…

From what I've seen it's too early to migrate from Push. With the last update they emphasized Pull as the default, but Push is not deprecated yet.

And even NATS Streaming still works: it is deprecated, not removed.

Unlike databases, a good thing about messaging and streaming solutions is that you don't have to pick one: you can make them talk to each other as long as there are bridges. This also applies to different approaches to messaging/streaming provided by a single platform.

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#53
post #20

Struggling to figure out what "cloud native" actually means. Unfortunately their own web site doesn't say and the CNCF web site they link to leaves me none the wiser. At a guess they are talking about applications being built from the ground up to dynamically allocate resources using cloud providers APIs directly rather than relying on an assumption fixed resources are already provisioned and the application runs wit…

> Struggling to figure out what "cloud native" actually means.

Same as "web scale" and "big data".

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#54
post #27

Earlier quoted context omitted.

NATS is not a queue. It's distributed pub/sub message broker for communicating between applications. NATS's only responsibility is to route messages in near-real-time from publishers to consumers. Messages are ephemeral and dropped immediately after delivery; if nobody is listening, the messages vanish. Messages are only queued temporarily in RAM if the consumer is busy, and they can get dropped if a consumer doesn't…

Isn't it a big problem if readers miss messages if either the topic readers go offline briefly or message throughout exceeds readers?

You wouldn't/shouldn't choose NATS if your application needs different semantics.

NATS does not have stateful storage. So when a consumer disconnects and reconnects, there is nowhere for NATS to store the messages temporarily. You can solve this by storing messages in a stateful storage first, then use NATS as a way to distribute them. You would need your own mechanism to replay messages on reconnect. This is coincidentally what Jetstream does. It uses NATS internally as a network protocol, but it's a separate thing.

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#55
post #52
post #48

We have had pretty bad experience with NATS. Apart from core NATS which is just in memory, you need their sub modules to build any kind of serious business on top of it. We initially built our code around NATSStreaming, they then went ahead and deprecated that. But since we saw so many big companies using NATS, they thought it might be a good idea to stick with it and we did a year long migration to their shiny new N…

From what I've seen it's too early to migrate from Push. With the last update they emphasized Pull as the default, but Push is not deprecated yet. And even NATS Streaming still works: it is deprecated, not removed. Unlike databases, a good thing about messaging and streaming solutions is that you don't have to pick one: you can make them talk to each other as long as there are bridges. This also applies to different…

Just like you don't want to support tens of different databases you shouldn't build your solution on lots of bridged message busses. If stuff goes down at 2 am you don't want to have to find out which message queue was the problem or what tooling you should use to debug it.

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#56
post #55
post #52

Earlier quoted context omitted.

From what I've seen it's too early to migrate from Push. With the last update they emphasized Pull as the default, but Push is not deprecated yet. And even NATS Streaming still works: it is deprecated, not removed. Unlike databases, a good thing about messaging and streaming solutions is that you don't have to pick one: you can make them talk to each other as long as there are bridges. This also applies to different…

Just like you don't want to support tens of different databases you shouldn't build your solution on lots of bridged message busses. If stuff goes down at 2 am you don't want to have to find out which message queue was the problem or what tooling you should use to debug it.

Sure, but it's an inconvenience, not a total blocker. With databases you lose foreign keys and you have to replace a single query with N. And you don't want to have ten different solutions of course, but two is fine and allows for comfortable migrations.

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#57
post #21

I got into it again about 2 weeks ago. Been doing a small project with the microservice api. Compared to HTTP/REST I miss the debugabilty of it. Since all messages are sent over websocket and received, in binary form, since there is no support from Chrome or Firefox, one has to tediously manually extract the json payload and try to make sense of it. Jetstream is still quite the mystery. It just doesn't want to work t…

You can subscribe to all topics by using a *. Why not have a debug subscriber that listens to all messages and plays it in the terminal? I agree that the auth system is cumbersome. I wanted to use it on the edge for IoT devices where the device only has the same permissions as the user who the device belongs to, but not very easy. Their auth isn’t very customizable.

This is actually a good idea, I'll try that, thanks :)

But, you know, it's still a bit more complicated than opening the browser's debug console and inspecting request and response in the networking tab.

Yeah you have operators, which are essentially the auth admins or orgs, however you'd like to look at it, then there are accounts which are an alias for "project name" and then you have users, which are "client name"s. And that's fine for their infrastructure only. The problem is, in the real world you have an external identity service (OIDC, IAM,...) and the JWT this service creates includes a subject, but the NATS auth system has no support for external bridges. Also you have to decide, would you like to use operator/account/user or single JWT, or passphrase or manually managed user/password accounts? In a professional world, you'd have various customer applications each divided by their operators, accounts and users. And then fine grained user capabilities.. headache time.

So what you have to do is distribute a default user certificate with a client (client meaning actual client in the client/server sense), then do the signin/sso process and have some middleware checking this token from the OIDC auth process and check is a user with this subject exists, if not, create a new user and cert and send that cert back to the client which will then create a new connection with the new certificate.

Very complicated.

The initial default client certificate is required for "logging in" via NATS. I guess it could be a standalone program, the oidc to nats user mapper/manager.

So compared to HTTP/REST, more work involved.

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#58
post #27

Earlier quoted context omitted.

Isn't it a big problem if readers miss messages if either the topic readers go offline briefly or message throughout exceeds readers?

You wouldn't/shouldn't choose NATS if your application needs different semantics. NATS does not have stateful storage. So when a consumer disconnects and reconnects, there is nowhere for NATS to store the messages temporarily. You can solve this by storing messages in a stateful storage first, then use NATS as a way to distribute them. You would need your own mechanism to replay messages on reconnect. This is coincid…

Jetstream is distributed as an optional part of NATS, so while it exists as a layer on top of NATS Core and provides a different model, describing NATS as entirely stateless is not totally accurate

Though I suspect that Synadia people might do at least one more minor iteration of Jetstream: it still seems a little more complex than it needs to be, unlike Core

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#59
post #20

Struggling to figure out what "cloud native" actually means. Unfortunately their own web site doesn't say and the CNCF web site they link to leaves me none the wiser. At a guess they are talking about applications being built from the ground up to dynamically allocate resources using cloud providers APIs directly rather than relying on an assumption fixed resources are already provisioned and the application runs wit…

> applications being built from the ground up to dynamically allocate resources using cloud providers APIs directly If that indeed is what cloud native means, it sounds interesting. But the problem is that all these APIs and especially "managed" services are super proprietary and you'll vendor lock yourself pretty hard. But I suppose that ship has sailed a long time ago.

NATS is 100% open source

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#60
post #2

With all the message queue whatever thingies, as an outside I’m quite confused about ideal use cases. NATS vs MQTT vs Kafka vs Redis Queue vs Amazon SQS - how do they all stack up?

NATS is not a queue. It's distributed pub/sub message broker for communicating between applications. NATS's only responsibility is to route messages in near-real-time from publishers to consumers. Messages are ephemeral and dropped immediately after delivery; if nobody is listening, the messages vanish. Messages are only queued temporarily in RAM if the consumer is busy, and they can get dropped if a consumer doesn't…

[deleted]
Post reply on HN