Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

High-Performance server for NATS.io, the cloud and edge native messaging system

github.com

Post reply on HN
Print view View original

21–30 of 75 posts

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#21

Post by lakomen »

I got into it again about 2 weeks ago. Been doing a small project with the microservice api.

Compared to HTTP/REST I miss the debugabilty of it. Since all messages are sent over websocket and received, in binary form, since there is no support from Chrome or Firefox, one has to tediously manually extract the json payload and try to make sense of it.

Jetstream is still quite the mystery. It just doesn't want to work the way I want it to, especially on the JS client side, blocking script execution and timing out.

Then there's auth, after a few shutdowns and reboots I got locked out of my local installation. Oidc subject to nats user mapping doesn't exist and has to be done manually.

So TL;DR the core functionality is great. Everything else seems to be WIP.

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#23

Post by [deleted] »

zmmmmm wrote: post #20

Struggling to figure out what "cloud native" actually means. Unfortunately their own web site doesn't say and the CNCF web site they link to leaves me none the wiser. At a guess they are talking about applications being built from the ground up to dynamically allocate resources using cloud providers APIs directly rather than relying on an assumption fixed resources are already provisioned and the application runs wit…

[deleted]

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#24

Post by jerrygenser »

zmmmmm wrote: post #20

Struggling to figure out what "cloud native" actually means. Unfortunately their own web site doesn't say and the CNCF web site they link to leaves me none the wiser. At a guess they are talking about applications being built from the ground up to dynamically allocate resources using cloud providers APIs directly rather than relying on an assumption fixed resources are already provisioned and the application runs wit…

There are a bunch of OSS that solve problems when running distributed scale out workloads in "the cloud". For example prometheus is a metric db for storing monitoring data.

Ceph is a storage backend. Most of them tend to be built for or work well with kubernetes.

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#25

Post by vosper »

protochron wrote: post #15
vosper wrote: post #8

Earlier quoted context omitted.

Any thoughts on NATS' JetStream? Looks quite compelling, to me. https://docs.nats.io/using-nats/developer/develop_jetstream

Jetstream is the Kafka or Kinesis bit built out on top of the NATS core protocol. It's much easier to work with than either of them in my experience

Thanks, do you know if it's a good idea to use JetStream's KV or Object stores to implement a join? I'm thinking of "fan-in", where multiple streams carrying different pieces of data about the same object all meet in the same place, and the output is a unified object carrying all those bits of data. Seems the store could be used by the joining consumer to hold onto the various pieces of data related to an object until it has the full set, then emit the object into the next stream?

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#26

Post by yes29922 »

vosper wrote: post #25
protochron wrote: post #15

Earlier quoted context omitted.

Jetstream is the Kafka or Kinesis bit built out on top of the NATS core protocol. It's much easier to work with than either of them in my experience

Thanks, do you know if it's a good idea to use JetStream's KV or Object stores to implement a join? I'm thinking of "fan-in", where multiple streams carrying different pieces of data about the same object all meet in the same place, and the output is a unified object carrying all those bits of data. Seems the store could be used by the joining consumer to hold onto the various pieces of data related to an object unti…

At this point, where you are doing KSQL level stuff, isn't it better to just use Kafka, NATS is very appealing for it's simplicity but if I reach a point of re-implementing KSQL or spending 3 hours setting up zookeeper and understanding how to do Kafka distributed for a devops point then Im choosing the latter

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#27

Post by jadbox »

atombender wrote: post #7
jitl wrote: post #2

With all the message queue whatever thingies, as an outside I’m quite confused about ideal use cases. NATS vs MQTT vs Kafka vs Redis Queue vs Amazon SQS - how do they all stack up?

NATS is not a queue. It's distributed pub/sub message broker for communicating between applications. NATS's only responsibility is to route messages in near-real-time from publishers to consumers. Messages are ephemeral and dropped immediately after delivery; if nobody is listening, the messages vanish. Messages are only queued temporarily in RAM if the consumer is busy, and they can get dropped if a consumer doesn't…

Isn't it a big problem if readers miss messages if either the topic readers go offline briefly or message throughout exceeds readers?

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#28

Post by re-thc »

zmmmmm wrote: post #20

Struggling to figure out what "cloud native" actually means. Unfortunately their own web site doesn't say and the CNCF web site they link to leaves me none the wiser. At a guess they are talking about applications being built from the ground up to dynamically allocate resources using cloud providers APIs directly rather than relying on an assumption fixed resources are already provisioned and the application runs wit…

> Struggling to figure out what "cloud native" actually means.

- very fast startup

- low memory

- can be easily distributed or is stateless

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#29

Post by devmunchies »

lakomen wrote: post #21

I got into it again about 2 weeks ago. Been doing a small project with the microservice api. Compared to HTTP/REST I miss the debugabilty of it. Since all messages are sent over websocket and received, in binary form, since there is no support from Chrome or Firefox, one has to tediously manually extract the json payload and try to make sense of it. Jetstream is still quite the mystery. It just doesn't want to work t…

You can subscribe to all topics by using a *. Why not have a debug subscriber that listens to all messages and plays it in the terminal?

I agree that the auth system is cumbersome. I wanted to use it on the edge for IoT devices where the device only has the same permissions as the user who the device belongs to, but not very easy. Their auth isn’t very customizable.

Re: High-Performance server for NATS.io, the cloud and edge native messaging system

#30

Post by protochron »

vosper wrote: post #25
protochron wrote: post #15

Earlier quoted context omitted.

Jetstream is the Kafka or Kinesis bit built out on top of the NATS core protocol. It's much easier to work with than either of them in my experience

Thanks, do you know if it's a good idea to use JetStream's KV or Object stores to implement a join? I'm thinking of "fan-in", where multiple streams carrying different pieces of data about the same object all meet in the same place, and the output is a unified object carrying all those bits of data. Seems the store could be used by the joining consumer to hold onto the various pieces of data related to an object unti…

You could do something in that vein, but it might be easier to have those streams all actually deliver to a set of consumers and write a bit of code to join them back together. https://natsbyexample.com/examples/jetstream/multi-stream-co... has an example (also a great resource for learning about NATS use cases!)
Post reply on HN