Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

eBPF will help solve service mesh by getting rid of sidecars

isovalent.com

Post reply on HN
Print view View original

61–70 of 110 posts

Re: eBPF will help solve service mesh by getting rid of sidecars

#61

Post by [deleted] »

topspin wrote: post #57
dharmab wrote: post #31

Earlier quoted context omitted.

That's if you're using a NodePort service, which the documentation explains is for niche use cases such as if you don't have a compatible dedicated load balancer. In most professional setups you do have such a load balancer and can use other types of routing that avoid this. https://kubernetes.io/docs/concepts/services-networking/serv...

> In most professional setups you do have such a load balancer May I ask what one might use in an AWS cloud environment to provide that load balancer within a Region? Does IPv6 address any of these issues? It seems to me that IPv6 is capable of providing every component in the system its own globally routable address, identity (mTLS perhaps) and transparent encryption with no extra sidecars, eBPF pieces, etc.

[deleted]

Re: eBPF will help solve service mesh by getting rid of sidecars

#62

Post by dharmab »

topspin wrote: post #57
dharmab wrote: post #31

Earlier quoted context omitted.

That's if you're using a NodePort service, which the documentation explains is for niche use cases such as if you don't have a compatible dedicated load balancer. In most professional setups you do have such a load balancer and can use other types of routing that avoid this. https://kubernetes.io/docs/concepts/services-networking/serv...

> In most professional setups you do have such a load balancer May I ask what one might use in an AWS cloud environment to provide that load balancer within a Region? Does IPv6 address any of these issues? It seems to me that IPv6 is capable of providing every component in the system its own globally routable address, identity (mTLS perhaps) and transparent encryption with no extra sidecars, eBPF pieces, etc.

> May I ask what one might use in an AWS cloud environment to provide that load balancer within a Region?

The AWS cloud controller will automatically set up an ALB for you if you configure a LoadBalancer service in Kubernetes. I've also done custom setups with AWS NLBs.

> Does IPv6 address any of these issues?

It could address some issues- you could conceivably create a CNI plugin which allocates an externally addressable IP to your Pods. Although you would probably still want a load balancer for custom routing rules and the improved reliability over DNS round robin.

Re: eBPF will help solve service mesh by getting rid of sidecars

#63

Post by tptacek »

the_duke wrote: post #37
tptacek wrote: post #26

Earlier quoted context omitted.

The complexity is an issue (but sidecars are plenty complex too), but the security not so much. BPF C is incredibly limiting (you can't even have loops if the verifier can't prove to its satisfaction that the loop has a low static bound). It's nothing at all like writing kernel C.

You don't have to use C. There are two projects that enable writing eBPF with Rust [1][2]. I'm sure there is an equivalent with nicer wrappers for C++. [1] https://github.com/foniod/redbpf [2] https://github.com/aya-rs/aya

It doesn't make any difference which language you use; the security promises are coming from the verifier, which is analyzing the CFG of the compiled program. C is what most people use, since the underlying APIs are in C, and since the verifier is so limiting that most high-level constructions are off the table.

Re: eBPF will help solve service mesh by getting rid of sidecars

#64

Post by tptacek »

tgraf wrote: post #48
Matthias247 wrote: post #40

I understand how BPF works for transparently steering TCP connections. But the article mentions gRPC - which means HTTP2. How can the BPF module be a replacement for a proxy here. My understanding is it would need to understand http2 framing and having buffers - which all sound like capabilities that require more than BPF? Are they implementing a http2 capable proxy in native kernel C code and making APIs to that acc…

The model I'm describing contains two pieces: 1) Moving away from sidecars to per-node proxies that can be better integrated into the Linux kernel concept of namespacing instead of artificially injecting them with complicated iptables redirection logic at the network level. 2) Providing the HTTP awareness directly with eBPF using eBPF-based protocol parsers. The parser itself is written in eBPF which has a ton of sec…

What does an HTTP parser written in BPF look like? Bounded loops only --- meaning no string libraries --- seems like a hell of a constraint there.

Re: eBPF will help solve service mesh by getting rid of sidecars

#65

Post by pjmlp »

jayd16 wrote: post #50
pjmlp wrote: post #34

Earlier quoted context omitted.

It sure scales, I am yet to work in organisations where everything goes. There are a set of sanctioned languages and that is about it.

The subtle aspect of the comment you're replying to is that _they write everything_. Hard to cram a new library into some closed source vendor app.

Depends how it was written and made extensible.

Re: eBPF will help solve service mesh by getting rid of sidecars

#66

Post by malkia »

MayeulC wrote: post #38
zdw wrote: post #8

So instead of making the applications use a good RPC library, we're going to shove more crap into the kernel? No thanks, from a security context and complexity perspective. Per https://blog.dave.tf/post/new-kubernetes/ , the way that this was solved in Borg was: > "Borg solves that complexity by fiat, decreeing that Thou Shalt Use Our Client Libraries For Everything, so there’s an obvious point at which to plug in ar…

> a good RPC library I like that approach. If you use client libraries, new RPC mechanisms are "free" to implement (until you need to troubleshoot upgrades). It's also an argument against statically linking. For instance, if running services on the same machine, io-uring can probably be used? (I'm a noob at this). eBPF for packet switching/forwarding between different hosts, etc.

This may no longer be the case, but back at Google I remember one day having my java library no longer using the client library logger, but spawning some other app and talking (sending logs to it). That other app used to be fat-client, linked in our app, supported by another team. First I was wtf.. Then it hit me - this other team can update their "logging" binary at different cycle than us (hence we don't have to be on the same "build" cycle). All they needed to do for us is provide with very "thin" and rarelly changing interface library. And they can write it in any language they like (Java, c++, go, rust, etc.)

Also no need to be .so/ (or .dll/.dylib) - just some quick IPC to send messages around. Actually can be better. For one, if their app is still buffering messages, my app can exit, while theirs still run. Or security reasons (or not having to think about these), etc. etc. So still statically linked but processes talking to each other. (Granted does not always work for some special apps, like audio/video plugins, but I think works fine for the case above).

Re: eBPF will help solve service mesh by getting rid of sidecars

#68

Post by zaphar »

hestefisk wrote: post #46
__alexs wrote: post #29

Earlier quoted context omitted.

I'm sure someone will write leftPad in eBPF any day now.

Indeed. We could even embed a WASM runtime (headless v8?) so one can execute arbitrary JavaScript in-kernel… wait :)

eBPF is far too limited to run a WASM runtime. That's why the proposed article approach is even possible.

Re: eBPF will help solve service mesh by getting rid of sidecars

#69

Post by kklimonda »

q3k wrote: post #7
dijit wrote: post #3

Honestly after I learned that the majority of Kubernetes nodes just proxy traffic between each other using iptables and that a load balancer can't tell the nodes apart (ones where your app lives vs ones that will proxy connection to your app) I got really worried about any kind of persistent connection in k8s land. Since some number of persistent connections will get force terminated on scale down or node replacement…

> Honestly after I learned that the majority of Kubernetes nodes just proxy traffic between each other using iptables and that a load balancer can't tell the nodes apart (ones where your app lives vs ones that will proxy connection to your app) I got really worried about any kind of persistent connection in k8s land. There can be a difference, if your LoadBalancer-type service integration is well implemented. The ext…

How do you keep ecmp hashing stable between rollouts?

Re: eBPF will help solve service mesh by getting rid of sidecars

#70

Post by zinclozenge »

It's not clear how eBPF will deal with mTLS. I actually asked that when interviewing at a company using eBPF for observability into Kubernetes the answer was they didn't know.

Yea, if you're getting TLS termination at the load balancer prior to k8s ingress then it's pretty nice.

Post reply on HN