Live data from Hacker News

The service mesh era: Using Istio and Stackdriver to build an SRE service

cloud.google.com

21–30 of 33 posts

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#21
post #8

I would interested in anyones feedback of embracing and rolling out a service mesh/Istio in a non-GCP environment. I apologize if this is a naive question but how come this wasn't included as part of the Kubernetes project given that it has the same Google origins?

The Istio project is not supposed to be tied to Kubernetes. It is supposed to be a general-purpose service mesh.

That being said, I have been looking for a while and I can't find anyone who uses it in production on a platform other than Kubernetes.

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#22
post #17

Earlier quoted context omitted.

We have hundreds of customers (and thousands of engineers) who are willing to make the trade-off. It's OK that you're not, but I hope you can agree that engineering observability isn't cheap nor easy - and if you're using standard libraries, frameworks, and tooling (and not going way off the rails) we have observed that, for the most part, our agent works as intended. We always recommend our customers run the agent i…

I definitely value ease of use and zero-setup solutions - the issue for me here is that the situations where I would consider running a service mesh are the same situations where predictability and reproducibility would trump ease of use - namely any kind of production setting where down-time is sufficiently undesirable. Testing with the agent would certainly help, but then you lose some of the "ease of use" benefits…

Well, I’m guessing a competitor flagged my OP; but I digress, I was just trying to raise awareness to what is actually possible (even if it’s not free, but I’d argue nothing of value is ever free - even open source, you still have to implement and operate it).

Anyways, I feel like we’ve come to an impasse, there is no monitoring solution out there which is bug-free (even opentracing and it’s various implementations have caused performance/stability issues, re: https://github.com/opentracing-contrib/java-spring-web/pull/...)

Regarding CI, our agent has no requirements other than a supported OS - you could be running your integration tests as a bare JVM and our agent would detect, instrument, and monitor it the same way if it were running inside a CRI-O container on K8S (though I’d question why you would run your integration tests in that manner).

IRT your examples, I’ll be brief in my responses because you’re not wrong, but the engineers on our team have taken great care to ensure we don’t break our customers environments (we run on systems which process 10’s of millions of requests an hour and where minutes of downtime cause losses in the 100s of thousands).

We dynamically unload our sensors/instrumentation when the agent is unloaded - so the likelihood of the issue which was mentioned earlier happening is slim (though nothing is impossible)

We also don’t instrument serialization methods (unless you were to decide to use our SDK to do so) so that’d literally never happen. We hook onto methods which handle communication between systems — HTTP request handlers, DB handlers, Messaging System handlers, Schedulers, etc.

Our sensors are open source, so you can check out the code if you’d like (https://github.com/instana). As I said earlier, we live in a world of trade-offs. I’d argue that systems which require the use of a service mesh are significantly complex enough to warrant the use of this level of automation to provide visibility that quite frankly 99.9% of organizations don’t have the time to do themselves.

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#23
post #8

I would interested in anyones feedback of embracing and rolling out a service mesh/Istio in a non-GCP environment. I apologize if this is a naive question but how come this wasn't included as part of the Kubernetes project given that it has the same Google origins?

The Istio project is not supposed to be tied to Kubernetes. It is supposed to be a general-purpose service mesh. That being said, I have been looking for a while and I can't find anyone who uses it in production on a platform other than Kubernetes.

Also worth noting that Istio is not part of the CNCF while Linkerd is.

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#24
post #7

Earlier quoted context omitted.

OpenCensus seems like it's really a Google-only project. OpenTracing and OpenMetrics appear to have more community and vendor engagement. E.G. Datadog is basing their newer tracing libraries on OpenTracing, and Prometheus devs are behind OpenMetrics.

That's exactly the fragmentation that we don't need. OpenCensus has backing from Microsoft too and is a designed as a single API and library to support both tracing and metrics using the same context. OpenTracing and OpenMetrics are more like API specs with libraries left to others to implement, and they're never really used standalone for them to be separate projects. The best option for the industry would be to fol…

Why is that best for the industry?

Or more precisely, which, industry?

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#25

Earlier quoted context omitted.

The Istio project is not supposed to be tied to Kubernetes. It is supposed to be a general-purpose service mesh. That being said, I have been looking for a while and I can't find anyone who uses it in production on a platform other than Kubernetes.

Also worth noting that Istio is not part of the CNCF while Linkerd is.

It is a good note to make. How does this bear weight on the question above?

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#26
post #9

Earlier quoted context omitted.

There is magic, (caveat: I work here), but Instana ( https://instana.com ) will instrument most major languages and frameworks auto-magically. As in, I don't have to declare a dependency, change a configuration, or anything - our agent finds processes running on the system and bootstraps the libraries while running and monkey patches a huge number of standard libraries and frameworks with no restarts. (don't believe…

That sounds horrific for whoever is going to be supporting that system... The last thing I would want in a production environment is to have some 3rd party software monkey-patching the code at runtime. What happens when: - a bug only occurs (due to timing or some other extremely subtle issue) when this monkey-patching is applied. - there's a bug in the monkey-patching itself (sounds like a fun debugging session!) - a…

Not all, but some other vendor solutions autoinstrument popular languages, frameworks, runtimes as well. Many, many customers leverage these capabilities happily, relying on those engineered solutions (akin to the reliance others place on the engineering put into OC).

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#27
post #10

n00b question, I always see service meshes used in the context of containers and mostly with kube. Would they work with more monolithic/traditional n-tier architecture deployed directly on host OS as well? Or maybe put another way, are there likely to be pain points that don't exist in containerized architectures?

Meshes are there to abstract away stuff that is much more manageable in a monolith. If you have 100 microservices implemented using 5 different platforms you need libraries and programming discipline to implement retries with backoffs, circuit breakers, health checks, tracing, service discovery and other similar stuff in every single one of them.

In a monolith you need to implement some of this stuff only once and you don't need a lot of it at all because you are not making remote procedure calls.

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#28
post #10

n00b question, I always see service meshes used in the context of containers and mostly with kube. Would they work with more monolithic/traditional n-tier architecture deployed directly on host OS as well? Or maybe put another way, are there likely to be pain points that don't exist in containerized architectures?

If your monolith interacts with many other monoliths, service meshes might be useful. If not, maybe not so much.

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#29
post #8

I would interested in anyones feedback of embracing and rolling out a service mesh/Istio in a non-GCP environment. I apologize if this is a naive question but how come this wasn't included as part of the Kubernetes project given that it has the same Google origins?

So kubernetes seems to be one of Googles biggest efforts at really building a healthy open source project for Borg v2. Perhaps they wanted it to be mostly community driven? It’s for this reason that they collaborated with others to introduce ISTIO so the community doesn’t feel like Google is taking over or whatever fears a lot of OSS folks have of the company.

Re: The service mesh era: Using Istio and Stackdriver to build an SRE service

#30

Earlier quoted context omitted.

Also worth noting that Istio is not part of the CNCF while Linkerd is.

It is a good note to make. How does this bear weight on the question above?

He was asking why Istio was not included as part of Kubernetes itself despite both projects originating at Google. I was implying that there must be some reason as Istio is not in the CNCF while Linkerd (arguably an Istio competitor) and Kubernetes are. To further that idea, it seems that Google wants to maintain direct control over Istio itself, rather than put it into an organization with multiple sources of institutional governance (Amazon, Huawei, Samsung, Microsoft, Oracle, etc.). If Istio or any other service mesh had been incorporated directly into Kubernetes by Google, they would have lost some control over it.

(There are also obvious technical reasons for decoupling something like this from Kubernetes, mostly the opinionated nature of forcing a service mesh over other potential solutions).

Post reply on HN