"Observability" existed 20 years ago and it consisted in application metrics, OS metrics, log analysis and correlation, SIEM. It was called monitoring.
which i think is the point of the article as well. :)
It very much is :-)
Observability vs. monitoring debate: An irreverent view
11–19 of 19 posts
Re: Observability vs. monitoring debate: An irreverent view
#12Both concepts are related but separate. I'd argue that monitoring is the act of alerting or taking action when some metric is out of bounds with normal values. Those values are based on absolute thresholds, or some changing thresholds based on historical values. Observability is the ability to see what's happening across and between multiple systems so that any issues raised by monitoring can be diagnosed quickly. "I…
I very much agree: observability is a property of a system, like transparency is for glass. Monitoring is what you need to do with a lot of systems to make them observable.
Re: Observability vs. monitoring debate: An irreverent view
#13Observability isn't just a rebranding of Monitoring, it's Monitoring + making it the most actionnable possible via standardization.
Specifically, how to make the sum of all monitored "pillars" more useful than each of them individually.
3 major pillars being:
- Metrics (whether application or higher-level of the stack, like OS)
- Logs (whether structured or unstructured)
- Traces
Observability is these major pillars and how to easily "jump" from one to another to very quickly identify the root cause of an issue. I.e. go Metrics Logs, Logs Traces, or Metrics Traces,
For instance, with good Metrics, one can easily figure out & get alerts when there is a large spike of 500 errors. But when Metrics & Logs can work together, one can easily see the exception from stack trace that are emitted with those 500 errors.
Similarly, with good Metrics, one can easily figure out that the frontend service latency p90 has increased by 5x. But with Metrics & Traces working together(for instance via Exemplar[1]), one can look at a bunch of the traces that have a very high latency, and identify the upstream service responsible for this increase.
With Monitoring only, you could get a nice Metrics solution in place, with fancy alerting rules, but all it was good at is informing you "Something bad is currently happening". With a good "Observability" setup, you should also be able to change it to "Something bad is currently happening and the root cause is right here."
[1] https://grafana.com/docs/grafana/latest/basics/exemplars/
Re: Observability vs. monitoring debate: An irreverent view
#14Observability isn't just a rebranding of Monitoring, it's Monitoring + making it the most actionnable possible via standardization. Specifically, how to make the sum of all monitored "pillars" more useful than each of them individually. 3 major pillars being: - Metrics (whether application or higher-level of the stack, like OS) - Logs (whether structured or unstructured) - Traces Observability is these major pillars…
logs, traces and profiling were all viable parts of a good monitoring stack even prior to the term observability being coined.
Re: Observability vs. monitoring debate: An irreverent view
#15I see OpenTelemetry as an application of the same idea pushed by Google via the k8s revolution. Create a great vendor-agnostic open source tech. Get everyone riled up about the dangers of vendor-locking solutions. Use the new tech to carve yourself a piece of the market from the current incumbent. It is pretty great and all, but sometimes it is easier to build your app with a simple vendor-locked tech than a super ge…
> It is pretty great and all, but sometimes it is easier to build your app with a simple vendor-locked tech than a super generic agnostic technology.
That's for sure golden. Until the product is bought over, or there was a merge of companies, or you name it. And then you end up with a pile of products with different vendor-locked log formats, metrics. At that point you'd like to get some standardization. And OpenTelemetry is a perfect candidate for common ground. Thus support of OpenTelemetry becomes a major decision factor when selecting a vendor or OSS solution for your problem, isn't it?
Re: Observability vs. monitoring debate: An irreverent view
#16Re: Observability vs. monitoring debate: An irreverent view
#17Observability isn't just a rebranding of Monitoring, it's Monitoring + making it the most actionnable possible via standardization. Specifically, how to make the sum of all monitored "pillars" more useful than each of them individually. 3 major pillars being: - Metrics (whether application or higher-level of the stack, like OS) - Logs (whether structured or unstructured) - Traces Observability is these major pillars…
That’s trope #1 right there :-)
Re: Observability vs. monitoring debate: An irreverent view
#18I see it as a tiered approach - Detailed Process Logs saved to a db, Audits, Regular logs streamed to your favorite log dump, Monitoring, Health Checks, Metrics, and Usage Statistics are all part of Observability. If it is a critical process, use detailed process logs. If it is an extremely critical process add Auditing.
Regular logs (text files) > Detailed Process Logs (DB) > Audits (read-only/tamper-proof)
Are Audits more compliance related, while detailed logs are business only?
It's not clear to me why to have a special detailed process log, if you can you can write important data to Audits, and less important to regular logs, i.e. once can use different log levels for business data vs engineering/debug data.
Can you please give an example?
Re: Observability vs. monitoring debate: An irreverent view
#19I see it as a tiered approach - Detailed Process Logs saved to a db, Audits, Regular logs streamed to your favorite log dump, Monitoring, Health Checks, Metrics, and Usage Statistics are all part of Observability. If it is a critical process, use detailed process logs. If it is an extremely critical process add Auditing.
Regular logs (text files) > Detailed Process Logs (DB) > Audits (read-only/tamper-proof) What's the difference between Regular logs, Detailed Process Logs and Audits? Are Audits more compliance related, while detailed logs are business only? It's not clear to me why to have a special detailed process log, if you can you can write important data to Audits, and less important to regular logs, i.e. once can use differen…
Regular logs are just whatever your app logs normally (engineering/debug).
Process logs are business processes that you need to report on or answer for later.
Audits are another business layer for safety or life critical data, or adversarial data (handling money based transactions). They could be compliance related. Auditing is a three step process - Ask a question, figure out how to answer it, and make sure you've answered it.
So specifically, if you have a magic bean shop and you want to log transfer of the beans, that would be a process log. Then at the end of the day, you would send a manifest of all the beans sent that day. The audit process would look at the final location of the beans, and also at the process log to answer your audit questions.
---
I think a lot of people make a hard distinction between business logging and engineering logging, and I'm not sure that makes sense.