Principles for Fast Tokio Applications
dial9-rs.github.io
Principles for Fast Tokio Applications
1–10 of 72 posts
Re: Principles for Fast Tokio Applications
#2One great use of agentic coding is being able to add and very granular tracing instrumentation to help with these sort of optimizations.
Re: Principles for Fast Tokio Applications
#3All of the significant server applications I have encountered in the industry have suffered from the same problem, which surprised their authors but seemed obvious to me: the application was spending the majority of its CPU time doing meta-work like entering and leaving epoll, stealing work from itself, etc. There are principles for writing Tokio servers and these are good points in the OP but I think they are little-known and too easy to violate.
Re: Principles for Fast Tokio Applications
#4One great use of agentic coding is being able to add and very granular tracing instrumentation to help with these sort of optimizations.
Also a great way to make sure that your app spends most of its time in observability overhead. For example even the latency histogram that the OP mentions is wildly expensive.
Re: Principles for Fast Tokio Applications
#5One great use of agentic coding is being able to add and very granular tracing instrumentation to help with these sort of optimizations.
Also a great way to make sure that your app spends most of its time in observability overhead. For example even the latency histogram that the OP mentions is wildly expensive.
If you’re not using eBPF to trace your app you’re doing it wrong.
Re: Principles for Fast Tokio Applications
#6Earlier quoted context omitted.
Also a great way to make sure that your app spends most of its time in observability overhead. For example even the latency histogram that the OP mentions is wildly expensive.
If you’re not using eBPF to trace your app you’re doing it wrong.
The low cost of eBPF tracing is another myth.
Re: Principles for Fast Tokio Applications
#7All of the significant server applications I have encountered in the industry have suffered from the same problem, which surprised their authors but seemed obvious to me: the application was spending the majority of its CPU time doing meta-work like entering and leaving epoll, stealing work from itself, etc. There are principles for writing Tokio servers and these are good points in the OP but I think they are little…
I can't say I'm surprised when I see the 100+ function stack traces that Axum built on Tokio produces.
Before you say Axum is "holding it wrong" the project lives under the tokio-rs GitHub org.
Re: Principles for Fast Tokio Applications
#8One great use of agentic coding is being able to add and very granular tracing instrumentation to help with these sort of optimizations.
Also a great way to make sure that your app spends most of its time in observability overhead. For example even the latency histogram that the OP mentions is wildly expensive.
One legitimately great thing about LLMs is that it makes it feasible to add these kind of tracing instrumentations temporarily for profiling and then throw them away so they never reach source control let alone production.
Re: Principles for Fast Tokio Applications
#9Earlier quoted context omitted.
Also a great way to make sure that your app spends most of its time in observability overhead. For example even the latency histogram that the OP mentions is wildly expensive.
One legitimately great thing about LLMs is that it makes it feasible to add these kind of tracing instrumentations temporarily for profiling and then throw them away so they never reach source control let alone production.
I can get an LLM to trace my incomprehensible Tokio application which was also written by an LLM, which is why I don't understand its behavior. Truly the future we were promised.
Re: Principles for Fast Tokio Applications
#10One great use of agentic coding is being able to add and very granular tracing instrumentation to help with these sort of optimizations.
Also a great way to make sure that your app spends most of its time in observability overhead. For example even the latency histogram that the OP mentions is wildly expensive.
That just sounds like bad tracing implementations. A good tracing implementation should be able to drive gigabytes per second of trace logs to memory. If you are generating it slow enough to allow actual offload then you should be in the 1—10% range even if you are saturating your offload.
You should, of course, upper bound this overhead by switching to a full time travel debugging solution, thus tracing everything, when you get to the 10-30% range.
The only way you get to “majority” is if your trace implementation is slower than time travel debugging and provides less information, but then why choose something worse in every dimension.