Show HN: Cost-aware network traffic analysis

This is one of the authors and developers of the system.

Traffic Refinery is a cost-aware network traffic analysis library implemented in Go. It works at line rate up to 10gbps. Traffic Refinery is tailored towards the development of Machine Learning models for network inference. To achieve this goal, the system allows to 1) collect different statistics from captured traffic and 2) profile the cost of the collection of each feature to better understand how feasibly such collection would be in production. The system uses a conveninent configuration system to specify which subportions of traffic you want to track and what statistics to collect from the flows belonging to the observed traffic. Subportions of traffic are specified using DNS properties (e.g., domain names) or by IP addresses. The systems comes with standard traffic statistics built-in and ready to be collected (e.g., packet/byte counters, jitter) but new ones can be easily implemented by the simple writing of a go function. Examples are available in the documentation.

We are happy to answer any questions either here or via email, and solve any issue you might have using the github repo issues page.

10gbps, amazing!

These comments are rather curious