Viewing profile — alonisser

How we've made infrastructure boring (e.g self served by developers, as part of their own service helm)

Our Databricks-based CDC processing job was lagging during traffic spikes and wasting money during quiet periods. Here’s how we taught it to scale itself.

Working around the gotchas, logging inside a pandas_udf, etc

How I've reduced processing and costs by 80% with a simple script

Lesson learned the hard way + Conway's law

What I’ve learned about spark streaming checkpointing, how it’s implemented, and how the implementation details of offsets might lead to surprising behavior.

What is dominating the COVID-19 online discourse in US cities and how can local governments address it? @Zencityio analyzed 1.5M social media interactions to extract people’s main …