What does your product/service do? Ad tech How did you develop and launch it? New features for an existing product. Most of them were switched on globally when we launched them. Which LLM(s) are you using (e.g. GPT-4, Claude, open source models)? We started using language models before ChatGPT and friends came out. We built our models from scratch, and still use and update them. What's your revenue model? Saas How mu…
Would love to read an MLOps lessons learned or approach you had or if you recommend any particular books that really hit the nail on the head
Some things that have were notable:
Model pipelines tend to be flakier than other pipelines you have. They are much more complicated, and it can be easy to hit a resource limit if you aren't careful, or have a unhandled exception accidentally kill a pipeline 10hrs into it.
Avoiding those outright is obviously the best path, but that can be easier said than done.
One thing that we found really helpful was creating an error record in a database for every piece of data that failed to get processed, where it failed in the pipeline, etc. Retries, and alters were easy to tack on after that.