Why ? Between 2008 and 2013, Lokad provided a data forecasting API : send us your sales history and we'll respond with a good sales forecast for each product. This didn't work because:
- most customers don't have the know-how or infrastructure to clean up their sales history (for instance, by setting up adequate SQL queries when exporting data to our services), and instead either sent us garbage data, which led to garbage results and therefore lost us customers, or they cleaned up the data manually in Excel, which prevented us from being a permanent, automated step in their supply chain processes, and therefore lost us customers.
- most customers don't care about knowing the future, they care about knowing how much they have to order from their suppliers. These decisions can certainly be optimized if you have a good model of what the future looks like, but that involves even more knowledge about the customer's business and even more data (expected margins for products, current stock levels, minimum order quantities for various suppliers, etc).
Most of our sucesses in this early period were premium customers for which we could afford to write and run custom C# code that did the initial data cleanup and pre-processing, and then used the forecasts to generate a list of actual business decisions.
As you might guess, C# is not the best language for this. It costs too much to write, deploy, maintain and run. We examined the various candidates available at the time and decided to write our own language, with the following characteristics:
- 100% online, no local setup needed
- excellent static analysis to detect issues ahead of time (no type errors, null references, off-by-one or out-of-bounds errors, join arity surprises, etc)
- good high-level guarantees (no infinite loops, no accidentally quadratic operations, fully transactional behaviour)
- fast enough without manual optimization
- predictable performance (small changes in the input data should not cause major changes in execution speed)
- handle the "hard developer stuff" behind the scenes (multi-threading, memory management, error recovery, caching intermediate results to make REPL faster, etc)
- our various high-performance black boxes (such as demand forecasting) are available as functions in the language
- a small, well-designed (supply chain) standard library