Congrats Ken and Chris! I think Patterns is very interesting, having closely followed since the early days (but not affiliated).
For those interested, I've been doing some exploring around automation tools more broadly, and this is my attempt at orientation, within the context that Patterns exists (my own interpretation).
The paradigm is within the context of data analysts and data engineers required to centralise data for reporting and automation (see "analytical + operational workflows" on the Patterns website). This process tool chain broadly looks like the following aggressively reduced flow of core components:
`Extract > Integrate > Analyse/Report > Automate` (and Ops)
- Extract: Regularly pull data into a centralised storage/compute location from multiple source systems such as SaaS APIs, databases, logs etc (eg Airbyte)
- Integrate: Combine data into a coherent unified standardised data warehouse (data warehouse is a thing you build not a tool you buy) (eg dbt, Pandas)
- Analyse: Explore data, discover facets/aspects/dimensions in the combined data. Typically notebook type tools, and includes data science and analytics (eg Jupyter)
- Reporting: Dashboards and regular reports on formalised data concepts such as metrics, using BI tools (eg Metabase)
- Automation: Tools that exist to trigger actions on the data available in the system, typically by sending those actions into other systems. (eg Patterns!, also Zapier et al, which are more consumer oriented, no version control etc)
- Ops: The tools needed to effectively achieve the above, many sub-categories (eg Airflow)
---
Some observations to then mention, specifically about Patterns/Automation:
1. Probably best to be used in conjunction with the previous components. Automation tools seemingly can exist without, but the Integrate, Analyse and Reporting are specific, highly related and likely required processes at any kind of scale (team, data volume or complexity) at least this has been true historically, and companies have deployed significant resources to implement these tools.
2. However, one obviously can use just an Automation tool alone, as they provide the Ops and technical components to run the full `Extract > Integrate > Analyse/Report > Automate` process, and possibly with the least complexity, and the best containment of abstractions. This is a huge gain for small teams with limited resources.
3. My reservation for bigger teams is the "Integrate" component, which if not done carefully (regardless of where/how) leads to a mountain of technical debt to try and maintain the data transformations (data modelling), and nothing but care solves this very time consuming process
4. Data is stored on Patterns, and it would be interesting to know how users with pre-existing processes would extract data into other tools, say for example to write scored lead IDs into another system for targeting (see Reverse ETL tools).
5. Most Automation tools lack a real "user input" component by default, as they seem not to be designed to build user-interface CRUD apps per-se. This is similar to Hex.Tech (as far as I can tell?) which has an "app" interface, but users cannot really change the state of the app. If they reload the page, their inputs will be lost (I think I am correct here for Patterns - could a "non technical user" change the lead scoring parameters without getting into the code?) Feels like a simple feature, but probably very complex to implement.
6. The Patterns on face value feels like a better way to deploy data analytics in terms of what users are trying to achieve (Create and share insight, and automate in specific instances), with nice abstractions over storage, streams, scheduling, compute (none of which are worth direct contact for an analyst if they can avoid)
Distinct but possibly worthwhile mentioning these categories:
- Reverse ETL: A similar desire to Automation; send the data back to a tool that can use it to automate something
- No-code: providing CRUD App development capabilities without (much) code (eg Retool)
- Feature Flag/Remote Config: Provide non-technical users with an interface into the configuration of a web-app (eg Flagsmith)