Earlier quoted context omitted.
100% agree. And we are not there yet. But I feel like open source is the best answer to 1 (building an ecosystem of integrations), and that it will also allow us to build a better platform for developers (2): why learn Aura components/Apex when you already know and can use Typescript/React?
An open-source core is only part of it, because different users will need different data models for different business domains. One of the biggest challenges here will be: how can you allow tenants to run arbitrary plugins that (a) execute arbitrary sandboxed server-side code, and (b) can store arbitrary new data models and custom fields within your instance's centralized storage alongside the canonical model types,…
So, as we go through the multi-tenant path, what you say is very relevant and will be challenging.
Regarding custom entities and custom fields, we plan to introduce a flexible data table backed by a meta-data, quite close to what salesforce is doing ; this article is gold about how they built it: https://architect.salesforce.com/fundamentals/platform-multi.... In short, you have a data table (uuid, objectid, tenantid, field1, ..., fiel500) where fields are VARCHARs and you build your own engine on top of that. This comes with a lot of challenges such as performances (indexation), typing (we lose Typescript/GraphQL power obviously as we deal with flexible data modeling)
Regarding plugins that we want users to be able to create and to activate on the marketplace without vetting, here is the way we see it right now:
1) Front-end: serve a dedicated JS depending on what workspace you are on. Rebuild this JS when you activate / update a plugin
2) Back-end: we will need to execute the code in a separate environment. We were thinking about serverless lambdas for the cloud version and keep it local on the main server for self-hosting ; kind of allowing two drivers (lambdas + local) to execute plugin code in the codebase but using lambdas only on the the cloud).
Would love to chat a bit more about it. We will likely open a Github discussion thread in the upcoming weeks about this specific topic) so we can get the feedback from anyone interested into it