Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
1–10 of 29 posts
Re: Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
#2I'm thoroughly convinced that software engineering efficiency can be increased by at least one order of magnitude by letting domain experts and product owners directly modify the product through a well defined DSL.
Re: Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
#3I'm thoroughly convinced that software engineering efficiency can be increased by at least one order of magnitude by letting domain experts and product owners directly modify the product through a well defined DSL.
To create that ideal dsl one has to both know the domain well enough and have the technical skills which is a rare combination so i'm rather sceptical of this claim.
Besides any dsl can only help with well understood repeatable problems, for problems that aren't covered by a dsl, software engineers are still required.
The fundamental role of software engineers is to build easy to use and insightful interfaces to understand complex data generated/collected from the real world. To do that one needs to have the skill to organise information and control complexity by data hiding not exactly the skills product owners and domain experts are known for.
Re: Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
#4Domain specific languages just didn't settle at the level we expected. We thought they would be for really narrow domains, like a language for each kind of business logic. But instead the domains are more at the architecture level and some very broad purpose (i.e. HTML, SQL, etc).
At my company, we use Ruby for web backend and system administration. We use Typescript for web frontend. We use C# for process automation. We use Rust and C++ for real time performance (we've got an in house 3D engine), and a little bit of high performance processing. We use Python for data processing and machine learning.
That's six "general purpose" languages, each applied to a specific domain, on six software developers, in three teams of two. Each happy with their choice of languages, effectively solving the business needs.
I think at some point in the past this situation was described as a nightmare. But I think at that point people thought developers were something you just opened a tin can of, and then applied to whatever problem you had. Nowadays we hire for a specific purpose (or at least I do). When I hire web dev experts, I expect them to be fluent in the industry standard web programming languages and platforms of our choice (i.e. for us it'd be Ruby and Typescript). When I hire 3D experts I expect them to be fluent in C++, and proficient in Rust (it's the future). And obviously it's Python for the data science types.
That's a very clear world to me, and to be extra clear, I'm not saying any of the languages I mentioned are the best choice. As long as we operate within the lines the industries have drawn, you can draw upon the best libraries and ecosystems for your particular problem. I would never approve a 3D engine to be developed in Ruby, or a data science pipeline in Javascript. I'd rather have a Ruby developer learn some Python so they'd be able to work on the data science pipeline (disregarding their complaints about Python's inelegance) than have them trying to kludge together subpar Ruby libraries.
Re: Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
#5I'm thoroughly convinced that software engineering efficiency can be increased by at least one order of magnitude by letting domain experts and product owners directly modify the product through a well defined DSL.
We built the first open-source feature store for ML, https://github.com/logicalclocks/hopsworks , when every existing proprietary feature store (Uber Michelangelo and Bighead at AirBnb) were shouting about how their DSL for feature engineering was the future.
Fast-forward 2 years and it is clear that Data Scientists want to work with Python, not with a DSL. We based our Feature Store on a Dataframe API for Python/PySpark. The DSL can never evolve at the same rate as libraries in a general-purpose programming language. So, your DSL is great for show-casing a Feature Store, but when you need to compute embeddings or train a GAN or done any type of feature engineering that is not a simple time-window aggregation, you pull out Python (or Scala/Java). I am old enough to have seen many DSLs in different domains (GUIs, aspect-oriented programming, feature engineering) have their day in the sun only to be replaced by general-purpose programming languages due to their unmatched utility.
Re: Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
#6It is a bit telling those were the best reasons they could think of for lack of adoption. Just "ignorance".
Not we thought that maybe specifying a DSL rather than a library would leave our users and clients in a state of having to banadage over the constraints of a DSL that doesn't handle future use cases, like say dealing with control flow in a half-assed YAML based language, I'm looking at you Ansible.
I don't want MAKE, or whatever DSL, I want to be able to drop into a real programming language when necessary. So libraries. Not frameworks, Not DSLs, libraries.
And thankfully, it seems the world agrees with me.
Re: Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
#7I'm thoroughly convinced that software engineering efficiency can be increased by at least one order of magnitude by letting domain experts and product owners directly modify the product through a well defined DSL.
We built the first open-source feature store for ML, https://github.com/logicalclocks/hopsworks , when every existing proprietary feature store (Uber Michelangelo and Bighead at AirBnb) were shouting about how their DSL for feature engineering was the future. Fast-forward 2 years and it is clear that Data Scientists want to work with Python, not with a DSL. We based our Feature Store on a Dataframe API for Python/PyS…
The article (controversely, maybe) classifies libraries for general-purpose programming languages as internal DSLs. One could argue that Data Scientists working with libraries in Pythin are already using a DSL, just with an escape hatch into the general-purpose world.
I don't think proper vertical DSLs should be made marketed towards people who are comfortable working in a general-purpose language. I see them as a way to help non-technical domain experts work on code instead of specification. Limiting the possibilities of what one can write, like with MPS' projectional editor, is a feature here and not a bug.
Re: Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
#8I'm thoroughly convinced that software engineering efficiency can be increased by at least one order of magnitude by letting domain experts and product owners directly modify the product through a well defined DSL.
I guess you haven't had the "pleasure" of cleaning up the mess made by domain experts and product owners using those well-defined DSLs.
Re: Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
#9It seems to me that the main reason DSLs aren't more widely adopted is that any DSL will be unsupported by other tools developers consider more important. Your IDE won't have syntax highlighting and auto completion for it, and a lot of developers nowadays seem crippled without those. Linters and semantic checkers won't understand it. Nor will debuggers. There will be no mocking libraries or makefile rules for it. The list goes on and on.
These are problems that every new non-domain-specific language has to address. It's quite a lot, and most of it is pretty tedious compared to designing the language itself. So even those who try to create DSLs often skip most of the "extra" bits, and other developers learn to hate DSLs.
Re: Reflections on the Lack of Adoption of Domain Specific Languages [pdf]
#10It is a bit telling those were the best reasons they could think of for lack of adoption. Just "ignorance". Not we thought that maybe specifying a DSL rather than a library would leave our users and clients in a state of having to banadage over the constraints of a DSL that doesn't handle future use cases, like say dealing with control flow in a half-assed YAML based language, I'm looking at you Ansible. I don't want…
> I don't want MAKE, or whatever DSL, I want to be able to drop into a real programming language when necessary. So libraries.
That's one opinion, sure.
However, DSLs are always in a far better position to solve domain-specific problems because, unlike generic programming libraries, they not only reflect domain-specific knowledge and best practices and represent standardized solutions for recurring problems.
Consequently, Makefiles and other DAG-related DSLs are omnipresent and dominate domains such as build systems, and generic library-based proposals always failed to gain any form of traction.
And it's not just Makefiles or build systems. There's also markup languages, infrastructure as code, CICD/processing pipelines, configuration, etc etc etc.