Because software plays a central role in an increasingly diverse array of fields, it is increasingly common that those people who know what the software should do are not the same as those who program the software. So for the software to work right, the domain expert's "brain contents" have to be formalized into an executable recipe we call a program. The traditional approach is that the domain expert communicates the requirements to the programmer more or less as prose text: as through conversation, via user stories, in Word documents or Excel files, or in Doors databases. The developer reads this text, tries to understand it and then implement them correctly. The process of formalization into code helps detect some ambiguities, inconsistencies or incompleteness; but ultimately, it is when the domain expert then plays with the finished software — or ideally writes Gherkin-style acceptance tests — that the final validation happens.
We all know that this approach doesn't work so well.
Domain-specific languages use a different approach. They allow the domain expert to specify the behavior of the software directly. The transformation from unstructured thought to executable specification happens in their brains. They directly get feedback about ambiguities, inconsistencies or incompleteness. The executable specifications, or models, created this way are then transformed into "real" source code by machinery developed by the software engineers.
Does this really work? It does under certain conditions. In particular, the language must be suitable for use by non-programmers. The primitives in the language should not be generic to "computation" — such as variables, conditions, loops, functions, monads or classes — but instead be specific to the domain, and therefore meaningful to the user: decision table, treatment step, tax rule or satellite telemetry message definition. The syntax should build on existing notations and conventions used in the domain — tables, symbols, diagrams and text — and not just consist of magenta-colored keywords and curly braces. DSLs are also usually less flexible in the sense that users can only compose new abstractions in very limited ways; while this would be a problem for general-purpose languages, it is a plus for DSLs because it ensures that programs are less complicated and therefore easier for tools, which leads to better error messages and IDE support.
DSLs are not a new idea, they have been around forever. However, they have been mostly used by programmers to simplify their tasks — the languages have been specific to technical domains. DSL optimized for non-computational domains have become practical only in the last 10 to 15 years mainly because of better tools for developing these languages. "Better" primarily means faster. Because a DSL by definition has a smaller user base, their development must be less effort compared to that for a general-purpose language in order for the business case to close. Language workbenches, tools optimized for language development, make this realistic. One particularly important feature is the ability to reuse (parts of) language defintions to avoid the need to redevelop your plus operator again and again. A second enabler for non-programmer DSLs is the ability to mix various notational styles to make sure the language is intuitive to use for the users in a wide range of domains. Again, industry-strength tools that support this have been around for maybe 10 years.
Do you actually need a DSL?
Build your own language, why and how?
A video recorded at GOTO 2018 in Amsterdam
The way I build DSLs is a mix between what is typically referred to as modeling and what people often call programming. Here is a paper that discusses the difference and why it doesn't really matter.
Rumors are that generated code is slow (and also ugly). Here is a case study where we developed a smart meter with DSLs despite stringent realtime requirements. And guess what, it works!
We have developed a functional language called KernelF which has since served us well as the "functional core" of several business DSLs. Read about its design, evolution and use.
Can a DSL and a custom generator be used in a safety-critical environment? Yes! Here is an example from healthcare.
That is an important question! After all, DSLs are also "just software" and testing, even test-driven development, is important to get the quality right. Learn from our experience here!.
There are lots of synergies between the two fields. Check out our booklet to learn the basics of model checking, SMT solving and dataflow analysis.