How my book about Vertically Integrated Architectures came into being

Paperback: 188 pages
Release: January 2019
Amazon link:

I imagine most books start with lots of jotted down notes and ideas. So far so good, but some of my first notes and drawings for this book are already thirty years old. During my years of study, I learned about real databases and how they magically hide a lot of technical details for the programmer.

With SQL, I saw the beauty of a fully thought through conceptual data model brought to life by a neat and powerful query language. However, I also remember asking myself whether tables are really the best choice to represent data. The relational model was obviously better than anything else out there. However, influenced by other methods I studied, like the Natural Language Information Analysis Method (NIAM, Sjir Nijssen), I began to imagine data more as a network of abstract objects (facts) joined together by relationships. In SQL, you have to specify the actual relationships based on attributes with every query again and again. And since it became apparent that applications were mostly not built using SQL, every query also needs its own glue code in order to fit inside the accompanying 3GL programming language. Why? These early thought experiments eventually became the main premise of my book about Vertically Integrated Architectures.

Why does the user interface not simply understand the underlying data model so that a lot of things can be arranged automatically? Why do we program in two, three, or four languages in order to build a single application? And why do we have to manually pass around strings with pieces of keys and data, like we do with JSON nowadays?

My inspiration to rethink these dilemmas over and over came from frustration, experimentation, study, and lots of discussions within my peer group. I was never a computer scientist, and as a very practical person, I preferred working on concrete projects. But I used every slightly more generic challenge in any project to think and experiment with potential solutions. It always helped me to go beyond simply passing around records between screens—for example, with generic data reporting solutions, code generation when useful, and fancy synchronization solutions. I also started studying scientific papers on related subjects. All this material comes together in this book.

Trying to convince people that two-tier architectures and the ideas behind 4GL/RAD-languages need a second chance, I begin with a thorough analysis of where we currently stand. Although I agree that most contemporary architectural principles were born out of necessity, I will explain how they eventually led to mostly disconnected tiers that we have to glue together again every time. While it is true that this gives us a lot of flexibility, it also forces us to write a lot of code that could be deduced from the system’s data model. And it also results in code duplication across layers and tiers. I argue that at least 70 to 80 percent of what we write does not concern the business logic the application is about.

At the same time, I recognize the problems with 4GL and RAD that made them fail. And although it helps that platforms like OutSystems and Mendix reintroduced the 4GL approach under the name low-code, I still see problems. Code generation cannot optimize for every real-life scenario; gluing lots of existing techniques together sounds compatible, but it is in fact very constraining, and the versioning of external interfaces is still as troublesome today as it was in the 90s.

We need to pursue fundamental new concepts that are general purpose and flexible at the same time rather than just trying to mimic what we currently do manually. I would like to preach going back to the drawing board, getting rid of the anxiety surrounding the creation of a totally new programming language, building an actual compiler, and escaping today’s dogmas.

I’m convinced that the second half of my book introduces, or at least seeds, solutions that will allow us to escape the current dilemmas: with a single unified conceptual data model, we can build what I call implicit services and a persistence-aware programming language to only express pure business logic. I show that what has made two-tier inflexible and not general purpose so far is the lack of support for data model versioning and a more conceptual approach to data modeling.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Up ↑

%d bloggers like this: