As a longtime DMN practitioner and from the beginning one of its biggest boosters, I am what you call a true believer. But while DMN continues to gain traction, even I would have to admit it has so far underperformed in the marketplace. The industry analysts say, “OK, we’re aware of it… But what’s the killer app?” It may not be what you think.

In the Beginning

Let’s go back to the beginning. In 2016, DMN was brand new. I had been posting about it, and I was invited to introduce it to the Decision Camp audience of decision management vendors and practitioners that year. As standards go, DMN was unusual: an executable modeling language designed for use by subject matter experts in a way that standards like BPMN were not. Executable logic would be defined graphically, using a set of standard tabular formats called boxed expressions, with a new business-friendly expression language called FEEL used in the table cells: Executable models without programming.

At that time, DMN was just a spec. There were no runtime implementations. But I also noticed that among tool vendors on the DMN 1.1 Task Force, which I had recently joined, there was no great urgency to implement it, and in my talk I complained about that. Of course, it was the things that made FEEL and boxed expressions business friendly that also made them difficult to implement: spaces in variable names, for example, and tables nested, potentially without limit, inside other tables.

But at that meeting, Mark Proctor of Red Hat told me, “Dont worry, my guy will have a DMN runtime in 6 weeks.” It actually took his guy, Edson Tirelli, more than 6 months, but it worked very well, and Red Hat made the FEEL parser and DMN runtime open source. So by 2017 we had an open source runtime, and there was little excuse for tool vendors not to fully implement DMN.

A few did, but most Decision Management vendors continued to resist FEEL and boxed expressions. In fact, they questioned the original premise that business users really wanted to go beyond defining requirements handed off to programmers. Or, if they did, that they had any interest in basing that on the DMN standard even though those had been central to the original DMN RFP.One vendor that did believe in the original premise, as I did, was Trisotech, and I soon moved my DMN training to their platform, which was aimed squarely at non-programmers.

Lets fast forward to today, 6 years later. The DMN standard continues to get better, every year a new version. But still, few tool vendors have implemented it beyond DRDs and decision tables. Interest in my DMN training has increased, but it is only now catching up to my BPMN training, which is focused on descriptive, that is non-executable, models. Its a little discouraging.Yet despite all that, I am more hopeful today than ever about the promise of DMNs key features FEEL and boxed expressions. And thats because they are the keystones of what I believe is a far larger opportunity.

What You Draw Is What You Execute

Back in 2010, the process modeling standard BPMN 2.0 promised business users, “What You Draw Is What You Execute.” I used to say that myself.

But not really. What makes a process model executable are the services implementing the tasks, and the process data and expressions used to orchestrate them. But for those things, BPMN has always required programming. That standard provides nothing like FEEL and boxed expressions for non-programmers. While the activity flow semantics can be defined by business users in diagrams, making those diagrams executable has always required Java programming. So diagrams created by business and handed off to programmers for implementation has been standard practice in Process Automation for a dozen years now.

A few years later, DMN set out to be different from that. If you are unfamiliar with OMG standards, they start with an RFP that lays out the objectives of the standard. And DMN was distinctly different from BPMN in that it wanted to enable business users to do more than create requirements handed off to programmers. To that end, the DMN standard itself defines a graphical Low-Code language for decision logic implementation: a set of standard tabular formats called boxed expressions and a friendly expression language for the table cells called FEEL. Although DMN allows tools to assert so-called Level 1 conformance without supporting FEEL and boxed expressions, FEEL and boxed expressions comprise the bulk of the DMN spec: the syntax, semantics, operational behavior and graphical formats.

Unfortunately, most decision management vendors especially those who formerly called themselves business rule engine vendors failed to adopt FEEL and boxed expressions, preferring the traditional arrangement in which business creates requirements handed off to IT for implementation in some proprietary language, and they continue to do so today although in so doing, they ignore the original promise of DMN.

Its important to understand that decision logic is more than decision tables. For example, it must refine raw input data into the variables that are used in the decision tables. FEEL and boxed expressions not only do that, but represent a complete Low-Code language suitable for any kind of business logic, not just that related to operational decisions. Even though FEEL and boxed expressions are defined within the DMN spec, their utility extends far beyond the boundaries of traditional decision management.

For example, over the past two years, the Trisotech platform has quietly been adding Automation capabilities to its BPMN models in a different way, a Low-Code way, leveraging FEEL and boxed expressions borrowed from DMN. By now that Business Automation platform is quite complete. So this goes beyond business users creating decision services. This is enabling them to create full-fledged Business Automation services themselves, graphically, so that What You Draw actually IS What You Execute.

Low-Code Business Automation

I use the term Business Automation to mean the combination of business logic automation and process automation, and DMN combined with BPMN offers a Low-Code approach to it based on standards.. If Subject Matter Experts learn to create decision models using DMN with FEEL and boxed expressions, its just a small additional step to do Low-Code Business Automation themselves: model, test, and cloud deploy as a REST service no programming required.

Low-Code means model-based software development, accessible to technically inclined business users, as well as to developers. The fact is, interest in Low-Code Business Automation is exploding; Gartner says 65% of all solution development will be Low-Code by 2024.Thats an addressable market larger by orders of magnitude than Decision Management. The key benefits: time to value model-driven development is simply faster than code – and avoiding the IT resource bottleneck, since subject matter experts can do much of it themselves.

That is why I see these DMN features FEEL and boxed expressions, which have frankly languished in the world of Decision Management – as the key to something new and exciting today, an opportunity that is much larger than Decision Management. Actually Low-Code Business Automation is DMNs killer app.

Why is this opportunity happening now? The fact is, over the past six years, the software world has changed completely:

Example:
Vacation Request Process

Let’s see how Low-Code Business Automation with DMN and BPMN works, and then go back to key questions that have dogged Decision Management vendors from the start: Do business users really want to do this? And does basing the tools on standards make a difference?

Here is an example of a Vacation Request process as it would be modeled using my non-executable BPMN methodology called Method and Style. Not a core business process but illustrative. Its very simple. A Vacation Request is processed by decision logic that either automatically approves it, refuses it, or refers it to the manager. If ultimately approved, the requested days are deducted from the employees accrued vacation time.

And here is the executable version of it using Low-Code Business Automation: The BPMN is similar but the activities now are more fine-grained, the icons in the top left corner indicating specific capabilities of each task type. But the most obvious difference is the model now includes process data and data flow.

• The dog-eared page icons, called data objects, represent the process variables. In Low-Code Business Automation, FEEL defines the data language the datatypes and expressions. In the runtime environment, FEEL is converted to json automatically by the platform. FEEL and json types are actually quite similar.
• The dotted arrows, called data associations, represent mappings between the process variables, which are defined by the process modeler, and task variables, most of which are defined by each invoked service. In Low-Code Business Automation, those mappings are defined using FEEL and boxed expressions.
• The activities with the spreadsheet icon, called decision tasks, invoke DMN decision services. As I mentioned, DMN is in reality a Low-Code language that can be used for ALL modeler-defined business logic, not just what we normally call decisions. Here we have two decision tasks: The first one validates the data input Vacation Request to avoid runtime errors; the second one is the automated approval logic.

Decision Tasks

A BPMN decision task is bound to a DMN decision service.

Here you see the data input validation decision. DMN is actually an excellent language for data validation. A Collect decision table, hit policy C, produces a list of error messages. Data validation logic typically uses some of the lesser known FEEL functions and operators: generalized unary tests and the instance of operator, as you see here, also the get entries function, which lets you test all components of a structure at once, and match, for pattern matching with regular expressions.

Just to repeat, DMN is not only about decision tables. Its a Low-Code language for any business logic. For example, the Vacation Request specifies the employees requested start and return date, but the approval logic requires knowing the count of requested days. FEEL has excellent support for calendar arithmetic, but finding the count of requested days is a little complicated. We cant simply subtract the dates; we need to exclude weekends and holidays as well.

This boxed expression is called a context. It lets you break up a complex value expression into pieces using local variables something most other Low-Code languages cannot do. For example, here there are 5 context entries, each defining a local variable and its value expression, and the final result expression references those local variables. Note that the fourth context entry uses a decision table, an example of a table nested in another table. Context boxed expressions let you make complex logic business friendly without cluttering up the DRD with dozens of tiny decisions.

In the DMN model, a decision service defines the parameters called by the BPMN decision task. The decision service outputs are selected decision nodes in the DRD, and the tool calculates the required service inputs. Here the service Validate Vacation Request has input parameters Vacation Request and Original Employee Record, and output parameters Validation Errors and Employee Record.

In BPMN the Decision task Validate Vacation Request is bound to that decision service. The task inputs are by definition the decision service inputs, and the task outputs are the service outputs.In the process model, the dotted arrows, called data associations, represent mappings between the process variables, here called Vacation Request and Employee Record, and the task inputs and outputs. So from the diagram you see there are input mappings from Vacation Request and Employee Record and an output mapping to Validation Errors.

Since all data is represented as FEEL in the modeling environment, the mappings use boxed expressions and FEEL, similar to a boxed invocation in DMN. Here the mappings are essentially identity mappings, but any FEEL expression can be used.

Service Tasks

An activity with the gear icon, called a service task, calls an external REST service. Simply bind it to a REST operation in the models Operation Library and map process variables to the service inputs and outputs.

EmployeeVacationData is a MySQL database table on my website. It holds the employees accrued vacation time.A third party service, Skyvia Connect, introspects the database and generates a REST endpoint for it and a metadata file used to configure a Trisotech connector, exposing 5 database operations on the table using OData. We will use the Find, or query operation, to retrieve the record for the requesting employee.

We bind the service task Get Employee Data to the Find, or query operation of this endpoint in the Operation Library.Then we model mappings between process variables and the service input and output parameters using FEEL and boxed expressions. Here the data mapping for the input parameter $filter uses a FEEL expression to create a query string in the OData language.

Once were done, the whole Vacation Request process is itself deployed as a REST service in the Trisotech cloud. This Business Automation service can be called by any REST client app with the appropriate credentials.

Hopefully this gives you the flavor of how subject matter experts can create Business Automation services using FEEL and boxed expressions without programming!

Where Can We Use This?

What kind of Business Automation services can make use of this? Currently the Trisotech platform is used most heavily in integration-centric processes in healthcare and financial services. Digital mortgage underwriting is a great example, as there are many complex rules. You might say thats a straight decision management problem, but its really a process. You need to first get the data from various sources, validate the data, transform it to testable values, apply some decision logic, and finally format and store the output. Low-Code Business Automation can do all that in a single composite service.

I am currently working with a large accounting firm to create general ledger entries triggered by business events related to financial portfolio assets and trades. Its high-volume, involves many database tables, a lot of computation perfect for DMN and BPMN.With Low-Code, solutions like these are much faster to build, test, and deploy than with Java, and easier to maintain.

So who can create apps like these? I admit its not every business user or subject matter expert. Most of my BPMN Method and Style students probably could not do it very well. But anyone who can create executable decision models using FEEL and boxed expressions certainly can. You need to have some basic facility with data and expressions, and the patience to debug when things dont work right the first time. Thats just a subset of business users, but in most companies, subject matter experts who can do that still greatly outnumber programming staff. Microsoft used to call them citizen developers; now they are calling them software makers, a term I like better.

To turn subject matter experts into makers, I had to completely revise my DMN training. Originally I had a course DMN Basics that focused on DRDs and decision tables, since thats all most DMN tools supported and Its something every business user can understand. The problem is you cannot do anything useful with only that except to provide requirements handed off to others. In the revised training I focus on FEEL and boxed expressions. Those are the critical elements. And yes, subject matter experts who are not programmers can learn to understand and use them well. You just have to show them how. They feel empowered, because they ARE empowered. Once you fully understand DMN, Low-Code Business Automation is mostly a matter of learning how to find and orchestrate REST APIs.

DMN vs Proprietary Low-Code Languages

OK, youre saying there are already a large number of Low-Code Business Automation tools available. Thats true. Basically all of them use their own proprietary scripting for the Low-Code piece, just as many Decision Management vendors use their own proprietary rule languages, which they also say are business-friendly.

But FEEL and boxed expressions are not only non-proprietary, they are actually more business friendly and often more powerful. For example, FEEL and boxed expressions make DMN more powerful and more business-friendly than Power FX, Microsofts Low-Code expression language used in Excel formulas and Microsoft PowerApps. Without boxed expressions like contexts, Power FX must pack complex logic into a single literal expression. And Power FXs lack of infix operators makes these expressions deeply nested functions that are difficult to understand.

For comparison, here is an example published by Microsoft marketing showing how Power FX can extract the last word in a string. They were very proud of this So, lets see, Is that 4 or 5 right parens at the end?Below it is the equivalent in FEEL: The split operator tokenizes the input string based on the space character and [-1] means take the last item. Now you tell me, which is more powerful AND more business friendly?

So why are DMN users not demanding their tool vendors support FEEL and boxed expressions?

The Larger Opportunity

That lack of support from incumbent vendors is why I have come to doubt that Decision Management is the best opportunity for DMN. A better opportunity is Low-Code Business Automation.

And this takes us all the way back to the original question:Do subject matter experts really want to create executable solutions themselves? Not all, obviously, but an increasing number of them do, and their employers today desperately need to acquire low-code tools and expand their pool of software makers. And even for professional developers, Low-Code is faster, easier to maintain, and more transparent to the business.

That is why I am becoming increasingly optimistic about engaging the Business Automation community with Low-Code Business Automation using BPMN and DMN. Its a far larger addressable market than Decision Management, its growing, and its perfectly matched to todays architecture based on the cloud and REST APIs.

But we know there are other Low-Code alternatives. So do they want to do it using standards, i.e. DMN in combination with BPMN? That is DMNs most obvious differentiator right now vs other Low-Code languages. Here are two reasons why they would.

• First, BPMN is widely used by business already for descriptive models. Whats been missing in the BPMN world is a way for non-programmers to automate their processes. Now integration with DMN offers them a way to do just that by creating executable business logic without programming, and orchestrating the many REST APIs that are now available.
• Second, BPMN users already recognize the value of using a vendor-independent standard. DMN, like BPMN, is an OMG standard, and the two standards were designed to work together. DMN has a published spec, governed through a vendor-neutral process, continually maintained and updated. It is supported by multiple tools. You are not locked in to a single vendor. There is a model interchange format, so business users and developers can use different tools. There is an open source runtime for it. There are multiple sources of information and training on how to use it. There is even a TCK that validates correct execution semantics.

So ideally,
what should happen now?

• Id like to see more DMN tool vendors include support for FEEL and boxed expressions, although Im not too confident that will happen.
• In addition, Id like to see new Low-Code Business Automation platforms emerge, likely focused on specific industry segments, that adopt FEEL and boxed expressions. Im seeing service providers in healthcare and financial services begin to explore moving to offer software solutions built with this Low-Code approach. An open source FEEL runtime is available, and this is now an IBM product, so maybe that will help.
• Existing DMN tools like Trisotech continue to add features that make it even more powerful and easier to implement, such as a graphical debugger and support for importing powerful function libraries.
• And of course, endorsements by industry groups are in principle helpful. We have those from BPM+ Health in the healthcare space and MISMO in the mortgage space, and hopefully those will lead to increased adoption in those industries. So well see.

What started as an effort to enable subject matter experts to create decision logic themselves, while so far resisted within the digital decisioning community, could ultimately find a better path forward in Low-Code Business Automation.

For anyone interested in the tools and techniques used in this approach, there is a lot of information on methodandstyle.com. I offer online training in DMN, focused on FEEL and boxed expressions, and in Low-Code Business Automation, and there is a bundled offering at a discount. Each course includes 60-day use of the Trisotech platform and post-class certification.

Cloud datastores are a new feature of the Trisotech Automation platform. They are most useful in BPMN, but they can serve also as input data in DMN. Datastores are a standard BPMN element representing persisted storage, accessed by a process task via data association but available for external access as well.

In Trisotech’s implementation, each datastore is a single relational table. It acts like a database but requires none of the extra work involved in using something like OData. For example, to incorporate a relational database table in your model with OData, you need to host the database yourself, subscribe to an OData wrapper service like Skyvia Connect, and use a Service task for the database operation. Datastores are much more convenient: They are built into the Trisotech platform, so they require no external components and they are directly accessible like any process variable. If your table is a few thousand records or fewer, datastores could make your Low-Code Business Automation project a lot simpler. In this post we’ll see how they work.

In the process diagram, a datastore is represented by the can shape, unlike the dogeared page shape used to represent data objects, data inputs, and data outputs. Like data objects, cloud datastores are process variables based on a FEEL datatype, which is a collection of the table row type. But a datastore differs from a data instance that accesses it is complete. Second, a datastore does not belong to a single process. It belongs to the entire workspace, with access granted to its creator and selected other workspace members. That means it can be used to store data shared between a process and an external entity or between multiple processes.

In Trisotech, datastore content is typically created and maintained with the help of Excel on the desktop. Importing an Excel file in a management utility creates and populates the table. For example, here we have an Excel table of the available vacation days for each employee. In the management utility, importing this Excel file populates the datastore and adds it to the list of the user’s defined datastores. You can also modify datastore content via Excel using the utility. Authorized external entities can also do that via an API, enabling interaction with the process via shared data, a familiar pattern from Method and Style.

Interest in Business Automation is being accelerated by the thousands of public REST APIs available from countless service providers.

Most are available on a Freemium basis – free low-volume use for development, with a modest monthly fee for production use. The Trisotech Low-Code Business Automation platform lets you incorporate these services in your BPMN models without programming. You just need to configure a connection to them using the model’s Operation Library. With many public APIs, that configuration requires simply importing an OpenAPI file from the service provider. OpenAPI – sometimes called Swagger – is a file format standard for REST APIs. The problem is that most public APIs don’t provide one. Instead they provide documentation that you can use to configure the Operation Library entry yourself. If you’re not a developer, that sounds daunting, but it’s really not that hard. This post will show you how to do it.

A REST API is organized as an interface containing a set of operations. The interface specifies a base address, or server URL, and some means of authorization, such as an API key. Each operation performs a specific function, and is addressed by a path appended to the base address or endpoint. To invoke the operation, a client app sends an http message that contains the required information to a URL concatenating the base address and path. That information includes:

• The http method: POST, GET, PUT, PATCH, or DELETE, indicating whether the operation is reading, writing, modifying, or deleting data.
• The authorization credential
• Input parameters for the operation, required and optional, typically specified as json

Upon receipt of the message, the service is executed and returns some response message, which includes:

• A standard http response code. A code of 200 indicates success, 404 means the base URL was not found, 401 not authorized, 500 either a bad request or internal server error, etc.
• Output values for the operation, separately for each response code, again specified as json

Compared to the earlier generation of SOAP-based web services, REST APIs are simple and standardized. This is their great attraction!

In my Business Automation training, we make use of a real-time stock quote service from Rapidapi.com, a popular aggregator of public APIs. The service we use is called YH Finance, a clone of the old Yahoo Finance API. The documentation page looks like this:

The interface is huge, with operations collected in groups. The one we want is in the group market, with the path /market/v2/get-quotes. Selecting that from the panel on the left, we see the operation documentation in the center panel, which specifies the parameters of the operation, and where in the message they are placed: in the message header, message body, or query portion of the URL. Here two parameters are required in the http message header: X-RapidAPI-Host is the server URL, yh-finance.p.rapidapi.com. X-RapidAPI-Key is the requester’s personal authorization credential. Two more parameters are required in the query: region, enumerated text, and symbols, a comma-delimited string containing stock tickers.

This documentation is all we need to create the input side of the Operation Library entry. Manually add an interface YH Finance and an operation Get Quotes. In the interface properties enter the Server URL and the authorization type. In the operation properties enter the method, path, and under Inputs the required parameters except for the API key, which is assigned to an individual and entered separately when the process containing the service invocation is compiled and published. Clicking the pencil for each input lets you specify its datatype, a FEEL type you must create from the documentation. It’s actually quite easy!

Configuring the operation output is slightly trickier. Notice in the API documentation the button Test Endpoint in the center panel. Clicking that executes the operation with the default parameters shown, in this case a string listing three stocks.

The documentation page then shows the resulting output, as you see here: an outer element quoteResponse with two components, an array of result – one per stock – and error. Expanding result, we see it contains 97 components – wow, that’s quite a lot – of which we only need three: quoteType, bid, and ask. But we do not need to construct a FEEL type with all 97 components. As long as we respect the basic structure, we can omit the 94 result components we don’t need! So we can create FEEL type tTradeQuote, a structure with enclosing element quoteResponse containing a collection of result plus error, and the type of result contains just the three components we need. When the service is invoked, the json response message is mapped automatically to the FEEL tTradeQuote structure.

To incorporate this operation in your Business Automation model, simply click the gear icon on a Service task and bind the task to the Operation Library entry. The service inputs – symbols, region, and x-rapidapi-host – become the task inputs, and the service output quotes becomes the task output. Then, using the Low-Code data mapping explained in a previous post, you just need to map data objects in the process to the task inputs and output. When you Cloud Publish the process, you will need to enter your API key. And that’s it!

So just because your API provider doesn’t offer an OpenAPI file, don’t think it’s inaccessible to Low-Code Business Automation. It’s actually quite easy.

In BPMN Method and Style, which deals with non-executable models, we don’t worry about process data. It’s left out of the models entirely. We just pretend that any data produced or received by process tasks is available downstream. In Business Automation, i.e., executable BPMN, that’s not the case.

Process data is not pervasive, available automatically wherever it’s needed. Instead it “flows”, point to point, between process variables shown in the diagram and process tasks. In fact, data flow is what makes the process executable. With most BPMN tools, defining the data flow requires programming, but Trisotech borrows boxed expressions and FEEL from DMN to make data flow – and by extension, executable process design – Low-Code. This post explains how that works.

First, we need to distinguish process variables from task variables. Process variables are shown in the BPMN diagram, as data inputs, data outputs, and regular data objects. A data input signifies data received by the process from outside. When we deploy an executable process as a cloud service, a data input is an input parameter of the API call that triggers the process. Similarly, a data output is an output parameter of the API call, the response. A data object is a process variable created and populated within a running process.

Task variables are not shown in the process diagram. They are properties of the tasks, and the BPMN spec does not specify how to display them to modelers. What is shown in the diagram are data associations, dotted arrows linking process variables to and from process tasks. A data input association from a process variable to a task means a mapping exists between that variable and a task input. Similarly, a data output association from a task to a process variable means a mapping exists between a task output and the process variable.The internal logic of each task, say a decision service or external cloud service, references the task variables. It makes no reference to process variables.

On the Trisotech platform, the task inputs and outputs are shown in the Data Mapping boxed expressions in the task configuration. Task variable definitions depend on the task type. For Service tasks, Decision tasks, and Call Activities, they are defined by the called element, not by the calling task. For Script and User tasks, they are defined within the process model, in fact within the Data Mapping configuration.

A Service task calls a REST service operation. The Operation Library in the BPMN tool provides a catalog of service operations available to the Service tasks in the process. Operation Library entries are typically created by the tool automatically by importing an OpenAPI or OData file from the service provider, but they can also be created manually from the API documentation. Each entry in the Operation Library specifies, among other details, the service input and output parameters. When you bind a Service task to an entry in the Operation Library, the Service task inputs are the service’s input parameters and the Service task outputs are the service’s output parameters.

A Decision task calls a DMN decision service on the Trisotech platform. In the DMN model, you create the decision service, specifying the output decisions, which become the service outputs, and the service inputs, which could be input data or supporting decisions. When you bind a Decision task to the decision service, the service inputs become the Decision task inputs and the service outputs become the Decision task outputs.

A Call Activity calls an executable process on the Trisotech platform. When you bind the Call Activity to the called process, the data inputs of the called process become the task inputs of the Call Activity, and the data outputs of the called process become the task outputs of the Call Activity.

With Service tasks, Decision tasks, and Call Activities, the task inputs and outputs are given. They cannot be changed in the calling process. It is not necessary for process variables in the calling process to have the same name or data type as the task variables they connect to. All that is required is that a mapping can be defined between them.

Let’s look at some examples. Below we see the DRD of a decision model for automated approval of an employee vacation request, and below that the decision service definition, Approve Vacation Request.

In BPMN, the Decision task Approve Vacation Request is bound to this decision service.

Here you see the process variables Vacation Request (data input), Employee Record (data object), and Requested Days (data object). Vacation Request and Employee Record have data input associations to the Decision task. There is a data output association to Requested Days and two more to variables not seen in this view.

The Data Mapping dialog of Approve Vacation Request shows the mapping between the process variables and the task inputs and outputs. Remember, with Decision tasks, the task inputs and outputs are determined already, as the input and output parameters of the decision service. In the Input Mapping below, the task inputs are shown in the Mapping section, labeled Inner data inputs. The process variables with data input associations are listed at the top, under Context. The Mapping expression is a FEEL expression of those process variables. In this case, it is very simple, the identity mapping, but that is not always the case.

In the Output Mapping below, the task outputs are shown in the Context section, and the process variables in the first column of the Mapping section. The Output Mapping is also quite simple, except note that the task output Vacation Approval is mapped to two different process variables, Approval and Approval out. That is because a process data output cannot be the source of an incoming data association to another task, so sometimes you have to duplicate the process variable.

Not all data mappings are so simple. Let’s look at the Service task Insert Trade in the process model Execute Trade, below.

Insert Trade is bound to the OData operation Create trades, which inserts a database record. Its Operation Library entry specifies the task input parameter trade, type NS.trade, shown below.

Actually, because the ID value is generated by the database, we need to exclude it from the Input Mapping. Also, the process variable trade uses a FEEL date and time value for Timestamp instead of the Unix timestamp used in NS.trade. We can accommodate these details by using a context in the Input Mapping instead of a simple literal expression. It looks like this:

The context allows each component of the task input trade to be mapped individually from the process variables trade and Quote (which provides the price at the time of trade execution). As we saw with the Decision task, the Context section lists the available process variables, and the first column of the Mapping section lists the task inputs. Note also that the Price mapping uses the conditional boxed expression for if..then..else, new in DMN 1.4, designed to make complex literal expressions more business-friendly.

Finally, let’s look at the Script task in this process, Validate Trade Value. Script and User tasks differ from the other task types in that the modeler actually defines the task inputs and outputs in the Data Mapping. On the Trisotech platform, a Script task has a single task output with a value determined by a single FEEL literal expression. As you can see in the process diagram, the process variables with incoming data associations to Validate Trade Value are trade, Quote, and portfolio, and the task output is passed to the variable Error list. This task checks to see if the value of a Buy trade exceeds the portfolio Cash balance or the number of shares in a Sell trade exceeds the portfolio share holdings.

In the Script task Input Mapping, the modeler defines the names and types of the task inputs in the first column of the Mapping section. The mapping to those task inputs here are more complex literal expressions. The script literal expression references those task inputs, not the process variables. User tasks work the same way, except they are not limited to a single task output.

Hopefully you see now how data flow turns a non-executable process into an executable one. The triggering API call populates one or more process data inputs, from which data is passed via Input Mapping to task inputs. The resulting task outputs are mapped to other process variables (data objects), and so forth, until the API response is returned from the process data output. On the Trisotech platform, none of this requires programming. It’s just a matter of drawing the data associations and filling in the Data Mapping tables using FEEL.

Debugging executable processes can be a challenge because, unlike DMN models, you cannot test them in the Modeler. You need to compile and deploy them first, and problems are often reported as runtime errors.

Until fairly recently, to zero in on the problem you needed to isolate it in a small fragment of the process by saving various fragments as test processes, compiling and deploying them, and running them with reconstructed input values. But earlier this year, Trisotech made it a lot easier with a new feature called Attended tasks. Attended tasks are a Trisotech extension, not part of the BPMN spec, but very useful in practice. This post shows how they work.

An automated task, such as a Decision task or Service task, that is marked as Attended allows its input and/or output data to be inspected and possibly modified by a user, either before or after the task runs. Attended tasks originally were introduced not for debugging but to support customers in Healthcare. While highly interested in technology and automation, Healthcare providers often want to confirm the values provided as inputs to automated tasks in order to correct them for the latest values observed, as some of the data fetched may come from an earlier encounter or may need to be adjusted based on the latest live observations. On the output side, providers want to be able to overwrite an automated decision for various reasons. Based on their own experience, they may disagree with the automated results, or may believe that certain details about the case were not properly considered in the automated result. Trisotech was seeking a way to ensure that a clinical pathway could remain the same for review, whether or not pre and/or post validations were required. By using Attended tasks, the same clinical model can introduce or remove data validation without having to modify the model for all possibilities of the Healthcare context.

My own experience with Attended tasks is not in Healthcare is focused more generally on the debugging problem. Marking an automated task as Attended acts as a “breakpoint” in the automation where the user – typically the process designer in this case, not the end user – can inspect process data at design time to better understand unexpected results. To illustrate, let’s look at a variant of the Vacation Request process in the Trisotech EU-Rent repository.

Employee Vacation Days is a table from the HR system, listing each employee with their accrued vacation days. Data input Vacation Request specifies the requesting employee’s name, id, and proposed vacation start and return dates. Script task Fetch Vacation Information extracts the employee’s record from Employee Vacation Days and passes it to the Decision task Approve Vacation Request, which could either approve the request, refuse it, or refer it for human decision. That’s pretty simple. Nothing much could go wrong here, right? But when we test it from the Service Library, we see a problem. The Decision task Approve Vacation Request failed with a runtime error.

We can mark that Decision task as Attended to take a look at what is going on. When we do that, we see this dialog:

This dialog lets us specify where we want to put the breakpoint: before or after the input data mapping, and/or before or after the output data mapping. Since this task is failing, we want to inspect the data before the input data mapping, which is the output of our Script task. If we want, we can enter the name of a process user as a Resource, who will receive the data in an email and can return corrected values. You would use that when the Attended task is a permanent feature of the process, as in the Healthcare context. But when using Attended tasks simply for debugging, we can leave this Resources field blank, as the user will see the data in the Service Library test interface. So our configured Attended task dialog looks like this:

Now in the process diagram, the task is marked as Attended with a little checkbox.

When we Save, Cloud Publish, and test, we see this:

The process has paused at the Decision task input mapping. We can inspect the data input and then click the link Verify data inputs.

Now we see the cause of the problem. The data object Current Vacation Status – the output of our Script task – is null. Even though this example is simpler than most you will encounter, the root cause of the problem – some data object is null – is far and away the most common cause of BPMN runtime errors. In real-world process models, it usually boils down to pinpointing which data object has the problem, and Attended tasks let you do that.

The Script task is just a simple filter of the Employee Vacation Days table. If no entry in that table matches the employeeid value of the Vacation Request, Current Vacation Status will be null, and that’s exactly what happened. Most likely the employee entered the wrong value for his employeeid. That certainly could happen, even though our original model did not consider the possibility. But we still need to fix our process so that we don’t get a runtime error in that case. To do that, we can just test for null in a gateway and exit to the end state “Employee not found”.

Now the process runs with no runtime errors even when Vacation Request.employeeid is not found in the HR system. Thorough testing of executable BPMN models is a must, as you cannot assume that only “good” input data will be provided. If you haven’t considered some possibility, no matter how unlikely it seems, sooner or later it will occur. When that happens, Attended tasks make it a lot easier to track down and fix the problem.

In recent posts, I have explained why anyone who can create rich spreadsheet models using Excel formulas can learn to turn those into Low-Code Business Automation services on the Trisotech platform, using BPMN and DMN, and why FEEL and boxed expressions are actually more business-friendly than PowerFX, the new name for Excel’s formula language. That’s the why. Now let’s discuss the how.

In recent client engagements, I’ve noticed a consistent pattern: A subject-matter expert has a great new product idea, which he or she can demonstrate in Excel. A business event, represented by a new row in Excel Table 1, generates some outcome represented by new or updated rows in Tables 2, 3, and so on. The unique IP is encapsulated in the business logic of that outcome. The challenge is to take those Excel examples and generalize the logic using BPMN and DMN, and replace Excel with a real database. This constitutes a Business Automation Service.

Business Automation Service

The Basic Pattern

In BPMN terms, a Business Automation service typically looks like this:

Each service is modeled as a short-running BPMN process, composed of Service tasks, Decision tasks, and Call Activities representing other short-running processes. The service is triggered by a business event, represented by a client API call. The first step is validating the business event. This typically involves some decision logic on the business event and some retrieved existing data. If valid, one or more Service tasks retrieve additional database records, followed by a Decision task that implements the subject matter expert’s business logic in DMN. In this methodology, DMN is used for all modeler-defined business logic, not just that related to decisions. The result of that business logic is saved in new or updated database table records, and selected portions of it may be returned in the service response. It’s a simple pattern, but fits a wide variety of circumstances, from enterprise applications to TurboTax on the desktop.

Implementation Methodology

Over the past year, I’ve developed a methodology for implementing that pattern.

It goes like this:

The first step is whiteboarding the business logic in Excel. In many cases, the subject matter expert has already done this! It’s important to create a full set of example business events, and to use Excel formulas – with references to cells in the event record – instead of hard-coded values. If you can do that, you can learn to do the rest. The subject matter expert must be able to verify the correctness of the outcome values for each business event.

Now, referring to the BPMN diagram, we start in the middle, with the Decision task labeled here Calculate new/updated table records. A BPMN Decision task – the spec uses the archaic term businessRule task – invokes a DMN decision service. So following whiteboarding, the next step in the methodology is creating a DMN model that generalizes the whiteboard business logic. I prefer to put all the business logic in a single DMN model with multiple output decisions in the resulting decision service, but some clients prefer to break it up into multiple Decision tasks each invoking a simpler decision model. That’s a little more work but easier for some stakeholders to understand. This DMN model is in one sense the hardest part of the project, although debugging DMN is today a lot easier than debugging BPMN, since you can test the DMN model within the modeling environment. The subject matter expert must confirm that the DMN model matches the whiteboard result for all test cases.

On the Trisotech platform, Decision tasks are synchronized to their target DMN service, so that any changes in the DMN logic are automatically reflected in the BPMN, without the need to recompile and deploy the decision service. That’s a big timesaver. Decision tasks also automatically import to the process all FEEL datatypes used in the DMN model, another major convenience.

Next we go back to the beginning of the process and configure each step. The key difference between executable BPMN models and the descriptive models we use in BPMN Method and Style is the data flow. Process variables, depicted in the BPMN diagram as data objects, are mapped to and from the inputs and outputs of the various tasks using dotted connectors called data associations. Trisotech had the brilliant idea of borrowing FEEL and boxed expressions from DMN to business-enable BPMN. In particular, the data mappings just mentioned are modeled as FEEL boxed expressions, sometimes a simple literal expression but possibly a context or decision table. If you know DMN, executable BPMN becomes straightforward. FEEL is also used in gateway logic. Unlike Method and Style, where the gate labels express the gateway logic, in executable BPMN they are just annotations. The actual gateway logic is boolean FEEL expression attached to each gate.

Database operations and interactions with external data use Service tasks. In Trisotech, a Service task executes a REST service operation, mapping process data to the operation input parameters and then mapping the service output to other process variables, again using FEEL and boxed expressions. The BPMN model’s Operation Library is a catalogue of service operations available to Service tasks. Entries in this catalogue come either from importing OpenAPI files and OData files obtained from the service provider or manually creating them from service provider documentation. It sounds difficult, but it’s actually straightforward, and business users can learn to do it.

Example

A Stock Trading App

Here is an example to illustrate the methodology. Suppose we want to create a Stock Trading App that allows users to buy and sell stocks, and maintains the portfolio value and performance using three database tables: Trade, a record of each Buy or Sell transaction; Position Balance, a record of the net open and closed position on each stock traded plus the Cash balance; and Portfolio, a summary of the Position Balance table. In Excel, a subject matter expert models those three tables. Each trade adds a row to the Trade table, which in turn adds two rows to the Position Balance table – one for the traded stock and one for Cash, with total account value and performance summarized in the Portfolio table. The only part of this that requires subject matter expertise is calculating the profit or loss when a stock is sold. When that is captured in the Excel whiteboard, translating that to DMN becomes straightforward. The BPMN Decision task executing this DMN service has the added benefit of automatically including the FEEL datatypes of the three tables in the process model.

Next we need to address data validation of a proposed trade. In a previous post, I explained how to do this in DMN. We need to ensure that all of the required data elements are present, and that they are of the proper type and allowed value. If we do not allow trading on margin or shorting the stock, we need to retrieve the account Portfolio record and make sure there is sufficient Cash to cover a buy trade or sufficient shares in the Portfolio to cover a sell trade. A full-featured trading app requires streaming real-time stock quotes, but in a simplified version you could use real-time stock quotes on demand via the Yahoo Finance API, which is free. As in the case of many public APIs, using it requires manually creating the Operation Library entry from service provider documentation, but this is not difficult. Any problems detected generate an error message, and a gateway prevents the trade from continuing.

Also as explained in previous posts, we can use the OData standard to provide Low-Code APIs to the database tables. OData relies on a data gateway that translates between the API call and the native protocol of each database, exposing all basic database operations as cloud REST calls. In its OData integration, Trisotech does not provide this gateway, but we can use a commercial product for that, Skyvia Connect. The Skyvia endpoint exposes Create, Retrieve, Update, and Delete operations on the three database tables, and Trisotech lets us map process variables to and from the operation parameters using boxed expressions and FEEL.

While this Low-Code methodology is straightforward, to become adept at it requires practice. Executable BPMN, even though it’s Low-Code, requires getting every detail right in order to avoid runtime errors. Debugging executable BPMN is an order of magnitude harder than DMN, because you need to deploy the process in order to test it, and instead of friendly validation errors such as you get in the DMN Modeler, you usually are confronted with a cryptic runtime error message. (Note: Trisotech is currently working on making this much easier.) So the modeler still must spend time on testing, debugging, and data validation up front – things developers take for granted but try the patience of most business users.

Using this methodology, it is possible for non-programmers to create a working cloud-based database app based on BPMN and DMN, something most people would consider impossible.

Learning the Methodology

It’s one thing to read about the methodology, quite another to become proficient at it. It takes attention to the small details – many of which are not obvious from the spec or tool documentation – and repeated practice. To that end, I have developed a new course Low-Code Business Automation with BPMN and DMN, where each student builds their own copy of the above-mentioned Stock Trading App. Students who successfully build the app and make trades with it are recognized with certification. Students need to know DMN already, including FEEL and boxed expressions, but the course teaches all the rest: data flow and mapping, Operation Library, OpenAPI and OData, configuration of the various BPMN task types for execution.

The course is now in beta. I expect some enhancements to the Trisotech platform that will be incorporated in the GA version around the end of the year. In the meantime, you can take a look at the Introduction and Overview here. Please contact me if you are interested in this course or have questions about it.