In BPMN Method and Style, which deals with non-executable models, we don’t worry about process data. It’s left out of the models entirely. We just pretend that any data produced or received by process tasks is available downstream. In Business Automation, i.e., executable BPMN, that’s not the case.

Process data is not pervasive, available automatically wherever it’s needed. Instead it “flows”, point to point, between process variables shown in the diagram and process tasks. In fact, data flow is what makes the process executable. With most BPMN tools, defining the data flow requires programming, but Trisotech borrows boxed expressions and FEEL from DMN to make data flow – and by extension, executable process design – Low-Code. This post explains how that works.

First, we need to distinguish process variables from task variables. Process variables are shown in the BPMN diagram, as data inputs, data outputs, and regular data objects. A data input signifies data received by the process from outside. When we deploy an executable process as a cloud service, a data input is an input parameter of the API call that triggers the process. Similarly, a data output is an output parameter of the API call, the response. A data object is a process variable created and populated within a running process.

Task variables are not shown in the process diagram. They are properties of the tasks, and the BPMN spec does not specify how to display them to modelers. What is shown in the diagram are data associations, dotted arrows linking process variables to and from process tasks. A data input association from a process variable to a task means a mapping exists between that variable and a task input. Similarly, a data output association from a task to a process variable means a mapping exists between a task output and the process variable.The internal logic of each task, say a decision service or external cloud service, references the task variables. It makes no reference to process variables.

On the Trisotech platform, the task inputs and outputs are shown in the Data Mapping boxed expressions in the task configuration. Task variable definitions depend on the task type. For Service tasks, Decision tasks, and Call Activities, they are defined by the called element, not by the calling task. For Script and User tasks, they are defined within the process model, in fact within the Data Mapping configuration.

A Service task calls a REST service operation. The Operation Library in the BPMN tool provides a catalog of service operations available to the Service tasks in the process. Operation Library entries are typically created by the tool automatically by importing an OpenAPI or OData file from the service provider, but they can also be created manually from the API documentation. Each entry in the Operation Library specifies, among other details, the service input and output parameters. When you bind a Service task to an entry in the Operation Library, the Service task inputs are the service’s input parameters and the Service task outputs are the service’s output parameters.

A Decision task calls a DMN decision service on the Trisotech platform. In the DMN model, you create the decision service, specifying the output decisions, which become the service outputs, and the service inputs, which could be input data or supporting decisions. When you bind a Decision task to the decision service, the service inputs become the Decision task inputs and the service outputs become the Decision task outputs.

A Call Activity calls an executable process on the Trisotech platform. When you bind the Call Activity to the called process, the data inputs of the called process become the task inputs of the Call Activity, and the data outputs of the called process become the task outputs of the Call Activity.

With Service tasks, Decision tasks, and Call Activities, the task inputs and outputs are given. They cannot be changed in the calling process. It is not necessary for process variables in the calling process to have the same name or data type as the task variables they connect to. All that is required is that a mapping can be defined between them.

Let’s look at some examples. Below we see the DRD of a decision model for automated approval of an employee vacation request, and below that the decision service definition, Approve Vacation Request.

In BPMN, the Decision task Approve Vacation Request is bound to this decision service.

Here you see the process variables Vacation Request (data input), Employee Record (data object), and Requested Days (data object). Vacation Request and Employee Record have data input associations to the Decision task. There is a data output association to Requested Days and two more to variables not seen in this view.

The Data Mapping dialog of Approve Vacation Request shows the mapping between the process variables and the task inputs and outputs. Remember, with Decision tasks, the task inputs and outputs are determined already, as the input and output parameters of the decision service. In the Input Mapping below, the task inputs are shown in the Mapping section, labeled Inner data inputs. The process variables with data input associations are listed at the top, under Context. The Mapping expression is a FEEL expression of those process variables. In this case, it is very simple, the identity mapping, but that is not always the case.

In the Output Mapping below, the task outputs are shown in the Context section, and the process variables in the first column of the Mapping section. The Output Mapping is also quite simple, except note that the task output Vacation Approval is mapped to two different process variables, Approval and Approval out. That is because a process data output cannot be the source of an incoming data association to another task, so sometimes you have to duplicate the process variable.

Not all data mappings are so simple. Let’s look at the Service task Insert Trade in the process model Execute Trade, below.

Insert Trade is bound to the OData operation Create trades, which inserts a database record. Its Operation Library entry specifies the task input parameter trade, type NS.trade, shown below.

Actually, because the ID value is generated by the database, we need to exclude it from the Input Mapping. Also, the process variable trade uses a FEEL date and time value for Timestamp instead of the Unix timestamp used in NS.trade. We can accommodate these details by using a context in the Input Mapping instead of a simple literal expression. It looks like this:

The context allows each component of the task input trade to be mapped individually from the process variables trade and Quote (which provides the price at the time of trade execution). As we saw with the Decision task, the Context section lists the available process variables, and the first column of the Mapping section lists the task inputs. Note also that the Price mapping uses the conditional boxed expression for if..then..else, new in DMN 1.4, designed to make complex literal expressions more business-friendly.

Finally, let’s look at the Script task in this process, Validate Trade Value. Script and User tasks differ from the other task types in that the modeler actually defines the task inputs and outputs in the Data Mapping. On the Trisotech platform, a Script task has a single task output with a value determined by a single FEEL literal expression. As you can see in the process diagram, the process variables with incoming data associations to Validate Trade Value are trade, Quote, and portfolio, and the task output is passed to the variable Error list. This task checks to see if the value of a Buy trade exceeds the portfolio Cash balance or the number of shares in a Sell trade exceeds the portfolio share holdings.

In the Script task Input Mapping, the modeler defines the names and types of the task inputs in the first column of the Mapping section. The mapping to those task inputs here are more complex literal expressions. The script literal expression references those task inputs, not the process variables. User tasks work the same way, except they are not limited to a single task output.

Hopefully you see now how data flow turns a non-executable process into an executable one. The triggering API call populates one or more process data inputs, from which data is passed via Input Mapping to task inputs. The resulting task outputs are mapped to other process variables (data objects), and so forth, until the API response is returned from the process data output. On the Trisotech platform, none of this requires programming. It’s just a matter of drawing the data associations and filling in the Data Mapping tables using FEEL.

BPMN has an element that looks very much like a subprocess, except drawn with a thick border. Perhaps you have used it yourself… probably incorrectly. Its name is call activity, and it is similar in several ways to a subprocess, but it is not the same. Both elements are important, but valuable for different reasons. This post will explain.

Call activity and subprocess share important characteristics. Both are simultaneously a single activity and an activity flow from start event to end event. And both may be rendered in the diagrams in two different ways: collapsed, denoted with a [+] marker at bottom center, or expanded, an enlarged rounded rectangle enclosing the flow. In BPMN 1.x, call activity and subprocess were actually variants of the same element. What we today know as a subprocess back then was called an embedded subprocess, and call activity was called a reusable subprocess. Of course, no one knew exactly what that meant. Some thought it had to with the shape – collapsed or expanded – but it had nothing to do with that. BPMN 2.0 resolved the distinction more clearly.

A subprocess is really just a graphical device. It allows a process fragment to be collapsed into a single activity to allow a higher level view of the end-to-end process. Rather than needing to create a high-level process model you can print on one page and a separate detailed model that stretches over six feet of wall space, and keeping those models in sync as the process definition evolves, subprocesses allow a single model of cross-functional end-to-end processes containing both high-level and detailed views. That model is not a single diagram but a hierarchy of diagrams, in which the contents of subprocesses are detailed in separate child-level diagrams. This presupposes use of the collapsed subprocess shape, with the child level drawn in a separate diagram, as opposed to the expanded shape, where parent and child levels are drawn in the same diagram.

In my work, I rarely if ever use the expanded subprocess shape. In Method and Style, my top-down methodology for non-executable models, you start with the collapsed subprocess in the parent-level diagram, then click the [+] marker to create a hyperlinked diagram where you add the child-level flow. Method and Style gives subprocesses additional importance by interpreting the labels of the child-level end events as the distinct end states of the subprocess, and requiring that any subprocess with multiple end states be followed by an XOR gateway, one gate per end state with matching labels. In this way, the end-to-end flow logic is evident from the printed diagrams even without defining process data.

While Method and Style simply would not work without subprocesses, the child-level flows enclosed by each subprocess have no independent meaning or definition. They are simply fragments of a larger process selected by the modeler to simplify visualization of the end-to-end process logic. This is fine for non-executable models, but it is far less valuable in executable BPMN. A subprocess is not executable. The unit of execution in BPMN is a process. So if you want to make some fragment of a larger process executable, you need to define that fragment as a top-level process and invoke it from the larger process using a call activity. The flow contained in a call activity is an independently-defined process. Where BPMN 1.x described call activity as a reusable subprocess, a better description would be an executable subprocess.

Call activities are as essential to Business Automation as subprocesses are to Method and Style . Executable models add to descriptive BPMN the elements of data flow – process variables (data objects) and their mapping to and from task inputs and outputs. In Business Automation, the shape labels so important in Method and Style effectively become annotations; the real logic is defined by the process data. Every task in executable BPMN defines a data mapping from process variables to the task inputs and from the task outputs to other process variables. The task inputs of a call activity are, by definition, the data inputs of the called process, and the task outputs of a call activity similarly are the data outputs of the called process. In the calling process, the call activity specifies a mapping from its process variables to and from the data inputs and outputs of the called process. The existence of such a mapping is indicated by data associations in the parent-level diagram. The data mappings themselves are defined in configuration of the call activity.

This mapping is what makes the called process reusable. Its data inputs and outputs are independent of the variables of any calling process, so long as there is a definable mapping. In contrast, the child level of a subprocess is not reusable, meaning defined once and used multiple times. It has no definition outside of the context of the larger process in which it is embedded. More to the point, a subprocess has no data inputs or data outputs. The individual tasks within the child level have them, but the subprocess as a whole does not. And this in fact is what makes subprocesses less useful in executable BPMN, because in the parent level diagram you cannot draw data flow to the boundary of a collapsed subprocess. Instead you need to draw the data flow in the child-level diagram only, losing the visual context of the flow from and back to the parent level. (You could fake it by using directional association connectors in the parent-level diagram – actually not a bad idea, and more consistent with Method and Style – but a real data association is not allowed by the metamodel.)

In Business Automation, a BPMN process can be compiled and deployed as a cloud REST service. As I have described numerous times in past posts, this requires just a single Cloud Publish mouse click on the Trisotech platform. And you could invoke that process from another BPMN process with a normal Service task. But when the calling process and the called process share a common platform, such as Trisotech, invocation with a call activity is much better. That’s because, while independently defined, the calling process and the called process are tightly coupled. The call does not have to travel across the internet to invoke a service generated by a specific version of the process. The call stays within the common platform and invokes the latest version of the called process, which does not even have to be compiled and deployed. That may seem like a small thing, but in practice it makes development much easier.

While it is possible to create executable models top-down as in Method and Style, more often I will define the called processes first as independent models and then assemble them in the calling process using call activities. In the Trisotech Workflow Modeler, to bind a call activity shape to the called process, you can either drag the called process onto the diagram from the model repository (called Digital Enterprise Graph) or click on the Reuse button on the menu, which lets you pick from processes in the model repository. The call activity shape then displays a lock icon, indicating it is bound to the called process, inheriting its defined datatypes and synchronized to any changes in its internal logic.

Another important benefit of call activities in Business Automation is testing, which often takes as much effort as the design itself. Unlike DMN, which you can test in bits and pieces in the modeling environment, testing BPMN requires compilation and deployment, possible only at the process level. Breaking a complex process into small executable fragments makes the testing much more manageable, and the larger process can then assemble those fragments – which are now also reusable in other scenarios – using call activities.

The bottom line is that call activities play as vital a role in executable BPMN as subprocesses do in non-executable BPMN. Call activities are useful in non-executable BPMN only in the rare case where a subprocess is used multiple times in the same process or, possibly, where an organization has standardized the names of subprocesses for reuse across the enterprise. And while subprocesses are important in non-executable BPMN – Method and Style, in particular – they are most often better replaced with call activities in executable models.

In recent posts, I have explained why anyone who can create rich spreadsheet models using Excel formulas can learn to turn those into Low-Code Business Automation services on the Trisotech platform, using BPMN and DMN, and why FEEL and boxed expressions are actually more business-friendly than PowerFX, the new name for Excel’s formula language. That’s the why. Now let’s discuss the how.

In recent client engagements, I’ve noticed a consistent pattern: A subject-matter expert has a great new product idea, which he or she can demonstrate in Excel. A business event, represented by a new row in Excel Table 1, generates some outcome represented by new or updated rows in Tables 2, 3, and so on. The unique IP is encapsulated in the business logic of that outcome. The challenge is to take those Excel examples and generalize the logic using BPMN and DMN, and replace Excel with a real database. This constitutes a Business Automation Service.

Business Automation Service

The Basic Pattern

In BPMN terms, a Business Automation service typically looks like this:

Each service is modeled as a short-running BPMN process, composed of Service tasks, Decision tasks, and Call Activities representing other short-running processes. The service is triggered by a business event, represented by a client API call. The first step is validating the business event. This typically involves some decision logic on the business event and some retrieved existing data. If valid, one or more Service tasks retrieve additional database records, followed by a Decision task that implements the subject matter expert’s business logic in DMN. In this methodology, DMN is used for all modeler-defined business logic, not just that related to decisions. The result of that business logic is saved in new or updated database table records, and selected portions of it may be returned in the service response. It’s a simple pattern, but fits a wide variety of circumstances, from enterprise applications to TurboTax on the desktop.

Implementation Methodology

Over the past year, I’ve developed a methodology for implementing that pattern.

It goes like this:

The first step is whiteboarding the business logic in Excel. In many cases, the subject matter expert has already done this! It’s important to create a full set of example business events, and to use Excel formulas – with references to cells in the event record – instead of hard-coded values. If you can do that, you can learn to do the rest. The subject matter expert must be able to verify the correctness of the outcome values for each business event.

Now, referring to the BPMN diagram, we start in the middle, with the Decision task labeled here Calculate new/updated table records. A BPMN Decision task – the spec uses the archaic term businessRule task – invokes a DMN decision service. So following whiteboarding, the next step in the methodology is creating a DMN model that generalizes the whiteboard business logic. I prefer to put all the business logic in a single DMN model with multiple output decisions in the resulting decision service, but some clients prefer to break it up into multiple Decision tasks each invoking a simpler decision model. That’s a little more work but easier for some stakeholders to understand. This DMN model is in one sense the hardest part of the project, although debugging DMN is today a lot easier than debugging BPMN, since you can test the DMN model within the modeling environment. The subject matter expert must confirm that the DMN model matches the whiteboard result for all test cases.

On the Trisotech platform, Decision tasks are synchronized to their target DMN service, so that any changes in the DMN logic are automatically reflected in the BPMN, without the need to recompile and deploy the decision service. That’s a big timesaver. Decision tasks also automatically import to the process all FEEL datatypes used in the DMN model, another major convenience.

Next we go back to the beginning of the process and configure each step. The key difference between executable BPMN models and the descriptive models we use in BPMN Method and Style is the data flow. Process variables, depicted in the BPMN diagram as data objects, are mapped to and from the inputs and outputs of the various tasks using dotted connectors called data associations. Trisotech had the brilliant idea of borrowing FEEL and boxed expressions from DMN to business-enable BPMN. In particular, the data mappings just mentioned are modeled as FEEL boxed expressions, sometimes a simple literal expression but possibly a context or decision table. If you know DMN, executable BPMN becomes straightforward. FEEL is also used in gateway logic. Unlike Method and Style, where the gate labels express the gateway logic, in executable BPMN they are just annotations. The actual gateway logic is boolean FEEL expression attached to each gate.

Database operations and interactions with external data use Service tasks. In Trisotech, a Service task executes a REST service operation, mapping process data to the operation input parameters and then mapping the service output to other process variables, again using FEEL and boxed expressions. The BPMN model’s Operation Library is a catalogue of service operations available to Service tasks. Entries in this catalogue come either from importing OpenAPI files and OData files obtained from the service provider or manually creating them from service provider documentation. It sounds difficult, but it’s actually straightforward, and business users can learn to do it.

Example

A Stock Trading App

Here is an example to illustrate the methodology. Suppose we want to create a Stock Trading App that allows users to buy and sell stocks, and maintains the portfolio value and performance using three database tables: Trade, a record of each Buy or Sell transaction; Position Balance, a record of the net open and closed position on each stock traded plus the Cash balance; and Portfolio, a summary of the Position Balance table. In Excel, a subject matter expert models those three tables. Each trade adds a row to the Trade table, which in turn adds two rows to the Position Balance table – one for the traded stock and one for Cash, with total account value and performance summarized in the Portfolio table. The only part of this that requires subject matter expertise is calculating the profit or loss when a stock is sold. When that is captured in the Excel whiteboard, translating that to DMN becomes straightforward. The BPMN Decision task executing this DMN service has the added benefit of automatically including the FEEL datatypes of the three tables in the process model.

Next we need to address data validation of a proposed trade. In a previous post, I explained how to do this in DMN. We need to ensure that all of the required data elements are present, and that they are of the proper type and allowed value. If we do not allow trading on margin or shorting the stock, we need to retrieve the account Portfolio record and make sure there is sufficient Cash to cover a buy trade or sufficient shares in the Portfolio to cover a sell trade. A full-featured trading app requires streaming real-time stock quotes, but in a simplified version you could use real-time stock quotes on demand via the Yahoo Finance API, which is free. As in the case of many public APIs, using it requires manually creating the Operation Library entry from service provider documentation, but this is not difficult. Any problems detected generate an error message, and a gateway prevents the trade from continuing.

Also as explained in previous posts, we can use the OData standard to provide Low-Code APIs to the database tables. OData relies on a data gateway that translates between the API call and the native protocol of each database, exposing all basic database operations as cloud REST calls. In its OData integration, Trisotech does not provide this gateway, but we can use a commercial product for that, Skyvia Connect. The Skyvia endpoint exposes Create, Retrieve, Update, and Delete operations on the three database tables, and Trisotech lets us map process variables to and from the operation parameters using boxed expressions and FEEL.

While this Low-Code methodology is straightforward, to become adept at it requires practice. Executable BPMN, even though it’s Low-Code, requires getting every detail right in order to avoid runtime errors. Debugging executable BPMN is an order of magnitude harder than DMN, because you need to deploy the process in order to test it, and instead of friendly validation errors such as you get in the DMN Modeler, you usually are confronted with a cryptic runtime error message. (Note: Trisotech is currently working on making this much easier.) So the modeler still must spend time on testing, debugging, and data validation up front – things developers take for granted but try the patience of most business users.

Using this methodology, it is possible for non-programmers to create a working cloud-based database app based on BPMN and DMN, something most people would consider impossible.

Learning the Methodology

It’s one thing to read about the methodology, quite another to become proficient at it. It takes attention to the small details – many of which are not obvious from the spec or tool documentation – and repeated practice. To that end, I have developed a new course Low-Code Business Automation with BPMN and DMN, where each student builds their own copy of the above-mentioned Stock Trading App. Students who successfully build the app and make trades with it are recognized with certification. Students need to know DMN already, including FEEL and boxed expressions, but the course teaches all the rest: data flow and mapping, Operation Library, OpenAPI and OData, configuration of the various BPMN task types for execution.

The course is now in beta. I expect some enhancements to the Trisotech platform that will be incorporated in the GA version around the end of the year. In the meantime, you can take a look at the Introduction and Overview here. Please contact me if you are interested in this course or have questions about it.

Last month I showed why Trisotech is a great Low-Code Business Automation platform, based on its use of FEEL and boxed expressions in executable BPMN. How ironic is it, then, that many decision management vendors don’t even include those features in their DMN tools!

The only part of DMN they use is the Decision Requirements Diagram (DRD), but what is the point of using a standard diagram to describe business requirements that are not even testable?
Answer: So they can claim their proprietary tools “follow the standard”.) When pressed, they say that FEEL and boxed expressions are for programmers, “too hard” for business users. In reality, they don’t want business users anywhere near the decision logic.

FEEL vs Excel

So let’s put that aside. A more useful discussion is FEEL vs Excel. Does anyone claim that Excel is for programmers, not for business users? Of course not. But I believe that FEEL and boxed expressions are actually more business-friendly than Excel formulas. Allow me to demonstrate.

First, however, I need to say that I love Excel, and I even use it as part of my Business Automation methodology. What makes Excel formulas truly great is that they are always live. A spreadsheet cell defined as a formula immediately displays the result whenever any cell it depends on changes. You don’t have to click a Submit button; the response is immediate and automatic. The other business-friendly feature of Excel formulas is that their arguments are selected by clicking in a cell, which references the value by its grid location, such as A1. Yes, it’s possible to assign names to cell ranges, but come on, do you do that? No one does that.

I contend that these features make Excel great for creating examples of the business logic, but not so good for generalizing the logic as you would do in an app. Here is an illustration.

In my Business Automation training, the class builds a Stock Trading App. In the methodology we start by whiteboarding the business logic using examples in Excel. With each new Trade event, two rows are added to the PositionBalance table, one for the traded ticker and one for Cash. Without going into the details, the diagram below is an example of that.

Let’s look at the CostBasis column after sale of a stock, such as cell I15. In this table, the value of CostBasis is a simple Excel formula. Whenever a cell definition starts with =, it’s a formula. And this formula references another column of the same row, and some columns from two rows earlier. A subject matter expert can easily create logic in this way based on examples. But why does it reference values from two rows earlier? The answer in this case is that it is the most recent row with the same value of column D, the Ticker. It won’t always be two rows earlier. To generalize the logic, you need a lookup formula.

No problem. Excel has many built-in functions in its Formulas ribbon. The modern lookup formula is called XLOOKUP, with the explanation below:

Using this, for cell I15, we can generalize the formula H15*I13/H13, where row 13 is the last row preceding row 15 in which the column D value is the same, as:

It’s always live, but is it business-friendly?

Hardly. And that’s not the half of it, because the simple Excel formula in cell I15 only applies to tickers other than Cash when TradeShares is negative. Other formulas apply for Cash and positive TradeShares! So to create the general case of the PositionBalance table in Excel would be extremely difficult.

Let’s compare FEEL and boxed expressions. The complexity of XLOOKUP was related simply to finding the last PositionBalance record for the ticker in question. In Low-Code BPMN, following a query of the PositionBalance table of records matching the ticker name, the variable Last position is a simple filter expression, looking for maximum ID value. It’s much simpler than XLOOKUP.

And remember, we need a different CostBasis formula depending on Cash or non-Cash, and Buy or Sell trade. In Excel formulas, you would need to wrap all that XLOOKUP business inside IF functions:

IF([cond], [complex XLOOKUP formula], IF([cond2], [complex XLOOKUP formula2], [else formula]))

What a mess! FEEL and boxed expressions handle that easily as well:

Here we have a decision table inside a context, where each context entry models a column of PositionBalance. The only complexity here is accounting for the possibility that there is no previous PositionBalance record for the ticker, and even that is a simple if..then..else.

And this illustrates a major advantage of FEEL over Excel formulas. Except for arithmetic operators (+, *, etc.) and comparison operators (=, >, etc.), Excel formulas rely entirely on nested functions. They don’t even have logical operators like and. FEEL allows nested functions as well, but it relies on a long list of operators that greatly simplify the syntax: if..then..else, for..in..return, filters, logical operators, and so forth. There is simply no comparison; FEEL is much more business-friendly than Excel formulas!

Here is another example, published by Microsoft to show off the Excel formula language, which is being rebranded Power FX, the Low-Code language behind Microsoft PowerApps. It’s actually very cool, but what’s cool is the always live part, not the formula language behind it.

The formula in cell A2 displays the text following the last space in cell A1, and it does this live as you type in cell A1. Take a look at the Power Apps version:

Doing this in the Excel formula language is impressive, but I could not decode the deeply nested function logic without instructions from StackOverflow:

LEN(A1)-LEN(SUBSTITUTE(A1," ","")) Count of spaces in the original string
SUBSTITUTE(A1," ","|", ... ) Replaces just thefinalspace with a|
FIND("|", ... ) Finds the absolute position of that replaced|(that was the final space)
Right(A1,LEN(A1) - ... )) Returns all characters after that|

Now Trisotech currently does not have the always live feature – I hope that it will someday! – but FEEL in a boxed context certainly can implement the Excel formula logic in a much more business-friendly way.

You don’t really even need the context, as the single literal expression is simple enough:

split(Input string, " ")[-1]

Now you tell me, which is more business-friendly, FEEL or Excel formula language, Power FX?

There is no hotter segment of Business Automation software today than Low-Code.

Low-Code refers to application development based on models – diagrams and tables – and business-accessible expression languages, not program code. Gartner assesses Low-Code as a $13.8 billion market in 2021, growing to $46.6 billion in 2023. And, they say, by 2024 it will account for 65% of all application development! According to Forrester, 100% of companies that have adopted Low-Code report satisfactory return on investment. KPMG research indicates that in the past year, the number of executives naming Low-Code as their most important automation investment has nearly tripled. So it’s a big deal.

Trisotech doesn’t call its platform Low-Code, and I understand why, but to me it absolutely is. It checks off all the boxes except for one – and that one isn’t all that important when we restrict ourselves to Business Automation Services. Moreover, the vast majority of Low-Code tools are based on proprietary diagrams and expression languages. Trisotech’s ace in the hole is its tools are based on vendor-independent standards: BPMN and DMN, of course, and also OpenAPI and OData. This post will explain why all this is important, and how you can take advantage of it.

What Is Low-Code?

Low-Code doesn’t mean full-blown apps can be created by ordinary business users with no developer assistance. Microsoft’s term for the user of Low-Code tools is citizen developer – possibly a technically oriented business user, possibly a developer. I would call any business user who can use the Formulas tab in Excel (not Macros, that’s code) a citizen developer. A survey by Low-Code vendor Creatio found that only one third of citizen developers using Low-Code were business users. In fact, some Low-Code implementations are done entirely by developers.

All surveys report the key benefit of Low-Code is time to value. Whether implemented by business users, developers, or a team involving both, Low-Code solutions that take months with traditional development can be rolled out in a few weeks with Low-Code. One factor is the tooling, as model-based development is just faster than code. Another factor is the scarcity of developer resources, especially in the wake of COVID. With Low-Code, subject matter experts – citizen developers – can get started, or even complete the job, themselves, without having to wait months for available IT resources.

The Business Automation Landscape

There are now hundreds of software products that call themselves Low-Code. Each typically focuses on one of three application types:

1. Automated Workflow Processes, long-running flows involving activities performed by multiple users and systems. While the majority of BPM Suites, including the BPMN-based ones, are still aimed at Java developers, a few are now focused on Low-Code.
2. Page Flows, web applications facing a single user involving interactions with multiple systems. A good example here is Microsoft PowerApps.
3. Straight-Through Processes, short-running flows involving interactions with multiple systems, but with no human involvement other than initiating their execution.

In addition to segmenting Business Automation tools as Code vs Low-Code, we can also segment by Proprietary vs Standards-Based. The key standards for Business Automation are BPMN, DMN, and (eventually) CMMN, which have the added benefit of being designed to work together. DMN, the standard for defining business logic, was conceived from the start as Low-Code, using standard tabular formats called boxed expressions in conjunction with FEEL, a business-friendly expression language. BPMN, the standard for service orchestration (workflow), was not: While the basics of the flow are described by a diagram, the details required to make the flow executable – the data and expression language – typically require a Java developer.

Trisotech’s brilliant idea was to borrow boxed expressions and FEEL from DMN and use them in BPMN. Unlike other platforms, Trisotech offers a standards-based Low-Code approach to Business Automation, in particular applications of the third type, Straight-Through Processes. This suggests the view of the Business Automation landscape pictured below, with Trisotech in the sweet spot, Low-Code and Standards-Based.

Features of a Low-Code Platform

To be included in their Magic Quadrant report on Low-Code Application Platforms, Gartner lists the following features as required:

Feature	Trisotech Support
Low-Code design of data	Yes
Includes database and UI features	Not yet
Able to call third-party APIs	Yes
Application versioning	Yes
Low-Code design of UIs, business logic, and data	Yes for logic and data; not yet for UIs
Tools accessible to business users	Yes
One-step deployment	Yes
Repository of reusable components	Yes

The glue that binds all this together in a Low-Code way is the combination of FEEL and boxed expressions. These are used not only in the DMN business logic but in BPMN as well, mapping between process variables (FEEL) and the inputs and outputs of Service tasks, Decision tasks, and User tasks, as well as for gateway logic. Basically, if you know your way around DMN, you can create Business Automation services yourself.

The Biggest Obstacle

In Creatio’s survey, 60% of respondents reported the biggest obstacle to Low-Code adoption is simply the lack of experience with the platforms. The solution there is obvious: Ask Trisotech for a trial account. But creating Business Automation services is not as easy as just turning on the tool. You need to know DMN inside out, at the level of my DMN Advanced training. And beyond that, executable BPMN involves countless details that are not covered in BPMN Method and Style, which is about non-executable processes. It requires an additional training course, and I’m working on that right now. Stay tuned for more details.

Students in my BPMN Method and Style training are often befuddled by how to start a process. I see Conditional events, Error events, all kinds of things. No, stop! While the BPMN spec provides many different types of start events, only three of them are relevant to the non-executable flows most modelers are trying to create. In Method and Style, those are the only ones allowed. This post will explain.

Just Three Ways to Start

There are really only three ways to start a process:

1. On external request. Here “external” means some person or entity that is not a task performer of the process. The Customer, for example, is always external. Even in internal employee-facing processes, the requester can be considered external if he or she performs no activity in the process, merely requests it and receives back some final status. By far the majority of BPMN processes start on external request.
2. On a regular schedule, for example on the first day of every month, or every 10 minutes. In non-executable BPMN it doesn’t matter whether the process is initiated automatically or manually, as long as the process runs repeatedly on a schedule.
3. Manually by a task performer. Occasionally, a person who performs tasks in the process initiates it manually. Some employee-facing processes work this way, although in many cases the initiator could be considered an external requester.

That’s it, just those three. This makes life much simpler for the process modeler, because it not only determines which start event to use but defines the process instance, as well.

Message Start

A process that starts on external request uses a Message start event. Method and Style says the label of the event should be “Receive [messageName]” and requires an incoming message flow labeled with the messageName. The messageName should be just a noun – the name of the message, like “Order” – not an action like “Receive Order” or a state like “Order received”.

The great thing about Message start events is that they tell you right away what the process instance represents. A process model defines a flow that is performed repeatedly in the course of business – not continuously – where each repetition, or instance, of the process has a precise start and end. Understanding the process instance is crucial for creating properly structured models, but students initially have a hard time with that. With a Message start event, it’s easy: Each instance is the handling of that start message, such as an Order. The reason this is important is that the instance of every activity in the process must have one-to-one correspondence with the process instance. (That is not Method and Style; that is from the spec, although not clearly stated as such.) So, for example, if the Message start event is “Receive Order,” every activity in the process must be performed per Order. An activity that adjusts discount codes every month, while related to this process, cannot be part of the process. It must be part of a different process. We’ll see an example of this later.

Timer Start

A process performed on a regular schedule, whether initiated manually or automatically, uses a Timer start event. The label of the event should be the frequency of occurrence, such as “Monthly” or “Every 10 minutes”.

Again, the start event identifies the process instance: It is that single occurrence. And again, every activity in the process must pertain to that occurrence, not a previous occurrence or future occurrence or some individual request received during that occurrence. For example, suppose there is a Project Review process that occurs monthly. The start event is Timer with label “Monthly”, and each instance is a single monthly occurrence. Suppose time runs out to cover all the issues, and some are deferred to next month. A gateway that loops back to the start to handle those would be incorrect! Handling of next month’s issues occurs in next month’s instance of Project Review.

None Start

A process initiated manually by a task performer uses a None start event, with no trigger icon.

It’s unfortunate that BPMN does not have a Manual event trigger, so manual start must use None start, which is also used for the start event of subprocesses (which are started by the incoming sequence flow, not an event) and also loosely specified processes where the start conditions are undefined. In manually started processes, the instance is not easily determined.

Fortunately, these processes are fairly uncommon. I find in most cases where students use a None start, the process is actually in response to an external request, so Message start would be correct.

Instance Alignment

As I mentioned, the BPMN spec requires that instances of all activities in a process must align with the process instance. Otherwise they must be modeled in a separate process that interacts with the first process either via message flows or a shared datastore. Let’s return to the case of an Order process that involves discount codes that are updated monthly. The activity that updates the discount codes cannot be part of the Order process because it is performed monthly, not per Order. It is part of a separate process, Update Discounts, modeled with a Timer start event labeled “Monthly”. The activity of validating the requested discount code against the current list is done per Order, so that is part of the Order process. The link between the two processes, which must be modeled as separate pools, is the datastore “Discount Codes”. The datastore is updated by the Update Discounts process and queried by the Order process.

Method and Style

Method and Style is a modeling methodology that makes the details of the process flow unambiguous from the printed diagram alone… clear even to those who don’t already know how the process works. It addresses the chief complaint of managers, which is the inability of their team’s process diagrams to be understood by all team members without an accompanying text document. The Trisotech platform supports Method and Style by including Style rule validation within the tool. For example, a Message start event without an incoming message flow generates a style error. My BPMN Method and Style training teaches you how to create proper process models consistent with both the spec and the methodology. It includes 60-day use of the Trisotech Workflow Modeler and post-class certification. Certification is based on an exam and completion of a process model reviewed by me and iterated until it is perfect. Perfection of the certification exercise is where most students internalize the classroom teaching.

BPMN has a way to say an activity should be performed more than once. In fact, it has multiple ways, and students in my BPMN Method and Style training sometimes get them confused. This post will clear things up.

A Loop activity is like a Do-While loop in programming. It means perform the activity once – it could be either a task or subprocess – and then test the loop exit condition, a Boolean expression of process data. If the condition is false, perform the activity again and evaluate the condition once more. If the condition is true, exit the activity on the normal outgoing sequence flow. Repeat until the exit condition is satisfied. It’s quite handy for activities that might require multiple tries to complete successfully. It is possible to set a maximum number of allowed iterations.

The semantics of a Loop activity are exactly the same as a non-Loop activity followed by an XOR gateway that either loops back to the activity or continues on. A Loop activity is indicated in the diagram by a circular arrow at the bottom center. This takes up less space in the diagram than adding the gateway and loopback, but it hides the loop exit condition. For that reason, Method and Style asks the modeler to insert a text annotation on any Loop activity, labeled “Loop until [condition]”.

A Multi-instance (MI) activity is like a For-Each loop in programming. It means perform the activity once – again, either a task or subprocess – for each item in a list, a process data element that contains multiple items. Processing each list item is a separate instance of the activity, and the MI activity as a whole is complete only when all the item instances are complete.

A Multi-instance activity is indicated by three parallel bars at bottom center. If the bars are vertical, it means the instances are performed at the same time, in parallel. If the bars are horizontal, which you often see with human activities, it means the instances are performed sequentially. With an MI activity, Method and Style asks the modeler to insert a text annotation, “For each [item]”.

Many beginners are confused by the difference between Loop and MI-sequential activities. They are not the same!

The key points about Loop activities are:

• The iterations are strictly sequential. Iteration 2 does not begin until iteration 1 has completed.
• You don’t know how many iterations are required when you begin. The activity simply repeats until its loop exit condition is satisfied, or the maximum number of iterations is reached.
• The loop condition is the same expression with each iteration, but each iteration can change data values used in the loop exit condition. Without that, the loop exit condition could never go from false to true!

MI-sequential is not the same:

• As with Loop, the iterations are strictly sequential. Iteration 2 does not begin until iteration 1 has completed.
• You do know how many iterations are required when you begin. It’s the number of items in the list!
• Each iteration is an independent instance, so the data values of one iteration generally do not affect the logic of other instances, although it is possible they could interact via a shared data store.

Let’s look at a simple example of an order.

When the order is received, the process first checks whether the items are in stock. Because an order may contain multiple items, Check stock must be done independently for each order item. In the diagram we indicate this by the text annotation. Let’s say an item is either in stock or out of stock, ignoring the case where the available stock only partially satisfies the order. Since this is processing a list, it’s MI – not Loop! The number of iterations is the number of items in the order.

Normally, in Method and Style, when an activity has two possible outcomes – in stock or out of stock – we follow it with a gateway with two gates, in stock and out of stock. But remember, with an MI activity, each instance has those two outcomes, so the MI activity as a whole has more than two possible end states. In practice, it is common to follow such an MI activity with a gateway that tests whether any item is in stock, or possibly if any item is out of stock.

Here we will continue the process if some order items are in stock. Otherwise we end the process in the end state Out of stock.

Now we Collect payment, which uses the customer’s credit card on file. That could fail. Maybe the card is expired, in which case the customer is advised to enter a valid credit card number. Collect payment is thus a Loop activity. You don’t know, when you start, how many iterations are required. You keep trying it until either it succeeds or you exhaust the allowed number of attempts. Again the loop exit condition and maximum iterations are indicated by a text annotation. There is only one instance of this activity. In the end it either succeeds or fails, so we follow it with a gateway with those two gates.

Here are some common mistakes beginners make with repeating activities:

• Loop within a loop. A Loop activity is the same as a non-Loop activity followed by a gateway that tests the exit condition and possibly loops back to the activity. But if you follow a Loop activity by such a gateway, that’s a loop within a loop… usually not what was intended. A second case of loop within a loop is a subprocess that contains a Loop marker on the collapsed shape in the parent level and a gateway loopback in the child level… again usually not what was intended.
• MI within an MI. Similar to the second case of loop within a loop, an MI subprocess has activities in the child level also marked MI. Again, this is not usually what was intended.
• Error boundary event on MI subprocess. In a Multi-instance subprocess, there are N separate instances of the child level flow. If any one of them throws an error, this terminates all N instances. That may be what you intended, but if you just want to handle the exception for that one instance, you need to do that within the child level, not throw it to the parent level subprocess boundary.

To make a Multi-instance activity executable in the Trisotech Digital Enterprise Suite, here is how you do it. From our previous example, consider the MI task Check stock.

The input is an Order, type tOrder shown below. It is a collection of Order items, each defined by a SKU, Quantity, and UnitPrice.

In a real process, each instance of Check stock would be a database lookup, but to keep it simple here we just use a script task against the process data input Stock table, a collection of items defined by SKU and QuantityAvailable. The data output In stock items is a list of Order items that are in stock.

We first need to map the data inputs to variables in the script task context using its Data Mapping attribute.

A Multi-instance activity always iterates over a collection, so we need to select it from those supplied by input data associations, here Order. We need to assign a name to the range variable or iterator, meaning one item in the collection. It can be anything we want, so here we name it OrderItem. In the Mapping section we define the variables used in the script task logic and their mappings from the input data associations and the iterator. So variable Stock table is just the input data association of that name, and we define a new variable CurrentItem mapped from the iterator OrderItem. The mapping expression of the iterated variable must be the iterator.

The script task logic is the FEEL expression

if Stock table[SKU = CurrentItem.SKU].QuantityAvailable[1] >= CurrentItem.Quantity then CurrentItem else null

For those unfamiliar with FEEL, here is how it works. Remember we are executing the script independently over each OrderItem, and collecting the results of all the instances.

Stock table[SKU=CurrentItem.SKU].QuantityAvailable

selects the row of Stock table where the SKU matches the CurrentItem.SKU, and then finds the QuantityAvailable for that row. Since SKU is a primary key, the filter selects a single row, but technically a FEEL filter always returns a list, so we need to append [1] to extract the item. Now the script says, if that QuantityAvailable is greater than or equal to the corresponding OrderItem quantity, return the CurrentItem, otherwise return null. We run this script for each OrderItem.

Now we define the data output mapping.

A script task always has a single output. We need to assign it to a variable, here In stock item. For Resulting collection, we select one of the output data associations, here the process data output In stock items. Collected variable says that each item in that collection is assigned the script task output In stock item. Thus the data output In stock items is a collection containing the original Order item if in stock or null if out of stock.

We use In stock items in the logic of the gateway Any in stock? The yes gate condition is defined by the FEEL Boolean expression:

nn count(In stock items)>0,

where nn count(list) is a Trisotech extension function equivalent to count(list[item!=null]).

When we test the logic in Service Library TryIt, we get the expected result. Order item A001 is in stock and Order item A003 is out of stock. Thus In stock items has two items, one of them null, and the process end state is “Some in stock”.

To recap, Loop and MI activities are frequently used in BPMN and they are not the same thing. Use Loop when you need to retry an activity until it succeeds. Use MI – either sequential or parallel – when you need to perform an action for each item in a list.

In BPMN, the most common way information is provided to a process from an external source is via a message. Incoming messages are indicated in the diagram by a dashed arrow connector called a message flow, with its tail on a black-box pool or a message node in another process pool and its arrow on a message node in the receiving process, either an activity or message event. In our BPMN Method and Style training, which focuses on non-executable processes, we discuss at length the semantics and usage patterns of messages and message events. But when we take the next step of making our BPMN model executable, we need to think about messages in a new way. This post explains how it works.

Below is a variant of an example from the training, illustrating the use of message boundary events.

The scenario is as follows:

• The Customer sends an Order message to the start event, creating a new instance of the process. Initiation by a request from an external entity is the most common way processes are started, i.e., a message flow to the process start event.
• At some time before completion of the process, the Customer could possibly send a Cancellation message, cancelling the order.
• The handling of this cancellation depends on the state of the process when this message is received. If payment has not yet been completed, all process activity is immediately terminated and the process ends in the end state Order cancelled. If payment has been completed but fulfillment has not, fulfillment is terminated, a refund of the payment is issued, and again the process ends in the state Order cancelled. If fulfillment completes without receiving a Cancellation message, the process ends in the state Order complete.

The semantics above are inherent in the definition of message start and boundary events. One thing to note: The state of the process is unknown to the Customer when he or she sends the Cancellation message. Thus a single Cancellation message must be represented in the diagram by separate message flows leading to different message events in the process, each event triggering a different way of handling the message. In the BPMN Method and Style training, we recommend that these message flows carry identical labels and the receiving message events do as well. In executable BPMN, these recommendations become absolute requirements.

When we make a model like this one executable, we need to think about messages from a new angle. An executable process is a service, and incoming messages are really API calls from an external client. On the Trisotech platform, for example, when we click Cloud Publish in a BPMN model, the tool compiles and deploys each process in the model – there could be more than one – as a REST service accessible through the Service Library. If the only interaction with the Customer were the initial Order, we might not draw the Customer pool and message flows at all. Instead we would make Order a process data input, which is equivalent to the start message. But when we have subsequent interactions, as in this model, we should use the message flow representation, because it implies that Cancellation messages addressed to this process are meant to cancel only this particular instance – this particular order – not all instances of the process.

Maybe this has got you thinking…

1. How do we address a message to a particular process?
2. How do we address a message to a particular instance of the process?
3. How does the instance know which message event receives the message?

All good questions, and we will answer them shortly.

The key difference between non-executable and executable BPMN is the latter defines process data in the form of data objects and defines mappings between process activities or events and these data objects. I have to tell you at the start that while all BPMN products generally follow the semantics defined in the spec, the implementation details of executable BPMN vary from product to product. What follows is how it works in the Trisotech Digital Enterprise Suite. I like this platform because it is designed for non-programmers, borrowing from DMN a business-friendly way of modeling and mapping data objects.

When we add the data objects to the diagram, it looks like this:

The diagram now looks cluttered and hard to read, but you can right-click and Hide the data objects, replacing them with three dots at the bottom of each activity mapped to or from the data object. The data objects with black arrows are data outputs, meaning their values are returned upon execution. In this case we simply want to capture in the output the Status of the order – “Received”, “Completed”, or “Cancelled” – and timestamps indicating precisely when the order was Received, Completed, or Cancelled, along with any Refund info. The reason we need to break these out as separate data outputs as opposed to components of a single structured output is that a data output association – that dotted arrow connector from a task or event to the data object – rewrites the entire data object; it does not simply update selected components.

So let’s see how to make this executable. We start with the datatypes, defined on the BPMN ribbon. The Order message is type tOrder, a structure that looks like this:

We assign the message flow Order to this datatype, and this associates the message flow with a message named Order, type tOrder. We do something similar to the message flows named Cancellation, using the type tCancellation.

Later we’ll see how the OrderID component of Cancellation is used to select a particular instance of the Order process.

We also assign the type tOrder to the message start event Receive order. With an incoming message, we always need to save the message payload to one or more data objects, here represented in the diagram as data associations to Order info (type tOrder) and Received, which captures the timestamp of receival. The data mapping from the start event is shown below. Received is populated with the current timestamp, using the Trisotech FEEL extension function now(), and Status is populated with the value “Received”. Order info passes order details to the other activities in the process.

Following the scenario described previously, we collect payment, fulfill the order, and complete. If a Cancellation message for this order is received while in the activity Collect payment, we set the Status to “Cancelled” and end the process in the state Order cancelled. If the Cancellation is received while in the activity Fulfill order, we again set the Status to “Cancelled”, calculate the amount to refund, and end in Order Cancelled. If Fulfill order completes with no Cancellation, we set the Status to “Completed” end in Order complete.

We model the Cancellation message flows in a similar way: Assign them and their catching boundary events to the type tCancellation and map the message content to Cancel info, Status, and Cancelled as we did with Order. For this example we ignore the details of the tasks Process payment and Fulfill order, except that completion of Fulfill order sets the Status and Completed timestamp, as shown by the output data mapping below.

Process Refund calculates the refunded amount. Here we are using simply Order info.Amount, but we made it a decision task to allow for more complex logic, such as a change in the price between time of ordering and cancellation. The data output Refund info reports the refunded amount and a timestamp.

Now we also need to provide a way to identify this particular instance of the order process. The primary method used by Trisotech is called instance tagging. At the process level you define one or more strings that uniquely identify the instance. These must be strings not structured data, so if you want to associate a value, such as the Order ID, with a label, you need to concatenate the label and the value. In the BPMN diagram, right-click Attributes/Process Instance Tags and define one or more label-value strings that identify a single process instance or possibly a small collection of instances:

Since each Order has a unique ID, this tag always identifies a single instance.

We want to tell the Receive cancellation events to accept only Cancellation messages for the current instance. Right-click the boundary event and select Message correlation. As you see below, Trisotech uses a three-stage filter for correlation, although typically one or more of the stages is not used.

The first stage, a message content filter, is a boolean expression of message data that makes no reference to the process instance. The second stage, instance tag matching, is the most common, and what we use in our example. It is a FEEL expression of type Text referencing attributes of the incoming message (here, Cancellation). Only strings matching a process instance tag are received, i.e., Cancellation messages in which the component OrderID matches Order info.ID. Because the instance tags are indexed, the runtime can do this correlation matching very quickly.

The third stage is a boolean expression of process instance data. It is very flexible but unless you have cut down the list of available messages using the first two filters, it could be slow. It should be used only in special cases.

So let’s now test our executable process. To do that, on the Execution ribbon click Cloud Publish, which compiles the logic and deploys it as a REST service accessible through the Service Library. There you see the various endpoints created for this service:

This answers the first question posed at the start of this post: How do we address a message to a particular process? It’s via the endpoint URL. In the modeling tool, my process has the name message flows5 executable, and this is version 1.0 of that process. So a POST to [base]/bpmn/api/run/test/bpmn/message-flows5-executable/1.0 is equivalent to sending the message Order to the start event. You can expand that box to see the required input data and the data returned when the process completes for various response codes:

As you can see, these are the json equivalents of the FEEL message definitions. Also note that POSTing an API call to [base]/trigger/test/bpmn/message-flows5-executable/1.0/default/Cancellation is equivalent to sending the Cancellation message to the boundary events. Here the name of both the process and the message event are part of the endpoint URL. Correlation to the particular instance of the process – this specific order – is achieved via the Process Instance Tag.

We can test execution of this process from the Service Library. The TryIt button exposes a form where you can input Order data.

Clicking Run advances the process to Process payment.

Here the Status is “Received” and the data output Received contains the timestamp of receival. The Continue service section lets you select either Process payment, meaning run this activity, or Receive cancellation, the boundary event. If we click Process payment, we see the data mapped to this task, and we can define any output data values. Clicking Run again advances to Fulfill order. Again the Continue service section lets us either run the task or receive the Cancellation message. This time let’s click Receive cancellation. We are prompted to define the Cancellation message:

Clicking Run now should exit on the exception flow, run the decision task, and end with Status “Cancelled”, end state “Order cancelled”, with a Cancellation timestamp and Refund info populated. You can see from the data outputs that it does exactly that.

The answer to the third question posed at the top – How does the instance know which event catches the message? – is simply based on timing. It can be received only by an active node at runtime.

Let’s recap. As you can see, executable BPMN is a bit more involved than normal non-executable Method and Style. Modeling real-world processes almost always involves incoming messages, which in executable BPMN are API calls to the deployed BPMN service. We model message data in the normal way, via FEEL datatypes, and use data mapping to save that data to data objects used in the process. At runtime, messages are routed to the right process and message event via the endpoint URL, and correlated to a particular instance via Process Instance Tags. We can test and debug the logic, including waiting for message events, via the Service Library. And best of all, none of this requires programming.

If this interests you – and it should! – contact Trisotech to find out how they can set you up for success.