Sandy Kemsley’s Vlog - Business automation best practices #1 - Introduction
Sandy Kemsley Photo
Vlog

Business automation best practices

#1 – Introduction

By Sandy Kemsley

Video Time: 4 Minutes

Hi, I’m Sandy Kemsley of column2.com. I’m here for the Trisotech blog with the first in a series on best practices in business automation application development.

I want to set the stage for this series by talking today about what I call the automation imperative. We have a lot of cool technology now, that can be applied to automating business processes and decisions:

Now, all this technology makes more automation possible. These newer technologies let us automate things faster and better, and in fact automate things that were never possible before. But this has a downside. Yours isn’t the only organization with access to these technologies and everyone else is trying to automate their businesses using the same methods and the same technologies. That means whatever market you’re in it’s become more competitive. Since that rising tide of new technology is floating all the boats, not just yours. More nimble competitors will force you to automate in order to compete, or you’ll die in the marketplace. This is the automation imperative. It’s essential that you leverage automation in your business, or you just won’t be able to survive.

So easy right? Just plug in some of these new technologies and off you go? Sadly not that easy. A study done by Boston Consulting Group, showed that 70% of “digital transformation projects” don’t meet their targets. 70! 7-0, that’s a big number. Well okay, digital transformation is one of those loose terms that gets applied to pretty much any IT project these days, so let’s focus that down a little bit. Ernst & Young looked at RPA projects. Now, robotic process automation vendors claim that you’ll have your return on investment before they even get driven all the way out of your parking lot.

Now, what E&Y found though, is that 30 to 50 percent of initial RPA projects fail. So, if organizations aren’t succeeding with a technology that’s supposed to be the most risk-free way to improve your business through automation, then there might be some problems with the technology, but probably, someone is doing something wrong with how they’re implementing these projects as well. We have billions of dollars being spent on technology projects that fail. A lot of companies are failing at business automation. You don’t have to be one of them.

Now, this series of videos is focusing on best practices, that will help you to maximize your chances of success in application development for business automation. Or if you want to look at it in a more negative light, they will minimize your chances of failure.

I’m going to split the series into three parts:

So we’re going to be covering quite a bit of territory and looking at a number of things.

Now, in each of these videos, I’m going to discuss a couple of things you should be doing. So those best practices and also some of the indicators that you might be failing. So some of these anti-patterns that you can be looking for as you were going through your projects to figure out if something’s going off the rails maybe before you have a serious failure. So stay tuned for the coming videos in this series.

That’s it for today. You can find more of my writing and videos on the Trisotech blog or on my own blog at column2.com. See you next time.

Blog Articles

Sandy Kemsley

View all

All Blog Articles

Read our experts’ll blog

View all

Bruce Silver's blog post - DMN's Killer App
Bruce Silver
Blog

DMN’s Killer App

By Bruce Silver

Read Time: 7 Minutes

Presented at Decision Camp 2022

Watch the video
View the Slides

As a longtime DMN practitioner and from the beginning one of its biggest boosters, I am what you call a true believer. But while DMN continues to gain traction, even I would have to admit it has so far underperformed in the marketplace. The industry analysts say, “OK, we’re aware of it… But what’s the killer app?” It may not be what you think.

In the Beginning

Let’s go back to the beginning. In 2016, DMN was brand new. I had been posting about it, and I was invited to introduce it to the Decision Camp audience of decision management vendors and practitioners that year. As standards go, DMN was unusual: an executable modeling language designed for use by subject matter experts in a way that standards like BPMN were not. Executable logic would be defined graphically, using a set of standard tabular formats called boxed expressions, with a new business-friendly expression language called FEEL used in the table cells: Executable models without programming.

At that time, DMN was just a spec. There were no runtime implementations. But I also noticed that among tool vendors on the DMN 1.1 Task Force, which I had recently joined, there was no great urgency to implement it, and in my talk I complained about that. Of course, it was the things that made FEEL and boxed expressions business friendly that also made them difficult to implement: spaces in variable names, for example, and tables nested, potentially without limit, inside other tables.

But at that meeting, Mark Proctor of Red Hat told me, “Dont worry, my guy will have a DMN runtime in 6 weeks.” It actually took his guy, Edson Tirelli, more than 6 months, but it worked very well, and Red Hat made the FEEL parser and DMN runtime open source. So by 2017 we had an open source runtime, and there was little excuse for tool vendors not to fully implement DMN.

A few did, but most Decision Management vendors continued to resist FEEL and boxed expressions. In fact, they questioned the original premise that business users really wanted to go beyond defining requirements handed off to programmers. Or, if they did, that they had any interest in basing that on the DMN standard even though those had been central to the original DMN RFP.One vendor that did believe in the original premise, as I did, was Trisotech, and I soon moved my DMN training to their platform, which was aimed squarely at non-programmers.

Lets fast forward to today, 6 years later. The DMN standard continues to get better, every year a new version. But still, few tool vendors have implemented it beyond DRDs and decision tables. Interest in my DMN training has increased, but it is only now catching up to my BPMN training, which is focused on descriptive, that is non-executable, models. Its a little discouraging.Yet despite all that, I am more hopeful today than ever about the promise of DMNs key features FEEL and boxed expressions. And thats because they are the keystones of what I believe is a far larger opportunity.

What You Draw Is What You Execute

Back in 2010, the process modeling standard BPMN 2.0 promised business users, “What You Draw Is What You Execute.” I used to say that myself.

But not really. What makes a process model executable are the services implementing the tasks, and the process data and expressions used to orchestrate them. But for those things, BPMN has always required programming. That standard provides nothing like FEEL and boxed expressions for non-programmers. While the activity flow semantics can be defined by business users in diagrams, making those diagrams executable has always required Java programming. So diagrams created by business and handed off to programmers for implementation has been standard practice in Process Automation for a dozen years now.

A few years later, DMN set out to be different from that. If you are unfamiliar with OMG standards, they start with an RFP that lays out the objectives of the standard. And DMN was distinctly different from BPMN in that it wanted to enable business users to do more than create requirements handed off to programmers. To that end, the DMN standard itself defines a graphical Low-Code language for decision logic implementation: a set of standard tabular formats called boxed expressions and a friendly expression language for the table cells called FEEL. Although DMN allows tools to assert so-called Level 1 conformance without supporting FEEL and boxed expressions, FEEL and boxed expressions comprise the bulk of the DMN spec: the syntax, semantics, operational behavior and graphical formats.

Unfortunately, most decision management vendors especially those who formerly called themselves business rule engine vendors failed to adopt FEEL and boxed expressions, preferring the traditional arrangement in which business creates requirements handed off to IT for implementation in some proprietary language, and they continue to do so today although in so doing, they ignore the original promise of DMN.

Its important to understand that decision logic is more than decision tables. For example, it must refine raw input data into the variables that are used in the decision tables. FEEL and boxed expressions not only do that, but represent a complete Low-Code language suitable for any kind of business logic, not just that related to operational decisions. Even though FEEL and boxed expressions are defined within the DMN spec, their utility extends far beyond the boundaries of traditional decision management.

For example, over the past two years, the Trisotech platform has quietly been adding Automation capabilities to its BPMN models in a different way, a Low-Code way, leveraging FEEL and boxed expressions borrowed from DMN. By now that Business Automation platform is quite complete. So this goes beyond business users creating decision services. This is enabling them to create full-fledged Business Automation services themselves, graphically, so that What You Draw actually IS What You Execute.

Low-Code Business Automation

I use the term Business Automation to mean the combination of business logic automation and process automation, and DMN combined with BPMN offers a Low-Code approach to it based on standards.. If Subject Matter Experts learn to create decision models using DMN with FEEL and boxed expressions, its just a small additional step to do Low-Code Business Automation themselves: model, test, and cloud deploy as a REST service no programming required.

Low-Code means model-based software development, accessible to technically inclined business users, as well as to developers. The fact is, interest in Low-Code Business Automation is exploding; Gartner says 65% of all solution development will be Low-Code by 2024.Thats an addressable market larger by orders of magnitude than Decision Management. The key benefits: time to value model-driven development is simply faster than code – and avoiding the IT resource bottleneck, since subject matter experts can do much of it themselves.

That is why I see these DMN features FEEL and boxed expressions, which have frankly languished in the world of Decision Management – as the key to something new and exciting today, an opportunity that is much larger than Decision Management. Actually Low-Code Business Automation is DMNs killer app.

Why is this opportunity happening now? The fact is, over the past six years, the software world has changed completely:

Reason 1
the cloud

The cloud has revolutionized software, both tools and runtime. Large enterprises have largely overcome their resistance to putting critical data in the cloud. In addition to its well-known benefits to IT, the cloud makes Business Automation tools and runtime directly accessible to business users; they dont need to wait for IT to provision or maintain the tools or the runtime. The Trisotech platform, for example, is entirely cloud-based and business-user-focused. In one click BPMN and DMN models are compiled and deployed as REST services in the cloud.

Reason 2
REST APIs

Today you can find a public REST API for just about anything you need. You dont have to build it, someone has already done it. Its there, accessible to you via a simple web message. Aggregators like RapidAPI.com provide a centralized search, subscription, and SDK for thousands of public REST APIs. In addition, standards like OData allow you to generate REST APIs for your own applications and databases. For example, Skyvia Connect generates OData endpoints for over 100 apps and databases. Each endpoint exposes 5 REST operations per database table.And on the Trisotech platform, DMN itself generates a REST API for ANY business logic you create yourself, and BPMN similarly generates REST APIs for your executable process logic.So today REST APIs are everywhere. Business Automation is really just orchestration of services, some created by you, but mostly created by others.

Reason 3
Covid and its aftermath, the current labor shortage

It is nearly impossible today to hire skilled programming resources. So getting new projects off the ground just takes too long.But what if you could teach subject matter experts to create solutions themselves? Thats what the current excitement over Low-Code and No-Code tools is about. No-Code is used primarily for situational apps in HR and CRM. Im more interested in Low-Code logic equivalent to a DMN decision model that in combination with the cloud and REST APIs can automate critical core business activities in financial services and healthcare.

Example:
Vacation Request Process

Let’s see how Low-Code Business Automation with DMN and BPMN works, and then go back to key questions that have dogged Decision Management vendors from the start: Do business users really want to do this? And does basing the tools on standards make a difference?

Here is an example of a Vacation Request process as it would be modeled using my non-executable BPMN methodology called Method and Style. Not a core business process but illustrative. Its very simple. A Vacation Request is processed by decision logic that either automatically approves it, refuses it, or refers it to the manager. If ultimately approved, the requested days are deducted from the employees accrued vacation time.

And here is the executable version of it using Low-Code Business Automation: The BPMN is similar but the activities now are more fine-grained, the icons in the top left corner indicating specific capabilities of each task type. But the most obvious difference is the model now includes process data and data flow.

Decision Tasks

A BPMN decision task is bound to a DMN decision service.

Here you see the data input validation decision. DMN is actually an excellent language for data validation. A Collect decision table, hit policy C, produces a list of error messages. Data validation logic typically uses some of the lesser known FEEL functions and operators: generalized unary tests and the instance of operator, as you see here, also the get entries function, which lets you test all components of a structure at once, and match, for pattern matching with regular expressions.

Just to repeat, DMN is not only about decision tables. Its a Low-Code language for any business logic. For example, the Vacation Request specifies the employees requested start and return date, but the approval logic requires knowing the count of requested days. FEEL has excellent support for calendar arithmetic, but finding the count of requested days is a little complicated. We cant simply subtract the dates; we need to exclude weekends and holidays as well.

This boxed expression is called a context. It lets you break up a complex value expression into pieces using local variables something most other Low-Code languages cannot do. For example, here there are 5 context entries, each defining a local variable and its value expression, and the final result expression references those local variables. Note that the fourth context entry uses a decision table, an example of a table nested in another table. Context boxed expressions let you make complex logic business friendly without cluttering up the DRD with dozens of tiny decisions.

In the DMN model, a decision service defines the parameters called by the BPMN decision task. The decision service outputs are selected decision nodes in the DRD, and the tool calculates the required service inputs. Here the service Validate Vacation Request has input parameters Vacation Request and Original Employee Record, and output parameters Validation Errors and Employee Record.

In BPMN the Decision task Validate Vacation Request is bound to that decision service. The task inputs are by definition the decision service inputs, and the task outputs are the service outputs.In the process model, the dotted arrows, called data associations, represent mappings between the process variables, here called Vacation Request and Employee Record, and the task inputs and outputs. So from the diagram you see there are input mappings from Vacation Request and Employee Record and an output mapping to Validation Errors.

Since all data is represented as FEEL in the modeling environment, the mappings use boxed expressions and FEEL, similar to a boxed invocation in DMN. Here the mappings are essentially identity mappings, but any FEEL expression can be used.

Service Tasks

An activity with the gear icon, called a service task, calls an external REST service. Simply bind it to a REST operation in the models Operation Library and map process variables to the service inputs and outputs.

EmployeeVacationData is a MySQL database table on my website. It holds the employees accrued vacation time.A third party service, Skyvia Connect, introspects the database and generates a REST endpoint for it and a metadata file used to configure a Trisotech connector, exposing 5 database operations on the table using OData. We will use the Find, or query operation, to retrieve the record for the requesting employee.

We bind the service task Get Employee Data to the Find, or query operation of this endpoint in the Operation Library.Then we model mappings between process variables and the service input and output parameters using FEEL and boxed expressions. Here the data mapping for the input parameter $filter uses a FEEL expression to create a query string in the OData language.

Once were done, the whole Vacation Request process is itself deployed as a REST service in the Trisotech cloud. This Business Automation service can be called by any REST client app with the appropriate credentials.

Hopefully this gives you the flavor of how subject matter experts can create Business Automation services using FEEL and boxed expressions without programming!

Where Can We Use This?

What kind of Business Automation services can make use of this? Currently the Trisotech platform is used most heavily in integration-centric processes in healthcare and financial services. Digital mortgage underwriting is a great example, as there are many complex rules. You might say thats a straight decision management problem, but its really a process. You need to first get the data from various sources, validate the data, transform it to testable values, apply some decision logic, and finally format and store the output. Low-Code Business Automation can do all that in a single composite service.

I am currently working with a large accounting firm to create general ledger entries triggered by business events related to financial portfolio assets and trades. Its high-volume, involves many database tables, a lot of computation perfect for DMN and BPMN.With Low-Code, solutions like these are much faster to build, test, and deploy than with Java, and easier to maintain.

So who can create apps like these? I admit its not every business user or subject matter expert. Most of my BPMN Method and Style students probably could not do it very well. But anyone who can create executable decision models using FEEL and boxed expressions certainly can. You need to have some basic facility with data and expressions, and the patience to debug when things dont work right the first time. Thats just a subset of business users, but in most companies, subject matter experts who can do that still greatly outnumber programming staff. Microsoft used to call them citizen developers; now they are calling them software makers, a term I like better.

To turn subject matter experts into makers, I had to completely revise my DMN training. Originally I had a course DMN Basics that focused on DRDs and decision tables, since thats all most DMN tools supported and Its something every business user can understand. The problem is you cannot do anything useful with only that except to provide requirements handed off to others. In the revised training I focus on FEEL and boxed expressions. Those are the critical elements. And yes, subject matter experts who are not programmers can learn to understand and use them well. You just have to show them how. They feel empowered, because they ARE empowered. Once you fully understand DMN, Low-Code Business Automation is mostly a matter of learning how to find and orchestrate REST APIs.

DMN vs Proprietary Low-Code Languages

OK, youre saying there are already a large number of Low-Code Business Automation tools available. Thats true. Basically all of them use their own proprietary scripting for the Low-Code piece, just as many Decision Management vendors use their own proprietary rule languages, which they also say are business-friendly.

But FEEL and boxed expressions are not only non-proprietary, they are actually more business friendly and often more powerful. For example, FEEL and boxed expressions make DMN more powerful and more business-friendly than Power FX, Microsofts Low-Code expression language used in Excel formulas and Microsoft PowerApps. Without boxed expressions like contexts, Power FX must pack complex logic into a single literal expression. And Power FXs lack of infix operators makes these expressions deeply nested functions that are difficult to understand.

For comparison, here is an example published by Microsoft marketing showing how Power FX can extract the last word in a string. They were very proud of this So, lets see, Is that 4 or 5 right parens at the end?Below it is the equivalent in FEEL: The split operator tokenizes the input string based on the space character and [-1] means take the last item. Now you tell me, which is more powerful AND more business friendly?

So why are DMN users not demanding their tool vendors support FEEL and boxed expressions?

1

Its mostly lack of awareness. Most of what other folks write about DMN just talks about DRDs and decision tables. Possibly even many of you here at Decision Camp were not aware of what FEEL and boxed expressions can do.

2

Its also the need for training. You need to learn the language. Developers can generally learn from books and articles, but business users by and large cannot. They need training, a methodology that leads them through it step by step, and it must be hands-on using the tools. Thats an investment in money and time, definitely a barrier.

3

As everyone here knows, Automation is unforgiving. If the slightest thing in your model is wrong you will get an incorrect output or worse, a runtime error. So Automation requires debugging, and not all business users have the patience or discipline for that.

4

But a major reason also, probably the most important reason, is the determined resistance from Decision Management vendors. Even though DMN is a standard, most vendors on the RTF still do not support it beyond DRDs and decision tables. FEEL is too hard, they say. Business users dont want to create implementations, just requirements. They say it all the time. FEEL and boxed expressions are a distraction, not fundamental to DMN. We even heard this last year at Decision Camp.

The Larger Opportunity

That lack of support from incumbent vendors is why I have come to doubt that Decision Management is the best opportunity for DMN. A better opportunity is Low-Code Business Automation.

And this takes us all the way back to the original question:Do subject matter experts really want to create executable solutions themselves? Not all, obviously, but an increasing number of them do, and their employers today desperately need to acquire low-code tools and expand their pool of software makers. And even for professional developers, Low-Code is faster, easier to maintain, and more transparent to the business.

That is why I am becoming increasingly optimistic about engaging the Business Automation community with Low-Code Business Automation using BPMN and DMN. Its a far larger addressable market than Decision Management, its growing, and its perfectly matched to todays architecture based on the cloud and REST APIs.

But we know there are other Low-Code alternatives. So do they want to do it using standards, i.e. DMN in combination with BPMN? That is DMNs most obvious differentiator right now vs other Low-Code languages. Here are two reasons why they would.

So ideally,
what should happen now?

What started as an effort to enable subject matter experts to create decision logic themselves, while so far resisted within the digital decisioning community, could ultimately find a better path forward in Low-Code Business Automation.

For anyone interested in the tools and techniques used in this approach, there is a lot of information on methodandstyle.com. I offer online training in DMN, focused on FEEL and boxed expressions, and in Low-Code Business Automation, and there is a bundled offering at a discount. Each course includes 60-day use of the Trisotech platform and post-class certification.

Follow Bruce Silver on Method & Style.

Blog Articles

Bruce Silver

View all

All Blog Articles

Read our experts’ blog

View all