To my surprise Why do Process Modeling Projects fail? made it quite high in the ranks. But if I think about it – yes – I understand the question very well.

I had my own share of failed projects, especially when I was novice in the area. And each failed project taught me something that I did not know before. After a while trying again and again I discovered stunning facts that were the foundation for succeeding success.

In my eyes this is absolutely natural in the area of knowledge work. And modeling is knowledge work. Therefore it is inevitable to try, to fail, to learn and finally to succeed.

If somebody promises: “I have the silverbullet method.” something is wrong. Either the task is too simple and therefore mainly irrelevant, or it is a lie. Business Processes are far from being simple, and if they are, then they are not worth modeling them, because they describe an area where only commodity business is done.

It looks so simple on the first glance. But it isn’t. For example take the other question: How to model one process with different variations at once? It is not so simple to answer this question. In my experience what is necessary to succeed in business process modeling is a deeper understanding about the laws of the business processes domain. Otherwise the result will be too complex to be used in practical terms. For example a car can be engineered, if the engineer understands the underlying laws of physics. Even if the car modeling tools look simple in the first place, they alone are not sufficient, nor are “methods” to model. Tools can help, methods can help. But they are not sufficient. Many training classes teach tools or methods. But to achieve a deeper understanding of the laws of the business processes domain in general, and the laws of the business processes from the special domain that I am modeling, they do not suffice.

The need for methods and tools is also expressed in the questions What are the 5 golden rules of process design? And Which business process management tool and notation suit my needs best? Given what I said I would answer this:

1. We need to find out the laws of business processes domain and the special domain that I am modeling.
2. We need to find 5 rules that respect these domain laws.
3. We need to identify the notation, that best fits to these domain laws and
4. We need to build or select a tool that supports this notation and is convenient to use.

Of course this has been tried many times. BPMN 2.0 and BPMN in general is just another attempt towards this goal. But still one of the most asked questions is: Why do Process Modeling Projects fail? I have my own opinion about the natural laws of business processes. I have modeled many hundreds Business Processes with 1000+ models from the areas of Customer Relationship Management, Supplier Relationship Management, Logistics, Supply Chain Planning, Financial Accounting, Project Management, Human Resources and others. I think BPMN respects some of the laws of business processes, but not all. For example Flexibility and Extensibility is not respected enough. It is not respected enough, that the result of a process is more important than the steps within the process, because the result is needed for another process, but the steps are irrelevant. Therefore there are too many variations of the processes, which are actually not needed, if the results of processes are modeled instead of the steps leading to the result. Another opinion about BPMN type of modeling I have is that most processes are over specified. This means the model is much less flexible than the reality.

In these days the Workflow Coalition will publish a book “Mastering the Unpredictable”. There we – process thought leaders from industry – argue that today’s process technology does not yet reflect the laws of unpredictable business processes and that there exist many of those.

Two recent Gartner key Predictions give insight in this very interesting and emerging part of process technology:

By 2012, 20 per cent of customer-facing processes will be knowledge-adaptable and assembled just in time to meet the demands and preferences of each customer, assisted by BPM technologies.

By 2013, dynamic BPM will be an imperative for companies seeking process efficiencies in increasingly chaotic environments.

So the whole process technology topic is still making very necessary progress. New domain specific languages will emerge, of which BPMN is but one. These new languages will respect the now known laws of the business process domain in a better way than today.

This all is nice and good. But what should I do today? Wait for a better future? Certainly not.

What are practical findings from using BPMN? is a very clear question and it is the most asked question. And in my opinion this is the best question. Because it is possible to use BPMN in a way, that respects the domain laws of business processes, even if they are not yet built into the notation. And this knowledge is exchanged by practical tips solving practical problems. Yes, there is a need to better understand the theory as the question What is new in BPMN 2.0? indicates. But even more there is the need for practical tips.

In my experience this can be achieved by coaching and governance. This is supported by the question What is required for BPMN coaching and governance? which is also quite high in the ranks. The answer is another question: Why you need a center of excellence BPM? The answer is: To provide the necessary coaching and governance. Of course there is much more to know about what is required for BPMN coaching and governance.

In some weeks I will start a coaching program. You are invited to it. You should have theoretical background already and you should have some practical experience already as well. The number of participants is limited to 7 for one year. It includes a Kick off meeting, weekly Web-Meetings as well as monthly meetings in a city somewhere in the middle of all participants. I addition to this coaching program I will offer training classes for the public as need arises. One training class will certainly address BPMN 2.0, another probably about how to organize coaching and governance inside your company- all with a very pragmatic approach. Of course the blog also will continue and I may publish another book or articles once in a while. I am looking forward to meet many of you in person who are reading this blog now for a while.

After that, I am sure we will engage in an interesting discussion about these. This can take any form that is required. It may be Blog Posts, Webinars, Network Meetings, Seminars, writing a book about it or a scientific article or pointing to events, where the topic is discussed.

I am really interested in the results.

Model Business Process

First of all the Business Process itself is modeled. The focus of this activity is to identify the steps that need to be performed to reach the goal and their dependency. This is done irrespective of the participants, which are not completely clear at this point in time.

Break into Participants

This is a design decision. Often Participants are companies of the business world. Still it is a design decision how fine or coarse granular they are designed and which part of the process is executed in which participant. For example it is possible to define different departments within a company as one or many participants.

Model Choreography

Now the interaction between participants can be modeled by means of the choreography model. This is a top-down approach. Therefore it defines the frame of subsequent detailed process modeling within the participants.

Model Public Processes / Services of Participants

As a next step the public processes and services of the participants can be modeled. It is decisive to model which services which participant exposes and which constraints exist between the service operations (e.g. an order needs to be confirmed, before it can be delivered).

If a complete system is designed from scratch, the public process models and the services are designed as To-Be processes and services. But in most instances the participants will at least in part already exist. Therefore As-Is models will be created and must be aligned with the To-Be model. This is the most challenging part of all. Because if the public process of a participant cannot be freely designed, because the cost to change it is too high, then the choreography must be changed, which in turn means, that the other participant may need to change. This is the most challenging negotiation in the design process.

Derive Business Objects

Before an implementation can start, Business Objects, which implement the behavior of the participants, need to be derived. The behavior of the Business Objects needs to be made consistent with the public behavior of the participant.

In the end the public behavior of the participant will be an abstract process, while the business objects will be concrete. This will also be very clear, when it is time to model correlation rules – i.e. which message instance is processed by which process instance. At this point in time, the most natural process instance will be the business object instance. Remember: The public process is only an abstract process – therefore it does not have instances.

Model Common Monitoring Process

Now the details have been modeled and we want to re-aggregate the execution of the processes into a common process view on instance level. The Monitoring Process View is needed as a re-simplification of the already modeled details. In the optimal case the monitoring process is equal to the original process model which stood at the beginning. Most probably there will be deviations for good reasons. This of course is also a good exercise to re-confirm the original requirements and justify the deviations from it.

This is my BPMN 2.0 Overview Map.

It shows a Choreography model in the middle, Orchestration with public Processes and private Processes, that belong to the public Processes.

Systems integration is the realization of the underlying business processes. This sounds so simple. But in reality there is often a misalignment between the business process design and the systems design. It is the wrong way to just implement business processes, that are as they are today or that are designed without discretion of the principles of loose coupling.

Often system designers need knowledge about how business processes work, but on the other hand business process designers need knowledge about how system integration works. As long as both sides are willing to learn and willing to share the knowledge, it is possible to come up with common principles of modeling, that avoid the most common mistakes, that lead to project cost overrun or failure.  I say it clearly what I mean: using BPMN 2.0 choreography modeling language in itself is no gurantee for success. But: It is a VERY useful tool for the communication between the business and system experts, which is a necessary condition of success.

So in other words, it is necessary, that the process design follows the principles of loose coupling of business processes. That is no design task, that can be solved by system designers alone, if the business process is modeled in a tightly coupled way. In other words: If the business process is designed in the right way – in the loosely coupled way – then the system design is without a hitch. If the business process is designed in the wrong way – there is no way to save the project on the system design level.

So are business experts forced to design business processes different, just to make the job of system designers easier? No. If it is designed that way, it is a better business process. It would work even better even if there was no system, but just paper and phone. It is more tolerant to errors. It gives the individual more freedom to decide. It makes it easier to reach the goal. Yes, it requires a little bit more brain power than just modeling the sunny day case. But in the end it pays off abundantly.

And in my eyes this is good news. It is NOT the system programming that dominates the design and dictates the constraints. It can be the business process needs again, that prescribe the way to go. And that is why BPMN 2.0 is so helpful, because it starts with the business process model.

So if in this example the incomplete model is validated against the meta model, then it fails, because the meta model requires at least two participants for the conversation node. However it makes completely sense to have the cardinality 2..* in the metamodel, because there is no meaningful Conversation without at least two participants. A possible solution would be to water down the metamodel. This is not discussed for BPMN 2.0, but may be for XPDL.

So in this watered down metamodel the participants cardinality is only *, allowing for 0 participants as well. The validadion succeeds, but the semantic suffers. Furthermore the check has to be done whatsoever at some stage. So watering down the metamodel is not a good idea in my view. Instead the solution should be a model lifecycle.

In this proposal the model itself has a lifecycle, which can be active or inactive. If the model is inactive, the validation reacts gracefully against the metamodel. Only if the model is active, the validation fails, but this is indeed intended. So it is unter the control of the modeler, if he wants the full strength of the validation or not.

Such a model lifecycle is not planned in BPMN 2.0 as far as I know, but I think it is a good idea and should be considered. There is a MOF 2.0 Facility and Object Lifecycle Specification that might be considered to be used in order not to invent something new that exists already.

In the BPMN 2.0 Spec there is a proposal, that builds on a generic OMG proposal, which is not finalized and which will be standardized independent of the BPMN 2.0 standardization.

In this proposal the idea is to have a Diagram Interchange Model independent from the Domain Model, which is in my opinion, a good idea. It is generic, so that other diagram types could be modeled. The diagram validation is done agains a so called Diagram Definition Model, which is also new. The Diagram Definition Model defines, which Attributes a Diagram Interchange node or connector can have, the allowed references and additional (OCL) constraints. Furthermore each Diagram Interchange node or connector is refers to a Domain Model class – in this case a BPMN Task or Gateway or Sequence Flow for example.

Advantages I see:

• Seperation of Model and View – Model maintenance
• Multiple Diagrams for the same underlying BPMN model are possible
• Generic format for different diagram types – tools can use synergies

Disadvantages I see:

• Tools vendors can not use XSD validation. They have to implement the validation against the Diagram Definition.
• The constraints of the Domain model (e.g. Sequence Flows can only be connected to …) must be repeated in the Diagram Definition Model.
• Grafical Attributes (e.g. line thickness) must be repeated as per modeling element.
• Attribute Format (name / value) too generic in my view.

The XPDL Format used to include the Graphics Info into the Elements of the Domain model.

This is simple and straight, and current tools support it, but has the disadvantage, that it is not possible to have one modeling element (e.g. one process) appear in different diagrams. Instead, the modeling elements must be repeated as per diagram, which has many disadvantages in model execution, analysis and model driven development. The Signavio modeler, which was discussed at the meeting, follows the same approach. However it is good as long as only diagrams are drawn for visualization purposes.

Because the BPMN Digram Exchange Proposal was too generic for some, Bruce Silver proposed an BpmnDI.XSD for a concrete XSD validation.

This approach repeats basically all or many BPMN domain modeling elements. This of course is the disadvantage, because it is but a redundant repetition of the already defined domain model. However it is no complete repetition, because the gateway type for example is not included. So a tool must look into the domain model anyway to render the gateway.

Within this discussion it came to my mind, how the GMF (Graphical Modeling Framework) solves this question. I think it is well worth to have a look at this approach, because I think it may be a good compromise.

The GMF defines a Grafical Definition Model, which only defines Shapes. This is simple to understand and straight. Then it defines a Mapping Model. Within the mapping models, nodes and connections are grouped as needed. For example it defines, that there is a node lane which can contain node Activity and Sequence Flow. As a followon the Acitivty node is connected to the Activity domain class and the Shape, that descibes how an activity should look like. The shape can be re-used, which is an advantage of the BPMN Diagram Exchange Approach. The Structures of the Diagram is already contained in the Mapping Model, so it can be serialized to XMI and XSD, which is also an advantage. However still the diagram logic is separated from the Domain Model, which is an advantage over XPDL. And it is a very generic model, which is an advantage over a mere BpmnDI.xsd.

I personally worked with the GMF, and found it quite practical. I am not proposing to use GMF per se, but maybe the approach should be further considered.

I must admit, I was a little bit surprised about the result. For me BPMN is not flexible enough and it has too many modeling elements. While this is not the majority opinion, a grave minority has the same opinion.

Some comments:

I think BPMN is great – but we just had to add on a few bits to define value add and non value add activities so that larger maps had more to narrow your focus on areas that need attention

Alan Crean, CEO @ Process Master

I agree that this is necessary to not overload the modeler with too many modeling elements.

Flexibility is a vague term that can mean many things in different situations. I would define flexibility as the ability to make changes over time as needed to respond to changes in the situation. In order to be flexible, there have to be constructs that afford the kinds of changes you will need. For instance, flexibility implies some form of control. A common pattern might be that parts of the process diagram are controlled by different people. There is no consideration in BPMN for specifying who has control over different parts. Instead, there is a built in assumption that the entire diagram is designed and maintained as a whole. That reduces flexibility.

Different people controlling different parts of the process diagram is definitely a very important part of the flexibility requirements I see. There is no concept in BPMN for this. It may well be, that it is not possible to add this concept later, because of the current semantic of BPMN.

I consider BPMN still more as a (high level) programming language for automated processes which provides rather poor support for flexible human processes.

I agree.

Just one examples of too less flexibility: Consider a process where one task can be executed by different roles (e.g. via delegation). In which Lane should I place the task? Should I show it in the “Default”-Lane? Or a copy in any other related Lane? Or even draw it across multiple Lanes?

While I know that Lanes are just a graphical partition of the process and we could easily address the problem via performers (BPMN 2.0), this is not a graphical solution. Many users (and tools too) use Lanes synonymous to performers.

Interesting case.

I guess it depends on the tool you use. We have created a new tool generation which hides the complexity of the BPMN notation against the user by using intelligent algorithms which support a highly efficient modeling process.
Our BETA was anounced on Nov 10. Innovator for Business analysts will be available shortly. C. Bergner.

It is worth trying. However my doubts are that certain limitations in the metamodel can not be overcome by a tool. However a tool can improve.

We feel the technology driven approach of BPMN when it comes to high level process landscape description. This seems to be out of scope at least in the BPMN 1.x standard.BPMN is good for visualizing low level processes that ar candidates for workflow implementations.

I agree.

It’s really interesting to see your opinion. In my opinion BPMN has a workflow modeling background. That’s why it is good for workflow type of processes. However there are a lot more of other processes.

Yes, of course we can discuss about business rules and automated decisions. But in the end, the most intelligent entity on earth is still the user. Yes, we can discuss about AI and Deep Blue being Chess Champion. But my statement still holds.

Yes, it is all about decisions. But that does not imply, that the decisions need to be automated. Some can. But often the business process should push the necessary decision to the user.

And to allow for intelligent decisions, it is crucial to allow for enough decision alternatives. This may sound simplistic in the beginning. But it is not. Bare with me for a moment.

So what is as simple as an approval? Approve or Reject. The simplest decision in the world. But not so simple. Of course, if only Approve or Reject is modeled / implemented, then what should the user do, if he wants to do something else? What else could a user do other than Approve or Reject? He might want to send it back to revision for example. Another user might want to change the underlying approval object – what influence does this have on the process? A realignment of the process is necessary. What, if an approver wants to merge several approvals into one or if he wants to split one into several. Partly Approve. Partly Reject. Partly Edit. Define new Followup – Approval. Change the Approval Type. Merge unterlying objects. Split them.  And Realing the Process. Or substitute the Business Process by an equivalent different one. If you only begin to think about it, then you can hardly stop. And if you think, what that means for a process model, then I would say this is a real challenge. Certainly far more than giving only two branches: Approve and Reject.

So in my opinion adding intelligence to business process means also adding flexibility to the business process, that allows for more than simple alternatives for the user and to push decisions to the user. And to allow for automated decisions to be overruled by the user in many cases.

If there are many process instances in a system, it is also a lot of work to realign them, if necessary.

What do I mean by realignment of process instances?

For example, if there is a process instance – which is a simple example – that needs a one step approval in case of an order of below 1000€ and a two step approval in case of an order of 1000€ or above.

Now if there is an order of 900€ then there exists a process instance for a one step approval. If later the order is changed to have a value of 1500€, then the process instance needs to be realigned to have two steps of the approval.

The realignment can typically not be done from within the process instance itself.There is some logic needed, that does such a realignment and works in all cases. Furthermore as of my knowledge there is no generic algorithm to describe all needed realignments based on such changes. So the rule do assign one and two step approval to the values of below and above 1000€ is not enough. It is further necessary to describe the realignment logic.

This is especially challenging as it depends on the state of the process instance itself.For example if the one step approval is the manager approval and the two step approval is the revision department approval which has to take place before the manager approval. Then it may challenging to realign a process instance in the case of a value change, if the manager approval is already in progress, because a different step has to be inserted before.

So the process instance realignment problem is not simple, but a solution is needed, as soon as there are process instances in the system.