Jean-Jacques goes on to explain, that classical BPM has neglected the state transitions of resources and only focused on activities. He goes on to explain that there is a relationship between state transitions of resources and activities and that this will bring a new level of enlightenment to the BPM community. I can’t agree more. I hope that we can do some work about this.

However he argues that therefore processes are predictable. That is not true, because in creative processes it is not know a priori which resources (Business entities – for example research reports, experiments, designs, models, customer quotes, …) will result from the creative process.

The discussion goes on with some definitions and a medical example. Then JJ says:

It is like saying, to start a journey, a path must be chosen. All you need is a map, maybe even a compass would do.

I really like this comparison. The activity model is the path, the resource / business object composite state model is the map. I agree that the area of using business object composite state models has not been used in BPM as it should have been used. Therefore maybe JJ is more fighting for that idea. That I very strongly support as well. But what if you neither have a path nor a map? These situations happen very often.

To me it is not enough to assume a “could be” map or path. As a vivid example I have my company startup. Of course there are predefined process snippets for this and that (for example for the tax process – certainly for the tax process!!!). But in the end the tax depends on decisions I do – which legal form I choose, which depends on other considerations. Product and service definition, strategy, financing, partners, market strategy – the special combination of all of that are all unique to my new company (I hope!) and therefore the activities leading to the needed results are unique. And they depend on many decisions that I cannot predict as well. Of course if you do abstraction, then it is the same as with all startups: Write a business plan, Create a product or service, sell it – but I have argued before the abstraction leads to a useless plan. So what I am actually doing is to merge specific process snippets – some predefined like tax – some newly invented by me (product and service innovation) – into one big workstream that is unique, concrete and unpredictable.

To be continued…

Jean-Jacques

To be clear, the answer is “no” processes are not “unpredictable”, they appear “unpredictable” because you are not relating the activities of the processes to the lifecycles of the underlying business objects. Business entities can have a very complex lifecycle, even made of composite states (the entity is in more than one state at a time). Activities are performed to transition business entities from one state to another.

I have seen this statement very seldom and I can’t agree more to the observations about business objects and state. I have modeled (governed the modeling process of) 400 business objects of a whole ERP suite with composite states and yes, it works! In my opinion it is the best way to describe the business process that is encoded in business objects.

I do not think like Jacque, that it is necessary to relate unpredictable processes to these business objects. The idea is that we are addressing the complementary set of business processes with the concept of ACM. However I agree that there is big value in thinking about how to connect unpredictable processes to processes of business objects. This results in flexibility in how to achieve the goals, but in the end the result is reflected in standard processes. I have not seen this said so clearly before by anyone other than me.

BPMN is useful, it can help document the “happy” path, the most commonly taken path,… but rarely it can reflect all the possibilities that can arise if you associate activities to transitions between states of business entities.

Very good observation! At least this is the standard use of BPMN – the way most people use it. But BPMN can still be used to express all the possibilities in a manageable way. The knowledge of how to do this is not yet commodity. But I am looking forward to this discussion.

Jean-Jacques mentions the example of a project business object, that transitions through predefined states. By this, the process is not unpredictable, he says, because the states are predictable. But – the workitems of the project are not predictable. For some projects, they are. For others, they are not. For typical knowledge worker projects, they are not. I am not saying “Project” for knowledge work, but I rather prefer the term “Workstream”.

Keith argues that we are not omniscient, and therefore some processes are unpredictable. I agree. I will try to avoid a metaphysical discussion about the state of the universe and the unpredictability of the decisions in human minds. By all practical means we can agree that it is not possible to look into people’s mind and find out how they will decide. This is neither necessary nor desirable.

Take a chess game for example. It is possible to enumerate the alternative moves, that one player can choose. We know in advance, that it will be one of the moves. But we don’t know which one in advance. We might enumerate all possible responses to each of the possible moves, but where does this lead? It will lead into an explosion of alternatives. Yes, I know there are very good chess computers. But chess is a relatively limited and well formalized decision space. Still it becomes incredibly complex the more moves are taken into consideration. In sum none of the chess games is predictable. If it were, there is no point in playing it. The winner would be known from the beginning.

If we raise the abstraction level, the chess game becomes perfectly predictable. The chess game is either “Not started”, “Started” or “Finished”. The result may be “Winner Player 1″, “Winner Player 2″ or “stalemate”. This is perfectly predictable. But that’s only because the abstraction level has been raised. So Jean-Jacques is right by saying that all processes are predictable – and I add: if the abstraction level is raised so much, that the prediction is useless.

But in practice what we need to do to complete our work is to define goals, next steps, be concrete instead of abstract. Defining goals, next steps and responsibility is creative work and creative work is not predictable. Probably creative work is the most unpredictable process in the universe. How should anyone predict the result of a creative process? How should anyone predict if a creative process results in 1, 3 or 10 alternatives and how they look like? This is blatantly unpredictable. How could anybody have predicted the text that I am writing right now? No – that was a deliberate decision and a creative process. The only thing that was predictable was that it is a chain of characters from a predefined set. But I hope that is not all, that you take with you.

I want to comment more on the discussion, but next time.

AUTHORS OF ‘MASTERING THE UNPREDICTABLE’ ANSWER QUESTIONS, SHARE BEST PRACTICES ON MANAGING UNSTRUCTURED PROCESSES

Thursday, July 15 • Moderated by Connie Moore of Forrester Research

WHAT:

Connie Moore of Forrester Research http://www.forrester.com/rb/analyst/connie_moore along with authors of the newly published book Mastering the Unpredictable
http://www.masteringtheunpredictable.com/ will host a Tweet Jam to answer questions about the top challenges facing business and IT practitioners in managing the unpredictable, less structured business processes that remain major headaches for IT organizations – and how Adaptive Case Management (ACM) http://www.xpdl.org/nugen/p/adaptive-case-management/public.htm can help solve them.

According to Nathaniel Palmer, editor-in-chief of BPM.com and executive director of the Workflow Management Coalition (WfMC), “Right now, case management is the next new big thing in process technology and it’s critically important that we take a deep, informed look to make sure we understand how to apply it to our business-transformation efforts.”

Contributors to Mastering the Unpredictable include Palmer and leading experts in the field of business process management (BPM): Longtime WfMC associate Keith Swenson, as well as industry thought leaders Henk de Man, David Hollingsworth, Dana Khoyi, Frank Michael Kraft, Caffrey Lee, John T. Matthias, Dermot McCauley, Max J. Pucher, Tom Shepherd and Jacob Ukelson.

WHEN:    Thursday, July 15, 12:00 pm ET

WHERE: Online on Twitter at #acmjam hash tag or follow at Connie Moore (@cmooreforrester). You can also tune in to coverage of the Tweet Jam at:

http://www.masteringtheunpredictable.com/

http://www.wfmc.org/

http://bpm.com

TWEET JAM DETAILS:

Among the key topics to be discussed:

- What are the similarities, differences and key trends for ACM vs. Business Process Management (BPM)?

- How do I know if I need case management?

- Who in an organization should care about ACM? Why?

- What are some specific examples of knowledge work that ACM supports? 

- What is the primary benefit that a knowledge worker/case manager gets by using ACM? How about a manager?

- Is there such a thing as “Social BPM” or “Social Case Management”?  What does that mean to you?

- How do you measure success in an ACM implementation?

- What are some best practices for getting started with ACM?

For executives and managers of knowledge workers, Mastering the Unpredictable will:
- Explain the need and why previous technological approaches don’t meet the need
- Explain the current technology gap, and the new technology that can close the gap
- Lay out the options that can increase the efficiency and effectiveness of their organizations
- Equip them to best take advantage of this evolving trend

Ten books will be given to the most active and relevant participants in the discussion.

FOR MORE INFORMATION:

Please visit http://www.masteringtheunpredictable.com/ for additional information about the Tweet Jam, the book and the authors.

“Mastering the Unpredictable: How Adaptive Case Management Will Revolutionize the Way That Knowledge Workers Get Things Done” is available now at amazon.com (including for the Kindle) and Publisher’s Warehouse.

All knowledge workers, that I know, and I know a lot of them, don’t organize their knowledge work by using project planning techniques. And that has a reason. It is too clumsy. Changes are too rapid. It is not worth the effort. So certainly the solution to the Math of the Missed Deadline problem is not to use project planning, neither deterministic nor stochastic project planning.

But the Math of the Missed Deadline shows where the key is. The key is in coordination.

I am sure many of you knew that before already. But it is interesting to see which tools are used for coordination. The mostly used coordination Tool is Microsoft Excel, I propose. Why? It is used, because it is so easy to use.

But I must admit I was not satisfied. An Excel of 3.500 entries does not make sense any more. It is hard to structure them. There are so many Excels. One here, one there – for every project a different one. Don’t miss a deadline assigned to you in any number of Excels distributed on many file shares! That is a challenge.

I want to have it cloud based – accessible from everywhere. I want to calculate the remaining effort. Still, the maintenance must be very easy to use. I want a quick search.

I don’t want to explain the whole functionality here and now, because this is reserved for prospects. If you are interested in considering the system for use send me an email: frank.michael.kraft[at]bpmnforum.net

The only statement I want to make today is: Even if the problem “Math of the Missed Deadline” sounds quite complicated, the solution is very simple and easy to use.

I am not using Excel any more. Not for the purpose of coordinating Workstreams.

Maybe you think that working harder would solve the problem. But all the time you are struggling against a mathematical law.

Say we have a case that has to be broken down into workitems. These workitems have dependencies. Some have, some have not. So it is a network of workitems, or mathematically spoken a directed acyclic graph. You can think about it like a BPMN process without loops. Another mathematical term is lattice. Now we can estimate the duration of each workitem and by using standard project planning techniques we can calculate the planned end time of the project, buffer times of the workitems and the critical path – i.e. the workitems that have a buffer time of zero. You surely have heard of it.

When I was in the seminary learning the technique I said to the teacher: I have doubts about this technique. It is well known that most projects don’t follow this plan, but are late. So there must be something wrong with it. Why not plan more buffer time from the beginning? The teacher said I am not in the position to question such a technique, but I have to learn it. So far so bad.

Later I was in a seminary about statistical project planning. Very interesting. Each workitem did have a probability distribution instead of only a planned duration. Of course the probability distribution has an expectancy value and a variance. One tends to think the expectancy value of the workitems duration probability distribution should be equal to the planned duration in deterministic (i.e. non probabilistic) project planning. And one tends to think, that the expectancy value of the duration of the whole project is the sum of all expectancy values of the duration of workitems that are on the “critical path” of the project. But that is not true. Something unexpected happens here – mathematically.

If you use the means of statistics to calculate the probability distribution of the duration of the whole project it turns out, that the expectancy value of the project duration is always bigger or equal than the sum of the expectancy values of the workitems on the “critical path”. THAT is strange. Even if all workitems behave according to their probability distribution – i.e. some take longer but others are completed faster – even then the whole project takes longer.

I am not talking about bad estimations here. I am not talking about the problems that appear, if all workitems take longer than planned or additional workitems are needed. I talk about perfect guesses and a perfectly planned process. EVEN THEN – the whole project takes longer.

Why is this so?

The answer is this: There are dependencies between the workitems. Because of these dependencies certain workitems can’t be started, until others are completed. If one workitem in the chain is delayed, it delays the start of other workitems as well. But if one is faster than planned, it does not necessarily speed up other workitems. Simply said: delays add up while completing ahead of time don’t.

It is the same reason why there is a traffic jam on a highway simply because there are many cars. Why don’t they all drive 120 kilometers per hour? Just because the breaking adds up between the cars and accelerating the car does not.

Of course the effect is much stronger, if the variance of the probability distribution for the tasks is bigger – which is the case with knowledge work.

So – I got my satisfaction. The first teacher was wrong, I was right. I can prove it mathematically .

And you have a good excuse for the next time you miss a deadline. You can say you worked very hard, but there is a mathematical law ….

One goal of the TPS is to eliminate waste – i.e. to eliminate materials, tools and activities that are not needed for the production of the end product. This in itself is goal-driven, because waste is defined as not contributing to the goal. I think this is a good principle in general. But there is a major difference between producing cars and knowledge work.

As we already discussed, the production of a car is completely pre-planned and the process is repetitive. Therefore the process is planned by different people than those that execute it. As we have discussed in knowledge work the same people that execute the process are those that plan the process – at least they work together in small teams. And the knowledge work processes are often not repetitive and often not predictable. There is some repetition and some predictability, but the degree of it is much less as in car manufacturing. Therefore it is clear, that it is the task of the knowledge worker himself to eliminate waste. This is done by re-evaluating priorities of sub goals in relation to a goal or by withdrawing or postponing sub goals. It is not done by a process designer that does it for the knowledge worker.

Also there is a danger in knowledge work about eliminating waste. It is not possible a priori to judge what is waste and what not. Knowledge work is creative work. This also means that experiments must be made and the outcome of the experiments is unknown at the start. Creative minds need freedom to experiment and an atmosphere that primarily eliminates waste can be very counterproductive in this environment. Imagine a soccer match that is only optimized towards eliminating waste. Nobody takes any effort, if the success is not guaranteed. That is boring and that is the sure road to losing the match. If you have seen the match Germany – England you might remember the first goal. Miroslav Klose delivered a brave sprinting fight against the english defense before he scored the first goal. He did not know in advance that he would score. He had to fight for it, even if the result was unpredictable. Only he was able to decide in this situation, if it was worth a try or not.

Ok – I admit the coach does also decide. He does knowledge work on a different level. It is his responsibility to judge if a player does not or is not able to contribute towards the goal of the match and exchange players. The players obviously are not completely autonomous. The team only works as a team if the players fulfill their role within the team. This is in part predictable.

I don’t have all the answers in this discussion and I am also interested in your opinion. Today I only want to make clear that in knowledge work eliminating waste can be dangerous while it is still desirable to a degree and eliminating waste itself is knowledge work because it needs situational judgments and many of the decisions must be done by the knowledge workers themselves.

One aspect of the TPS is tact. Production lines are synchronized by using tact. There is a temptation to try to synchronize knowledge work by tact as well. I do not know how many out there try to do this, but I know of at least two companies.

Nobody is against recurring meetings to synchronize tasks and progress. This is no new invention and certainly not rocket science. We also can discuss SCRUM. But today I want to point out the limitations and difficulties with trying to synchronize knowledge work by tact.

There is a concept – I see it as a law of nature – called irreducible complexity. It means, that there is a minimum complexity for a system to be functional that cannot be reduced without harm. For example a car engine – if there is but one part missing, the engine very probably is not functional. The car engine must be absolutely complete before it is functional. It does not make any sense to assemble an engine into a car, if the engine is not absolutely complete or perfectly functional. In car assembly the “wedding” – i.e. the moment when the engine is assembled into the chassis, follows a strict tact. Thereby the chassis and the engine together become the car. So the engine line and the chassis line are synchronized by this tact.

What happens, if due to any small unpredicted deviation the engine can’t reach the “wedding” in time? The chassis line must be stopped as well. There is no other way. So the whole system works well as long as all work complies with the predicted times and if the work is executed completely perfect.

It becomes clear now, why knowledge work does not fit into this pattern. Knowledge work has many unpredictable components, be it the needed preconditions, the workitems themselves, their duration and their deviation from the expected duration. Imposing tact over knowledge work often results in odd behavior like delivering a half-baked document or installing a half-finished software patch. In Germany we say: “Verschlimmbessern”. This means making it worse by trying to make it better. This is especially the case, if the work does not create a new work result from scratch, but does modify an existing work result – which in my opinion is the main case.

Does this mean I argue for not synchronizing at all or to not plan at all? No! This is far from true. I only argue that the type of synchronizing and planning must respect the laws of nature that we find in knowledge work. This relieves the knowledge worker from the odd feeling and produces more reliable results for the project or management.

Of course, eliminating waste is a good thing. But how do we eliminate waste? Can we eliminate waste by standardizing processes and disallow everything else in knowledge work? No! If this was so, then what would knowledge work be all about? It is about making intelligent and informed decisions.

Toyota obviously produces cars. I once specified a production information system for a car manufacturer for a new car plant in the Far East. It was not Toyota. I’m only saying this, that you believe me that I know how cars are produced.

It is absolutely clear, that the persons, that plan the production process in a car plant and the persons that execute it, are different persons. There are even several levels of planning. For example the production plant layout must be planned. The production flow must be planned – i.e. which tact times will be executed for each production line. The production program must be planned – i.e. which cars are produced when and in which sequence. And the production supply must be planned – i.e. which parts are delivered to which assembly line – in which sequence. All this is planned by persons that are different persons than the person that executes the production.

The challenge for the planning is to make it as predictable as possible. It is using a stopwatch to determine the average duration of a work unit, and the statistical deviation. It is predicting the market demand as good as possible. It is exploding the whole bill of material for a car that is planned to be produced. It is synchronizing processes by imposing tact on the assembly lines.

Now imagine someone using a stopwatch to determine how long you need to read a page of a document. Of course this is an extreme example, but this ridiculous situation makes it clear, why the TPS can’t work for knowledge work. Also there is no bill of material for knowledge work. If we want to produce a car we know which components we need, and which parts for the components. We already know which suppliers will give us which component. That is because the car design is finished before the production planning even starts. In knowledge work, the design decisions are done together with the work. There is no a priori bill of material.

These are fundamental differences between discrete manufacturing, especially line production and knowledge work. We should not use fins for mountain climbing because they have proven successful in the sea.

In the next post I will discuss why tact does contradict irreducible complexity – a law of nature that we often find in knowledge work – and how goal driven and reducing waste relate to each other.