Rapid Application Development toolkit for building Administrative Web Applications

An Activity Based Workflow Engine for PHP

By Tony Marston

16th September 2004
Amended 1st Nov 2016

As of 10th April 2006 the software discussed in this article can be downloaded from www.radicore.org

Introduction

Workflow modelling with Petri Nets

Objects within a Petri Net

Triggers within a Petri Net

Routing within a Petri Net

Splitting and Joining within a Petri Net

A sample Workflow process

Database Design

Workflow E-R Diagram

WORKFLOW table

PLACE table

TRANSITION table

ARC table

CASE table

TOKEN table

WORKITEM table

Maintenance Screens

Processes

Places

Transitions

Arcs

Cases

Tokens

Workitems

The Workflow Engine

Creating a new workflow Case

Updating an existing workflow Case

Result of Creating a Token

Running batch tasks with AUTO triggers

Conclusion

Frequently Asked Questions (FAQ)

References

Amendment History

Introduction

A computer application contains a number of different 'tasks', 'transactions', 'programs' or 'modules', each of which performs a particular function. Sometimes the processing of one particular task is supposed to be followed by one or more other tasks in order to complete some higher process. For example, the task 'Take Customer Order' may have to be followed by 'Charge Customer', 'Pack Order' and 'Ship Order'. This higher process may have a name such as 'Order Fulfillment', but as you can see it cannot be handled by a single task and has to be broken down into its component parts.

Without a workflow system the processing of the component parts has to be handled manually, which is where mistakes can occur. Forgetting to charge the customer or ship the order is not a good way to run a business.

With a Workflow system it is possible to define 'Order Fulfillment' as a workflow process, with 'Charge Customer', 'Pack Order' and 'Ship Order' as components of that process. When an instance (or 'case') of the workflow process is created the workflow engine will then take responsibility for dealing with each component in turn. These components may be executed automatically, or they may be directed to appear in someone's inbox.

What is a 'workflow' system? The Workflow Management Coalition defines 'workflow' as:

The automation of a business process, in whole or part, during which documents, information or tasks are passed from one participant to another for action, according to a set of procedural rules.

There can be two basic types of workflow:

• Activity based - means that the processes, the workflows, are made of activities to be completed in order to get something done.
• Entity based - means that the focus is set on a given document and the states it has to go through in order to be completed.

This document will describe the activity based workflow system which I have constructed as an extension to my Development Infrastructure for PHP. This workflow system has the following component parts:

• A database which holds the definition of each workflow process (such as 'Order Fulfillment' in the above example), along with the sequence of individual tasks (such as 'Charge Customer', 'Pack Order' and 'Ship Order' in the above example) which must be performed in order to complete the process.
• A series of web-based screens which will allow the contents of this database to be maintained.
• A mechanism which detects when a new workflow instance (or 'case') has started, then progresses each case through the sequence of tasks according to the predefined rules.
• Where a task requires human intervention it will appear on a list of outstanding workitems. Each workitem will appear as a hyperlink, and by clicking on a link the relevant task will be automatically activated.

Workflow Modelling with Petri Nets

In order to implement a workflow system it is first necessary to find a suitable means of designing and modeling a workflow process. For this I take advantage of the work done by Carl Adam Petri who was the first to formulate a general theory for discrete parallel systems which gave birth to what are now known as Petri Nets.

Petri Nets is a formal and graphical language which is appropriate for modelling systems with concurrency and resource sharing. It is a generalisation of automata theory such that the concept of concurrently occurring events can be expressed.

Petri Nets have become so popular and widespread that there is a Platform Independent PetriNet Editor (PIPE) available, and it even has its own Petri Net Markup Language (PNML).

Objects within a Petri Net

The Petri Net language contains the following basic objects:

These objects are subject to the following rules:

• Places do nothing but hold Tokens representing the state of the process. A Place can hold zero or more Tokens at any moment in time.
• An Arc connects a Place to a Transition.
  • Place P is called an input place of transition T if there exists a directed arc from P to T
  • Place P is called an output place of transition T if there exists a directed arc from T to P.
• When an enabled transition is fired it moves tokens from all its input places to all its output places.
  • Transition T is said to be enabled if each input place P of T contains at least one FREE token.
  • How an enabled transition is actually fired depends on the type of trigger.
  • When transition T is fired it consumes one token from each input place P of T and produces one token in each output place P of T.
• Each workflow process has a single start place. It must have at least one inward arc going into a transition. It may have an outward arc coming from a transition in order to restart the process.
• Each workflow process has a single end place. It must have at least one outward arc coming from a transition (it may have more than one), but it cannot have any inward arcs going into transitions.

Triggers within a Petri Net

The time the transition is enabled and the time it fires are different. The thing that causes an enabled transition to fire is called a trigger. There are four different types of trigger:

	Automatic	A task is triggered the moment it is enabled instead of being placed in a queue.
	User	A task is triggered by a human participant, i.e., a user selects an enabled task instance to be executed. In a workflow management system each user has a so-called 'in-basket'. This in-basket, which will be displayed on the Menu/Home Page, contains task instances (workitems) that are enabled and may be executed by the user. By selecting and completing a workitem the corresponding task instance is triggered and the workflow case is advanced to the next stage in the process.
	Time	An enabled task instance is triggered by a clock, i.e., the task is executed when a predefined deadline is passed. For example, the task 'remove document' is triggered if a case is trapped in a specific state for more than 15 hours. This should be one of the options in an implicit OR split. Because this type of task can be triggered by a background process which runs at scheduled times it cannot have any dialog with the user. It is also possible to use an online screen to view timed events which have passed their deadline, from where individual workitems can be selected and triggered manually.
	Message	An external event (i.e. a message) triggers an enabled task instance. Examples of messages are telephone-calls, fax messages, e-mails or EDI messages. Each of these external events will probably require some action within an application task so that the workflow system is made aware that the event has taken place.

Routing within a Petri Net

The route between the start place and the end place within a workflow process can take several forms. These are:

Splitting and Joining within a Petri Net

In order implement these routings you may employ a selection of splits and joins. These are:

AND split
	An example of parallel routing where several tasks are performed in parallel or in no particular order. It is modeled by a transition with one input place and two or more output places. When fired the transition will create tokens in all output places.
AND join
	A transition with two or more input places and one output place. This will only be enabled once there is a token in all of the input places, which would be after each parallel thread of execution has finished.
Explicit OR split
	An example of conditional routing where the decision is made as early as possible. It is modeled by attaching conditions or guards to the arcs going out of a transition to different places. Guard - An expression attached to an arc, shown in brackets, that evaluates to either TRUE or FALSE. Tokens can only travel over arcs when their guard evaluates to TRUE. The expression will typically involve the case attributes. More than two arcs of this type can come out of the same transition. They must all have a condition except the last arc as this will be used as the default path if none of the other conditions evaluates to TRUE.
Implicit OR split
	An example of conditional routing where the decision is made as late as possible. Implicit or-splits are modeled as two or more arcs going from the same place but to different transitions. That way, the transition that happens to fire first will get the token. Once the token is gone, the remaining transitions are cancelled and thus cannot be fired. One of the transitions may have a timer as its trigger so that it will be fired if none of the other transitions is activated before the time limit expires. Expired transitions can either be triggered automatically via a background process which is running on a timer (e.g. cron), or manually via an online screen.
OR join (explicit and implicit)
	Is simply a place that serves as the output place of two different transitions. That way, the next transition after the or-join place will be enabled when either of the two conditional threads are done.

A sample Workflow process

A workflow is the formal definition of the process used to manage cases of a specific kind (e.g. order fulfillment, article publishing). Each kind of case will have its own workflow process. Here's an example of an order fulfillment process:

Order Fulfillment workflow

The explanation of this diagram is as follows:

• The circles are called places and represent the inboxes in the office-metaphor.
• The rectangles are called transitions and represent the tasks to be performed.

Places are inactive. All the places do is hold tokens representing the state of the process. If, for example, there's a token in place D above, then that means we're ready to pack the order.

Transitions are active. They move tokens from their input places (the places that have an arc pointing into the transition) to their output places (the places you get to by following the arcs going out of the transition). When this happens the transition is said to fire.

Transitions can only fire when there's at least one FREE token in each input place. When that is the case, the transition is enabled. That the transition is enabled means it is able to fire. It will fire when the conditions of its trigger are satisfied.

When the workflow is started, a token is placed in the start place (A in the example). This enables the automatic transition 'Charge Credit Card'.

The transition fires with a success or a failure. If it was successful, it produces a token in place D. If there was a failure, it produces a token in place B. Thus, the outcome of the attempt at charging the credit card governs the further routing of the process. The rule is that firing a transition consumes one token from each of its input places, and places a token on each of its output places for which the guard is true. The guard is a predicate, in this case the [success] and [failure] on the arcs going out of 'Charge Credit Card'. Guards are what enables us to do conditional routing. The 'Charge Credit Card' transition acts as an explicit or-split, because it chooses either one route or the other.

The other form of conditional routing, which is the implicit or-split, is what chooses between the transitions 'Update Billing Information' and 'Cancel Order'. Since there's only one token in place C, only one of the two transitions can have it. But, contrary to the explicit or-split, where the decision is explicitly made as soon as 'Charge Credit Card' finishes, the choice between 'Update Billing Information' and 'Cancel Order' is made as late as possible.

Both transitions will be enabled when there's a token in place C (i.e. when the spam has been sent). If the user updates his billing information before the timed 'Cancel Order' transition times out, 'Cancel Order' is never fired. And vice versa: If the order is canceled (which will probably involve spamming the user again to let him know that his order was cancelled), then he won't be able to update his billing information and will have to enter a new order. Thus, the choice is made implicitly, based on the timing.

The guard will generally depend on case attributes. The 'Charge Credit Card' transition above will set a case attribute to either 'success' or 'failure', and the guard will check this value to determine its result. Case attributes can hold more complex values than simple yes/no values, but the guard must always be either true or false.

What is missing from this diagram is the process which initiates a new workflow instance (or 'case') and puts a token in the start place. In the above example the case initiator would be 'Take Customer Order'. In this implementation the activity which initiates a workflow case is shown as start_task_id on the WORKFLOW table.

Database Design

The main issue for a workflow management system is answering the question 'who must do what, when and how'. Some of these components already exist within my application, which is available for download from http://www.radicore.org, while others will need to be created separately.

Workflow E-R Diagram

Here is the Entity-Relationship (E-R) Diagram for my workflow tables:

E-R Diagram

WORKFLOW table

The structure of this table is as follows:

CREATE TABLE `wf_workflow` (
  `workflow_id` smallint(5) unsigned NOT NULL default '0',
  `workflow_name` varchar(80) NOT NULL default '',
  `workflow_desc` text,
  `start_task_id` varchar(40) NOT NULL default '',
  `is_valid` char(1) NOT NULL default 'N',
  `workflow_errors` text,
  `start_date` date default NULL,
  `end_date` date default NULL,
  `created_date` datetime NOT NULL default '0000-00-00 00:00:00',
  `created_user` varchar(16) default NULL,
  `revised_date` datetime default NULL,
  `revised_user` varchar(16) default NULL,
 PRIMARY KEY  (`workflow_id`)
) ENGINE=MyISAM;

PLACE table

The structure of this table is as follows:

CREATE TABLE `wf_place` (
  `workflow_id` smallint(5) unsigned NOT NULL default '0',
  `place_id` smallint(5) unsigned NOT NULL default '0',
  `place_type` char(1) NOT NULL default '5',
  `place_name` varchar(80) NOT NULL default '',
  `place_desc` text,
  `created_date` datetime NOT NULL default '0000-00-00 00:00:00',
  `created_user` varchar(16) default NULL,
  `revised_date` datetime default NULL,
  `revised_user` varchar(16) default NULL,
  PRIMARY KEY  (`workflow_id`,`place_id`)
) ENGINE=MyISAM;

TRANSITION table

The structure of this table is as follows:

CREATE TABLE `wf_transition` (
  `workflow_id` smallint(5) unsigned NOT NULL default '0',
  `transition_id` smallint(5) unsigned NOT NULL default '0',
  `transition_name` varchar(80) NOT NULL default '',
  `transition_desc` text,
  `transition_trigger` varchar(4) NOT NULL default 'USER',
  `time_limit` int(11) unsigned default NULL,
  `task_id` varchar(40) NOT NULL default '',
  `role_id` varchar(16) default NULL,
  `created_date` datetime NOT NULL default '0000-00-00 00:00:00',
  `created_user` varchar(16) default NULL,
  `revised_date` datetime default NULL,
  `revised_user` varchar(16) default NULL,
  PRIMARY KEY  (`workflow_id`,`transition_id`)
) ENGINE=MyISAM;

ARC table

The structure of this table is as follows:

CREATE TABLE `wf_arc` (
  `workflow_id` smallint(5) unsigned NOT NULL default '0',
  `transition_id` smallint(5) unsigned NOT NULL default '0',
  `place_id` smallint(5) unsigned NOT NULL default '0',
  `direction` char(3) NOT NULL default '',
  `arc_type` varchar(10) NOT NULL default 'SEQ',
  `condition_field` varchar(40) NULL,      
  `condition_operator` varchar(4) NULL,
  `condition_value` varchar(40) NULL,
  `created_date` datetime NOT NULL default '0000-00-00 00:00:00',
  `created_user` varchar(16) default NULL,
  `revised_date` datetime default NULL,
  `revised_user` varchar(16) default NULL,
 PRIMARY KEY  (`workflow_id`,`transition_id`,`place_id`,`direction`),
  KEY `place_id` (`workflow_id`,`place_id`,`direction`),
  KEY `transition_id` (`workflow_id`,`transition_id`,`direction`)
) ENGINE=MyISAM;

if ($fieldarray['name1'] > 10) {
    return TRUE;
} else {
    return FALSE;
} // if

if (strlen($fieldarray['name1']) > 10) {
    return TRUE;
} else {
    return FALSE;
} // if

if ($fieldarray['name1'] > $fieldarray['name2']) {
    return TRUE;
} else {
    return FALSE;
} // if

CASE table

The structure of this table is as follows:

CREATE TABLE `wf_case` (
  `case_id` int(10) unsigned NOT NULL default '0',
  `workflow_id` smallint(5) unsigned NOT NULL default '0',
  `context` varchar(255) NOT NULL default '',
  `case_status` char(2) NOT NULL default 'OP',
  `start_date` datetime NOT NULL default '0000-00-00 00:00:00',
  `end_date` datetime default NULL,
  `created_date` datetime NOT NULL default '0000-00-00 00:00:00',
  `created_user` varchar(16) default NULL,
  `revised_date` datetime default NULL,
  `revised_user` varchar(16) default NULL,
  PRIMARY KEY  (`case_id`),
  KEY `workflow_id` (`workflow_id`)
) ENGINE=MyISAM;

TOKEN table

The structure of this table is as follows:

CREATE TABLE `wf_token` (
  `case_id` int(10) unsigned NOT NULL default '0',
  `token_id` smallint(5) unsigned NOT NULL default '0',
  `workflow_id` smallint(6) unsigned NOT NULL default '0',
  `place_id` smallint(5) unsigned NOT NULL default '0',
  `context` varchar(255) NOT NULL default '',
  `token_status` varchar(4) NOT NULL default 'FREE',
  `enabled_date` datetime NOT NULL default '0000-00-00 00:00:00',
  `cancelled_date` datetime default NULL,
  `consumed_date` datetime default NULL,
  PRIMARY KEY  (`case_id`,`token_id`),
  KEY `place_id` (`workflow_id`,`place_id`)
) ENGINE=MyISAM;

WORKITEM table

The structure of this table is as follows:

CREATE TABLE `wf_workitem` (
  `case_id` int(10) unsigned NOT NULL default '0',
  `workitem_id` smallint(5) unsigned NOT NULL default '0',
  `workflow_id` smallint(6) unsigned NOT NULL default '0',
  `transition_id` smallint(5) unsigned NOT NULL default '0',
  `transition_trigger` varchar(4) NOT NULL default 'USER',
  `task_id` varchar(40) NOT NULL default '',
  `context` varchar(255) NOT NULL default '',
  `workitem_status` char(2) NOT NULL default 'EN',
  `enabled_date` datetime default NULL,
  `cancelled_date` datetime default NULL,
  `finished_date` datetime default NULL,
  `deadline` datetime default NULL,
  `role_id` varchar(16) default NULL,
  `user_id` varchar(16) default NULL,
  PRIMARY KEY  (`case_id`,`workitem_id`),
  KEY `transition_id` (`workflow_id`,`transition_id`)
) ENGINE=MyISAM;

Maintenance Screens

These are the maintenance screens for the Workflow system:

Workflow Processes

List Workflow Process

Add Workflow Process

Delete Workflow Process

Enquire Workflow Process

Search Workflow Process

Update Workflow Process

Validate Workflow Process

Workflow Places

List Workflow Place

Add Workflow Place

Delete Workflow Place

Enquire Workflow Place

Search Workflow Place

Update Workflow Place

Workflow Transitions

List Workflow Transition

Add Workflow Transition

Delete Workflow Transition

Enquire Workflow Transition

Search Workflow Transition

Update Workflow Transition

Workflow Arcs

List Workflow Arc

Add Workflow Arc

Delete Workflow Arc

Enquire Workflow Arc

Search Workflow Arc

Update Workflow Arc

Workflow Cases

List Workflow Case

List Case within Workflow

Enquire Workflow Case

Search Workflow Case

Workflow Tokens

List Workflow Token

List Token within Case

Enquire Workflow Token

Search Workflow Token

Workflow Workitems

List Workflow Workitem

List Workitem within Case

Enquire Workflow Workitem

Search Workflow Workitem

List Outstanding Workitems

The Workflow Engine

Having defined a workflow process the next step is to have a means whereby instances (or 'cases') of that process can be created then progressed from one stage to the next. This is where the 'engine' comes in. This is the piece of software that monitors every action to see if it requires the creation of a new workflow case or the updating of an existing one.

One feature of my infrastructure design is that every database access (select, insert, update and delete) goes through my abstract table class. Because of this design feature I therefore have a single class in which it is possible to trap all database activity within the system, and therefore I can implement my workflow engine without having to make modifications to huge numbers of scripts.

There are two significant points to consider with this implementation:

• None of the application tasks is 'workflow aware'.

  No application task is aware that is part of a workflow, and none of the code within an application task has to be amended in order for it to be included in a workflow case.

• The workflow system is not 'application aware'.

  The workflow system itself does not contain any application code - all it does is sit in the background 'sniffing' at each task when it runs to find out if it is part of a workflow case or not. If it is, then when that task (workitem) is terminated (fired) the workflow system will move tokens from one place to another, which may cause another workitem (task) to be enabled.

Creating a new workflow Case

Trapping when a new database record has just been created is very simple as new records are created through the insertRecord() method of my generic table class. Similarly all updates go through the updateRecord() method. All I have to do is insert the following code at the end of these methods:

        if (empty($this->errors)) {
            // additions to the workflow database do not count
            if ($this->dbname != 'workflow') {
                // find out if this task/context has any workflow connections
                $this->_examineWorkflow($fieldarray);
            } // if
        } // if
        
        return $fieldarray;
    
    } // insertRecord

The _examineWorkflow() method will perform the following:

• Do a lookup on the WORKFLOW table using the following criteria:
  • task_id = identity of current TASK
  • is_valid = TRUE
  • start_date <= '$today'
  • end_date >= '$today'
• If no record found then return.
• If record found then:
  • Create a new entry on the CASE table and set context to the primary key of the application record which triggered this action.
  • Put a TOKEN in the start place (see Result of Creating a Token).

Updating an existing workflow Case

This involves looking for an outstanding WORKITEM during the execution of the insertRecord() and updateRecord() methods which matches the current task identity and context. The selection criteria is as follows:

• task_id = identity of current TASK
• workitem_status = 'EN'
• context = $where

Thus 'Pack Order where customer_id=1234' will involve a different WORKITEM to 'Pack Order where customer_id=5678'.

If a record is found where its primary key matches the current context then the following checks are made:

• If the role_id on the WORKITEM record is not the same as the role_id for the current USER then the task will be terminated with a suitable error message.
• If the (non-blank) user_id on the WORKITEM record is not the same as the current USER then the task will be terminated with a suitable error message.
• If the user_id on the WORKITEM record is blank then it will be updated with the identity of the current user. This has the effect of assigning the workitem to the current user and making it inaccessible to other users. When a workitem is being executed it is possible to manually assign the next workitem(s) to a different user by inserting a value into $this->wf_user_id.

If the insert or update is successful then the transition/workitem is said to have been fired, which has the following effect:

• The status of the WORKITEM is changed from 'ENABLED' to 'FINISHED'.
• One TOKEN is removed from each input place (token_status is set to 'CONSUMED').
• One TOKEN is added to each output place (see Result of Creating a Token).

Note that a transaction which is included in a workflow must perform a database commit() (with or without an actual database update) so that the workflow engine can be informed that the transaction has been completed successfully. If the transaction terminates without performing a commit() then the workflow engine will take no action.

Result of Creating a Token

Whenever a token is created in a place the following checks are made:

• If it is the end place then the CASE is closed.
• Find all inward arcs which go from this place into a transitions, and for each transition perform the following:
  • Get a list of all places which input to the transition.
  • Check that each of these places has a token (an AND-join must have one token for each input place). If the correct number of tokens is found then:
    • enable the transition (create a new WORKITEM record).
    • If the transition trigger is 'AUTO' then process that TASK immediately, otherwise wait for some other trigger.

Running batch tasks with AUTO triggers

Tasks in the menu system are designed to be run in one of two ways:

• Online, from a web page.
• Batch, from the command line.

Batch tasks are run in the background, do not have any dialog with the user, can be run automatically at scheduled times via cron or a task scheduler, and can stay active for extended periods.

When a transition is processed and a token is moved from an input place to an output place, this may enable a subsequent transition. If this subsequent transition has an AUTO trigger then it will be run immediately. While it is possible for an online transition to be followed by an AUTO-triggered transition which is either online or batch, it is only possible for a batch transition to be followed by an AUTO-triggered transition which is also batch. It is not possible for a batch transition to be followed by an AUTO-triggered transition which is designed to be run online.

Conclusion

As you can see designing and building a workflow system is not a trivial matter. The major steps are:

• Understanding the requirements. This is where the work on Petri Nets comes in very useful.
• Designing a database that can model a workflow process and hold the details of individual cases.
• Having a method whereby every activity can be monitored to see if it triggers a new case or updates an existing case. This is where my generic table class plays a great part as it provides a single place where all database activity can be monitored. Without this I would be faced with having to put workflow code in numerous places.

I do not claim that this design is perfect, but how does it compare to yours?

Frequently Asked Questions (FAQ)

1. How do I use the Workflow system?
2. Why doesn't the Workflow system have a facility for sending emails?
3. Why don't you store state-specific info in the 'context' field of a workflow case?
4. How can I customise the text of pending workflow items on the menu/home page?
5. How can I control the sequence in which the conditions on Explicit OR Splits are evaluated?
6. What happens when a place contains more than one token?
7. How are workitems assigned to users in the Workflow system?

References

• The Application of Petri Nets to Workflow Management (PDF) by W.M.P. van der Aalst

Amendment History