 |
The DMS, a neglected architectural component: how do you fit document management in your process-based architecture?
"Many organizations have implemented a Document Management Systems (DMS). But that doesn't mean that documents are now more integrated in the line-of-business applications then before. What is your strategy to integrate document flows in your ICT Architecture?" In today's tough economical conditions, there is more than ever the need to innovate and control costs and as such it is indispensable that an enterprise can respond very quickly to change, with minimal effort. Traditionally, projects start in the IT department, mostly without taking into account business requirements, but this just doesn't work anymore. Instead, projects should be driven by business needs, and evidently the underlying IT solution has to support this.
Introduction
It is commonly understood that a Document Management System (DMS) should be part of every organization's ICT Architecture. But today, in most organizations the DMS is as 'integrated' in that ICT Architecture as oil in water (i.e. they are neatly separated).
The problem is that, although documents are the most common information carriers in many organizations, they lead a separate life at a fair distance from transactional systems and even more from the process-based applications, if they exist. As a consequence the information worker, in his day-to-day job, has to joggle and switch between applications like the transactional systems he is using, the DMS and his e-mail in- and outbox. Compare this to having a car to drive on flat roads, a SUV for driving up- and down-hill and a little truck to go to the supermarket, without the possibility to drive your car on mountainous roads, the SUV on flat roads, or load any kind of groceries in your car or SUV: how practical would that be?
What is needed is an understanding of the relationship between documents and business processes, and documents and transactional data. This article explores those relationships, first by looking at the different types of documents that are stored in the DMS and then by defining a classification of business processes that create and use those types of documents. It also comments on the role of the user's task list in the context of process-based applications, and the implicit use of the e-mail inbox in that function. Finally, all of this is combined in the last paragraph into a strategy to integrate document flows into an architecture with process-based applications and a transactional back-end.
Short history of Document Management Systems
What was the original reason for existence of Document Management Systems (DMS)? What was the reason to have another mechanism to store documents centrally and share them between people other than the file system and shared drives? The reason was the need to have a more secure and reliable storage of documents and the possibility to enforce policies on how documents could be created, modified and deleted. The initial application areas that used a DMS confirm this: Quality Management and Documentation Management. The types of documents managed in those environments require a well managed document life cycle process: for each document type, the authorized creators, reviewers, approvers and readers are defined upfront, and for each type of change to the document the workflow that has to be followed is formally defined. Based on those definitions of actors and workflows the DMS makes sure that documents are created, modified and deleted in a controlled way.
What we see today however is that the DMS is evolving to an architectural component that needs to deal with completely different types of documents leading to completely different needs in terms of actors and workflows.
A classification of documents
The distinction that I would like to make here between very different types of documents stems from the distinction that is commonly made between different classes of business processes. Every book on Business Process Management will tell you that there are 3 fundamentally different types of business processes in every organization: Core Processes, Supporting Processes and Management Processes. Core Processes are the processes in your organization that deliver to your (external) end-customer (typically the Marketing and Sales Processes, Production, Delivery, etc.). Support Processes are the processes that deliver to internal customers (such as HR, Finance, and ICT). Management Processes deal with reviewing and setting the direction and objectives of the organization, defining how the organization should operate, and controlling the execution.
 Now let's get back to the reason for existence of the early DMS: the documents stored in the DMS have long been the output of the Management Processes in an organization. They are the type of documents that describe how the organization should work. That means that they contain meta-information on the organization, not operational information. Let's call them Procedural Documents. Compare e.g. a document containing the procedure for approving offers with a document containing the offer itself. The first one describes how the process should be executed, the second one contains information on (1 instance of) the process itself. Let's call the latter Operational or Process Documents.
Again this distinction is very important because it leads to 2 different sets of requirements when it comes to supporting the underlying processes in the life cycle of these documents.
The version control workflow: one size fits all (for Procedural Documents)
 Let's first stay with the procedural type of documents, the ones that describe how the organization should operate: procedures, manuals, documentation. The systems that manage this type of documents are often indicated with the acronym SHEQ (Safety, Health, Environment and Quality systems) of which the Q is most widely known as a result of the ISO certification of many organizations. Although the content of these documents cover a wide range of subjects, from a process/workflow perspective they share common requirements: the need to define a pre-defined set of roles that are authorized to work on the documents, and the need to have a pre-defined workflow that assures that new versions of the documents are created and released in a controlled way. The roles might be: requestor (who is authorized to request a change to a document), owner (who is authorized to modify the document), reviewer (who should review the document after modification), approver (who can authorize the publication of the new version of the document) and the audience (people who have to know the content of the document). Sometimes the acronym RACI is used for this collection of roles (Responsible, Accountable, Consulted, and Informed). From these roles you can easily derive the workflow:
- Someone request a change,
- The owner evaluates the request and rejects or accepts it, and
- When he accepts it, he modifies the document
- Whenever a document is modified, the reviewer reviews it,
- After review the approver authorizes the publication of the document
- And finally the audience is notified that there is a new version of the document.
What is typical for these procedural documents is that the above workflow fits for all documents, independent of the subject they are covering. This is a generic workflow that deals with the version management of any procedural document.
When you look at the standard out-of-the-box workflows delivered with Microsoft's Office SharePoint Server (MOSS) you will find a striking resemblance with the version control workflow pictured above. This (together with features like automatic versioning and targeted publishing using audiences) indicates that SharePoint can be used right out of the box as an appropriate DMS to manage all your SHEQ documents.
Every Operational Document has its Process
In the eighties, Lotus used the title of this chapter (or something similar, I don't remember the exact wording) when launching its new document-and-workflow software. It was true then and it still is true today. (Go on think of this for a while: do you have documents in your organization without any underlying process?) However, the types of documents we are talking about here are very different from the how-to documents in the previous chapter: here we are talking about the documents that are used in the core and supporting processes of your organization, i.e. operational documents: price quotations, purchase orders, expense notes, investment requests, incident forms, etc. In fact, very often the document is the process. What we see today is that these types of documents are stored more and more in the DMS too. This is relatively new, and is not without consequence. Firstly the underlying business processes of these documents are very different from the generic version control workflow described in the previous paragraph. Working with operational documents creates the need for a more powerful process engine in the DMS since every document has a completely different flow from the other. In this context one size does NOT fit all! Secondly, in most cases, the underlying business processes of these documents are also (partially) supported in the existing back-end systems. Hence the need to have a vision and architecture to integrate documents, processes and transactional data.
Operational Documents, Processes and Transactional Data
Today every ICT architect faces the challenge to integrate Operational Documents, Data and Processes in his organization's architecture. To help with this challenge, let's have a look at the intersections between the circles in the picture to the right (representing the different types of integration). Once you understand the relationships between these components, it will be much easier for you to come up with a strategy for your organization, as we will see in the last paragraph.
 Intersection 1 represents the overlap between the data related to the operational documents and the transactional data that sits in the back-end databases. The data related to the documents is stored in the DMS as properties of the document. It is obvious that most of the properties of operational documents correspond to transactional data in the back-end database. A document containing an Offer to a Customer that is stored in the DMS will probably have properties like Customer No, Customer Name, Project, Product, Division, etc., data that is also stored in the Offer entity of the back-end database. How do you synchronize that? How do make sure that when the Customer Name changes in the database that it also changes in the DMS?
Intersection 2 represents the overlap between documents and the business processes. Remember: every document has its process! So every operational document that is stored in the DMS is part of a process that you want to support with your process engine sooner or later. In this context, it may be useful to distinguish further between 2 types of operational documents:
- In some cases the document IS the process (e.g. an expense note)
- In other cases the document is more of an attribute of the process (e.g. in the process of making an offer, the Excel workbook containing the price calculation for the offer is created during of the process but is not the process itself).
How are you going to support this with your architecture?
Intersection 3 is not really in the scope of this article, but is still worth mentioning: the data gathered during the process is bound to be stored sooner or later in your back-end database. And vice versa: data from your back-end systems will be used in your processes. (I have written about this intersection in my article "BPM Misconception: the back-end doesn't need to know everything!" in the 2nd issue of the Microsoft Belux Architect Newsletter.)
Intersection 4 represents the most likely combination in many business processes: the combination of documents, transactional systems and processes. Let's illustrate this with the following example (Price Quotation Process):
In the first step a sales person receives a Request for Proposal document as attachment to an e-mail. The sales person saves the RFP document in the DMS and fills in properties like Customer Name, Project, Product Line, and Dead Line. By doing that he automatically triggers the process of responding to the RFP.
In the second step the responsible for the Product Line evaluates the request and decides to go ahead and answer the RFP, or to stop the process. If the process is continued, the RFP is also registered in the CRM application to update the sales pipeline. The integration between DMS and CRM can be implemented in 2 ways: either the Product Line Manager fills in additional properties in the DMS, and these properties are used to create automatically an entry in the CRM application, or the Product Manager creates an entry in the CRM application, and some fields are copied automatically from the CRM application to the properties in the DMS. (I prefer the first solution because the user can continue to work in the same application: the DMS). Etcetera. I think this gives you some idea of a true integration between DMS, process and transactional applications. How this might translate to a strategy for your organization is explained in the last paragraph.
All you need is … a task list
One of the consequences of working with process-based applications is that the user is informed of the work to be performed by him through a task list. Each time a next step in a process needs to be executed, that step is put on the task list of the related user. This way of working is becoming more and more common. Many components in the application landscape of an organization today have some kind of workflow and thus some kind of task list (although very seldom used). In a typical Microsoft stack for instance you will find a process engine and task list in SharePoint, in Dynamics CRM, AX and NAV. And Outlook also has a task list (though no process engine). But do you want to expose all of them to the user? Preferably, all tasks generated by all process engines are concentrated in one task list for the user. I believe the choice is obvious. What is the first application that you open in the morning (be it at home, in the office, or on the road)? Right: your e-mail application. When I present a process-based approach to organizations and explain the role of the task list to them, I often get the reaction that (their) people don't want to be instructed what to do (and when) by an application. In reality however, that is exactly what most information workers are already doing today (in various ways): their inbox tells them what to do next (and many people use their inbox as a memory pad for their to-do list). Yet, again, until today we failed to integrate the most used end-user application into our architectures and application landscape: the e-mail client. If, from a user experience perspective, it should be our goal to increase user productivity by limiting the application switching every information worker is experiencing today, then integrating the e-mail client in our solutions as the task list of choice will undoubtedly be a big step forward. (By the way, Microsoft's Dynamics CRM is doing a great job in that respect: it is very integrated into Outlook and uses the same look-and-feel.
And guess what: many organizations state that that is exactly why they have chosen Dynamics CRM and are very happy to work with it.)
Choose your strategy
What can you learn from all this? First of all: admit that you have a problem; if you don't see a problem, you will never find a solution. Admit that in your application architecture, documents are treated like if they operated on a different planet. Secondly, try to determine if your organization is mainly document-driven or data-driven by looking at your core and supporting processes: are most of your business processes triggered by documents, or by data? (Insurance companies e.g. are definitely document-driven organizations, whereas Maintenance Organizations are more data driven.) Once you have determined that, you are ready to work out a strategy.)
If you work in a document-driven organization, then your DMS should be the primary working place for your users, and the documents in your DMS should be the basis for the processes (and the data). This means a.o. that you must have a DMS with a grown-up process engine, not one with just a version control workflow. SharePoint would be a good starting point because it includes both document storage and process engine functions. The properties of the documents will get filled in throughout the process, and at the end of the process you insert these data into your transactional database. (See the article "BPM Misconception: the back-end doesn't need to know everything!" in the 2nd issue of the Microsoft Belux Architect Newsletter.)
If you work in a data driven environment, then your transactional application should be the primary work place for your users, and the transactions on the data are the basis for your processes. Microsoft Dynamics CRM or AX may be a good choice to build upon, because on top of the traditional functions for storing and managing data, they too have a (grown-up) process engine, and they can be integrated with SharePoint. In this case your documents probably need to be considered as attributes of the data stored in the application database. This means that you have to implement an architecture where documents can be uploaded as part of a transaction on the data and stored in a SharePoint environment where they can be accessed and reviewed by everyone outside the context of the transactional application.
Thirdly, when you start building solutions, keep in mind that your back-end system doesn't need to know everything!
And finally, whatever your choice is, make sure to go with the flow: make sure to work out a way to integrate your solution into your user's Outlook client, because that's where he probably spends most of his working time already.
Bio
|
Marc Vanderheyden is Managing Partner of SPIKES N.V. (www.spikestogether.be) a solution provider for process-based solutions based on Microsoft technologies.
Aged 52, he holds a Master in Romance Languages and a Bachelor in Information Technologies. Over the last 30 years he developed applications for various industries and with various technologies. In 2002 (when process engines started to emerge) Spikes was established with the vision that all business software development should start from the business process perspective.
|
Since then Marc has developed a clear vision on process modeling and process-based development and helped to develop solutions for companies like Fortis Lease, ING Car Lease, BMW Financial Services, Tiense Suikerraffinaderij (Südzucker), G4S, etc.
marc.vanderheyden@spikes.be
Mob.: +32 475 25 19 65 |
|
|
|
 |
Event driven architecture onto the Azure Services Platform
“The Azure Services platform constitutes a new architectural paradigm shift and Capgemini explains why EDA is a good architectural style for the Cloud and how to implement it using the Azure platform.”
It's all about shortcuts on desktops...
“Why have infrastructure architects failed to deliver the promise of a low TCO desktop platform? Avanade describes three practical solutions that help to establish a clear and simple application provisioning strategy.”
Get rich and famous thanks to the Azure Services Platform
"New CloudApp()" is a Microsoft developer challenge (Contest Site) that promotes the new opportunities and innovative ideas developers (.NET & PHP ) are creating with cloud computing on the Azure Services Platform.
|
|
