Apart from the waste of effort, re-entering data is a major opportunity for errors to creep in. Is the surname Johnson, Johnston or Johnstone? Spencer or Spenser? Errors are bad enough, and the customer wrongly addressed as Johnson when they’re a Johnston will undoubtedly feel irritated, but having that customer as Johnson in one database and Johnston in another can result in real problems.
If applications are connected in a way that enables data to be shared, the errors of inconsistency can be eliminated. InterSystems Ensemble is a development environment with middleware for integrating existing applications. When The Institute of Cancer Research wanted to integrate different departmental applications, they chose Ensemble for the development.
The Institute of Cancer Research is the world’s fourth largest cancer research charity, and the biggest of its kind in Europe. It is rigorous in its commitment to spending the largest possible amount of its funds on research, which means it can’t afford unnecessary spending on back office functions. However, those functions need to be efficient. The Institute recognised that the way employee data was handled could be made more efficient, as the software used for individual functions such as HR, email, and facilities management did not interact.
The Institute decided to provide a corporate employee lifecycle management system that would handle everything from a new employee being taken on, through the management of that employee’s data, to their eventual leaving. One reason why this is so important is that The Institute has a large number of visiting research fellows and temporary workers who are with The Institute for the length of a particular project, so has a high staff turnover. The existing system was based on several departmental systems, each chosen as most appropriate because of the functions provided. Unfortunately, the fact that the systems did not interact meant that various manual processes were in place between the different departments, resulting in duplication of work and the potential for inconsistencies to be introduced. The lack of communication between the departmental systems meant that it was not possible to ensure timely actions; for example, when an employee left, their email and network accounts would not be automatically closed.
The Institute of Cancer Research put out invitations for a competitive tender to see which integration software would best suit their needs. InterSystems came out best for number of reasons, including the fact that they demonstrated a working solution to a part of the system, showing how the data would flow from one part of the system to another.
The Institute uses Novell as their main system for both email and network identities. InterSystems showed how it would be easy to connect Novell’s Groupwise and eDirectory using Ensemble. Because Ensemble enables rapid prototyping, the team were able to show that element of the system working, and to model the process and some user cases.
When a new employee joins, they are added to the Human Resources database, and allocated a room, furniture and a phone. As in many organisations, many of the steps entailed in getting ‘signed up’ involved the new joiner going round various people with their details on a piece of paper so they could be added to the various systems. Mapping the correct process in terms of what should happen and in what order is an important part of the development, according to Pat McGibbon, one of the development team from InterSystems who worked on the project. “Our task was to help them to really hone in on exactly what was wanted. It’s important that everyone agrees, and that you can demonstrate the process management.”
These four shots show elements of the ICR system in action
Process management isn’t just about managing the workflow and the data flow through the system. It’s equally important to manage the people involved in the process, the ones who will actually be on the other end of the keyboard. They have to be sure their system will continue to deliver, and that they will be able to see and accept the details being entered.
The system linked together what had been five separate departmental systems, each of which had been under the control of the individual departments. When the data is treated as corporate rather than departmental, there still has to be some local control measures – no-one wants to have to lose control over what’s entered into their own system.
The solution for The Institute was to include Gatekeeper screens. Essentially, these are used to show what will be entered into the departmental system. If the person reviewing the data could see a problem with the data, they could then raise that as an issue, have the corrections made back at the original data entry point, and the correct data would then be propagated across all systems. The gatekeeper screens enabled the departmental data owners to have confidence in the whole process. Behind the scenes, Ensemble adapters connected to each departmental system. Wherever possible, generic choices were made, so that when connecting to the Novell eDirectory system, the team chose LDAP for the connector. This means that if in future the Institute decides to use a different directory system (Active Directory, for example), the adapters will still work. It makes sense to keep flexibility in the solution.
Five systems were in use. The HR system was the place where changes in an employee’s status are first recorded, whether that involves a new employee, a leaver, or a change. This system was based on a SQL Server database. The next system in the sequence was the facilities database. This added extra information for the employee such as the office location, phone number, and details of resources such as furniture. This system was also based on a SQL Server database. The finance system was the next in the process, and it used an Oracle database for its data store. The finance system is where employees can request resources and facilities for projects, and have costs approved. Novell systems for email, user login on the corporate network, and a home area on the file server for storing documents next came into play. The final system was the helpdesk system, where employees are registered as entities so that they can request IT help, log any IT issues, and so on.
The choice of third party products was in each case made by the separate departments looking for a best of breed solution. While the systems were separate this did not matter, as each department managed its own system, and had its own support people. When a corporate standard is set where all departments must choose from a related range of products, some departments will probably have to compromise on their ideal set of features for their application. Allowing each department to make their own choice avoids this, but does make it more difficult to create composite applications.
The Ensemble team said that the integration task did not present them with too many challenges, however. The databases were treated as standard SQL databases, and the adapters communicated with each of the databases. This is one of the most important jobs of integration hubs – linking the internal systems and the hub. Integration hubs also need to abstract data in a way that is meaningful in business terms. If the data that is made available is known to be correct, it doesn’t matter which system it came from, or where it’s going to. The final format is irrelevant; all that matters is that if a system needs data, it is available.
Components of the ICR system
What the Ensemble system does is to listen for changes in the HR system. When a change is made, whether that be the generation of a new employee record, or the changing of details in an existing record, those changes are abstracted and forwarded to the appropriate processes. For a new starter, this would mean an addition to the facilities system. This is a system where the basic employee details are stored, and details such as the phone number and office facilities are logged. By removing the need to rekey any details from the HR database, the new system reduces errors and clerical work. The data is inserted into the database from the web portal, though naturally there are validation rules in place. Once the employee is set up in the facilities database, parallel streams can be sent to the other systems – finance, the help desk, and the Novell systems. None of those systems add information that is relevant to any of the other systems. Each system needed its own validation in place, with the finance system in particular requiring a lot of validation to meet the complex rules on how data should be entered.
As Pat McGibbon pointed out, “We worked closely with each department. The financial system is used to manage project expenses, so that people can have their expenses charged against a particular project. Prior to our system being in place, there were occasions when someone would appear in the finance department but there would be no record of them in the financial system, or the project they were working on would not be registered, which would hold things up. Under the new system, it’s a simple matter of pressing a button to make a request to HR, who can then set up whichever is incorrect – the project or the person’s details.”
When integrating the Novell systems, the decision was taken to use LDAP rather than the Novell APIs as for the Novell APIs for the eDirectory system because LDAP is not tied to any particular vendor. If the ICR decides to change in the future, this will reduce the amount of work to change the system. The only way to work with the Groupwise email system was to use Novell’s COM ActiveX API for all calls to create a new account, make changes and so on.
The biggest improvement for all the departmental stakeholders was the ability to view the details to be added, check that everything was OK, then hit a button and the data would all be added to the system. The data error rates dropped significantly, and if a detail is entered incorrectly, it is at least consistently incorrect.
This might seem a strange way to measure success, but the question of who owns corporate data is something that has to be addressed with all connected systems. Taken in the abstract, it’s obvious that no one department can claim ownership, but that doesn’t alter the fact that departments tend to be very protective of their own local systems. Despite this, data can’t be allowed to be changed while on the way through a process. You need to know that there’s a universal record, that the common data will be recorded in the same way throughout. If it’s been accepted by the finance department, the people in charge of the email system can’t reject it. What has to happen is that the data is accepted as it is, and that any changes are made by pushing them back to the initial database – in this case the HR database, for the changes to be made and propagated across all the different systems. Consistency is the key.
One of the important ways Ensemble helped make the development a success is the way it handles Business Process Management (BPM). Ensemble lets you map out processes using BPM and make them visible. This becomes part of the development process, because the ability to visualise what the process is means that stakeholders can see what the process is, and think hard about whether that’s really what is needed. In many projects, when stakeholders see the BPM, they feedback with the fact that the steps in the process should be in a different order, or that there’s a step missing. This feedback loop can happen at a very early stage of the development, so eliminating later problems. The BPM diagrams are much more than just diagrams for looking at, though. Once signed off, they are then used as a compilable piece of the system. The diagrams are actually XML documents that are fed into the system at runtime. This means there is no room for mis-interpretation, no potential deviations from what was intended. The BPM diagram drives the whole of the integration in terms of this step follows this step. This means the stakeholders can have complete confidence that once they’ve signed off a process that will be what is delivered.
The main body of the development was completed in around six months by a team of two, one from InterSystems, and the other from Financial Objects, a partner in the health sector who was able to supply a consultant to work onsite. Since then Financial Objects has been invited back by the ICR to carry out extensions to the system to integrate with their Health and Safety system, and plans are in place to add a connection to The Institute’s security card system in the near future.
The new system is working successfully, and has resulted in improved accuracy of data, a reduction in the time taken over clerical data entry, and a more timely catching of information. The handling of people leaving the ICR after their project ends has seen particular improvements in terms of how quickly their email and network accounts are closed down.
The final verdict on the project is from Jeremy Harrington, IT Director of The Institute of Cancer Research.
“The project has been very successful in meeting its central objective of reducing administrative re-work by ensuring that data in disparate systems is consistent. No longer do we have the related problems of changes not being applied consistently or of data remaining on systems after a person has left. It is so much more efficient for everyone concerned. We are now extending our use of the product to other key data, to other systems, to wider aspects of the life cycle and to workflow applications.”
“We chose InterSystems Ensemble with exactly this in mind, as a strategic tool because of its very extensive and mature systems integration and application development capabilities. Not only has it met our basic needs for data integration and account provisioning, but going forward we now have a capability to deploy fully featured object-oriented database functionality, which is key to our work, as well as to deliver workflow and business process management applications.”
Comments