Procecedures

Information Systems

Change Management and Control

This section presents the concept of change management and control within the computing environment. It describes: the reasons and objectives for the system and general flow of the system.

2.0 Reasons for Change Management

Our mission is to provide the [MSC Client] Offices with accurate and timely information in a reliable and consistent manner that they can count on time after time. Changes to the operating environment allow opportunities for errors that may disrupt our ability to provide high quality services to the [MSC Client] Offices. Managing those changes can help reduce the introduction of errors. As a result, this procedure is a process to manage changes within the [MSC Client] Data Center and its potential impact to the [MSC Client] Offices.

A formal control process is necessary to bring change plans together in a forum that is conducive to effectively disseminating information to those who are potentially affected by a change. If done consistently and conscientiously, this process can ensure that changes are implemented in a timely fashion with little or no adverse impact on the [MSC Client] Offices.

Change Management provides a means by which a historical trail of changes made to the system can be kept. This becomes valuable when the relationships between system failures and/or performance problems must be correlated and analyzed.

3.0 Change Management Process

Change Management is a vehicle for the authors of change to communicate their change implementation methodology to management, implementers, associates, and users. The process is intended to communicate both planning and implementation information.

3.1 Objectives

Specific objectives of change management are to maintain and improve system reliability; availability; serviceability and functionality.

3.2 Scope

Specific change tasks associated with the above areas are detailed in the [MSC Client] Change Request Lead-time Matrix in Appendix B. The scope of changes managed by this process may broaden or contract as the Change Approval Committee deems necessary i.e., a particular type of routine change is developing an unacceptable failure pattern or that a controlled change is becoming sufficiently routine that change management no longer appears necessary.

3.3 Overview of the System

In the request process, the change author documents the specifics of the change via the Change Request Form (see Appendix A) and submits to their department management for preliminary approval. All Change Requests must be recorded and tracked via the Help Desk System.

The planning step lays out the specific tasks, sequences and responsibilities that must be completed for a successful change. Prerequisites and dependencies are identified and specific times are also indicated where tight coordination is necessary.

The approval process is intended to verify that all of the steps defined in the request process have been carried out and are appropriate in view of the risk and impact to the organization.

In the schedule process, all approved changes are scheduled on a master activity schedule to enable all effected resources to be aware of the activities planned for the production environment. It also allows conflicts of resources and major impacts to be reviewed and revised, when necessary.

The implementation process establishes a mechanism in which changes can be applied in an effective, high-quality manner, and enables the change to be entirely removed if necessary without adversely impacting the system’s ability to perform in the same manner as before the change.

The review process assesses the successful implementation of the change. Was the change completed on time? Completed correctly? What if any problems were encountered? What processes and/or procedures may require modification to ensure successful completion in the future?

4.0 The Change Approval Committee

The Change Approval Committee is vested with the mandate of ensuring that proposed changes are planned, coordinated, tested, executed and verified effectively; and that business, operational and information security risks are appropriately identified and documented. This committee is the final approval body for all changes at [MSC Client].

The individuals selected to sit on Change Approval Committee are selected for their in-depth perspective and broad knowledge of the areas they represent as well as their appreciation for other functional areas involved. This degree of participation is necessary for making objective decisions on the most significant changes affecting the organization.

It is important to separate the change requesters from the change approvers in order to avoid natural conflicts.

If a committee member delegates their duties to a substitute, that substitute must be knowledgeable in the committee member’s area of responsibility and have authority vested in them to allow the delegate to perform the normal duties of the committee member.

4.1 Change Review Meetings

Change review meetings are held daily. The Daily Change meeting will be held at noon , to review the changes scheduled to be performed that night and/or the following night, depending upon the level of impact. Please refer to (Appendix B) for Change Request Lead-time requirements about a particular type of change.

The following types of changes will be reviewed during the daily change review meeting:

The Change Approval Committee meets on a weekly basis to go through the final approval process for upcoming changes. The standard list of changes for this weekly meeting includes but is not limited to:

The Change Approval Committee will also review all changes performed the previous week to ensure completeness of the change management process and identify areas of opportunity to improve the change management process, testing procedures or documentation processes.

The Weekly Change Approval Committee meeting will be held on Wednesdays. Change Requests for these Wednesday meetings must be submitted to the Change Coordinator by the previous Friday. Work will typically be performed on the following weekend. Please refer to the Appendix B for details regarding lead-time requirements for specific change types.

The change schedule, which is to be used at the Change Approval Committee meeting, should be produced in sufficient time prior to the meeting to allow the committee members an opportunity to review information with their respective staff for comments and recommendations. It is during this process that much of the technical assessment or behind the scenes activity takes place. Sufficient lead time will help ensure that all necessary coordination among the various parties involved with the change can be accomplished before the request is approved.

Once the committee has approved, disapproved, or suspended a change, the change schedule will be updated to reflect the new status. That information is then communicated to the requester and to the interested parties. If the change is approved, the requester may proceed with the change implementation on the schedule documented in the approved Change Request. If the change is disapproved or suspended, the request should be returned to the author for further action as required. This could be a recommendation for a new date; a request for additional information in further support of the change; or a request for improvement to the implementation and/or back out procedures.

It is the responsibility of the requester to recycle his change through the request and approval process again. In the case of a campus-generated change, the [MSC Client] Change Control Coordinator will take responsibility for communicating with the campus on the status of their particular change request.

4.2 Working Documents

Change Approval Committee will decide on what action should be taken with each change request. The Committee members should have available to them the following documents describing change activities:

· A current change schedule identifying all current and future changes that have been requested.

· Change Requests to be reviewed in those cases where more documentation is required to reasonably evaluate the change.

After discussing the requests, the Committee may approve; disapprove; place a change in a pending status indicating that the change itself is approved but not the proposed implementation date or method; or place the change on hold pending future considerations.

4.3 Approval Criteria

State of the production environment: Before determining if a change should be approved, the Committee subjectively (objectively?) evaluates the performance and availability of each system in the production environment during the past week. In general, if the production environment has performed well and has been available to the users/[MSC Client] Offices, the Committee is more likely to approve changes that provide new functionality or changes that might have a high risk of failure. Conversely, if the state of the production environment during the past week has been poor, the Committee is more likely to approve only those changes designed to correct problems.

Change level: As part of the approval process, the change level is examined along with the detail information and instructions attached to the Change Request. The attachments should detail the associated risk and impact of the change. Particularly important are the subjective comments provided by the change author indicating the reasons for the assigned change level.

Aggregate effect of all proposed changes: The Committee is the single place where all of the changes requested for the week come together. For example, the Committee has the ability to examine several changes, each of which may appear to carry a reasonable risk if taken independently; but when all changes proposed for a particular week are considered as a group, the composite effect may result in too much change activity, hence risk. When the Committee reaches this conclusion, their responsibility is to prioritize the various changes, approve the most significant ones, and recommend that the others be re-scheduled.

Resource availability: The Committee should be concerned about the availability of people, time, and system resources when considering the scheduling/approval of changes.

Criticality: There are issues that may alter the impact of the change as viewed by the author. For example, the change author may feel that the impact is relatively low because his change affects a small percentage of the user community. However, the Committee may judge the criticality to be high because that small percentage includes a particular user that it feels has a critical need.

5.0 The Request Process

5.1 Definition of Level Factors

It is the responsibility of the technician or author of a change request to evaluate it against six predetermined categories and assign the Change Level. While the change author normally has a technical perspective with respect to these categories, a business perspective must also be applied when assigning a level.

Risk considers the probability for success based on the difficulty and complexity of the implementation and back out procedures. The questions that need to be asked are: how certain are you that the change will be implemented successfully the first time; and if the change fails, how sure are you that the fall back procedures will return the system to the same state prior to the change?

Impact analyzes the overall impact to the organization based on machine, fiscal and people resources. The questions to be asked include: how many people/[MSC Client] Offices are affected by the change; how much machine time is involved; what is the budgetary impact of the change; and how easily can back out be achieved?

Communication requirements takes into account how many [MSC Client] Offices must be notified of the change and what the logistics are for adequately notifying the affected parties. Can you convey the message with enough time for the recipient to react accordingly? Those responsible for authorizing change must consider the communication requirements.

Install time considers the overall amount of time it takes to prepare the change, implement the change, and recover from a failed change. In other words, if the change takes so long that it cannot be removed with sufficient time to return normal services to [MSC Client] Offices, it should be leveled high, or be re-worked into a more flexible size.

Documentation assesses the degree to which procedures must be amended to adequately describe what has changed. It is imperative to ensure that systems, operations, and clerical documentation are properly updated.

Education/training needs considers how significant an impact the change will have on those using or operating the system and what it will take to reasonably expect them to adapt to the new situation.

5.2 Assigning Change Levels

After performing this evaluation, the change author assigns a single designation to his change: Level 3, Level 2, or Level 1.

Level 1 is the most significant change, impacting a large portion of the user community or [MSC Client] Offices, and/or they are highly disruptive. The back out process is not automatic and requires technical expertise to implement. On site support is required for implementation. Examples of Level 1 changes are: changes to the operating system or major subsystem; application upgrades; changes that are complex to implement. Level 1 changes require a Change Request with two (2) weeks advance lead time, and approval from the following: the originating department Director; and [MSC Client] Office Representative (the [MSC Client] Office Project Director or his/her delegate); and the Change Approval Committee.

Level 2 changes are originated by an application or system user. They require coordination among [MSC Client] Departments, and/or the [MSC Client] Office and have the potential to disrupt a substantial number of users. Examples of Level 2 changes are: new application releases; upgrades; configuration changes; hardware changes; restores; or projects requiring cache clearing; or any change that [MSC Client] deems warranting extra control. They include detailed implementation and back out planning. Level 2 changes require a Change Request submitted according to the lead-times specified in the Change Request Lead Time Matrix to allow for planning, scheduling and approval. Approval will come from the following: the originating department Director; and [MSC Client] Office Representative (the [MSC Client] Office Project Director or his/her delegate); and the Change Approval Committee.

Level 3 changes occur daily or on a very frequent basis and are normally non-disruptive or administrative in nature. The impact of failure is either highly unlikely or minimally limited in scope. Back out is readily available and reliable. Level 3 changes do not require Change Control paperwork or Change Approval Committee approval. They require approval only from the originating department Director. Does this include one time changes (e.g. updates to database information). If so, some sort of formalized approval and tracking procedure needs to be implemented.

The risk and impact factors are the two most critical in developing the change level, and the authors of change are advised to weigh these two factors most heavily.

5.3 The Change Request

Once the change level has been established, the change author should communicate the change through the use of a Change Request form. See a sample of a Change Control Request form in Appendix A. These forms are to be filled out electronically and printed for approval signatures. Other forms required as part of the Change Control package include the appropriate Hardware Inventory forms when hardware is being installed, removed, or moved to another location. Change Request forms are generally not the responsibility of the Requesting campus.

5.4 Change Request Lead Times

Requests for most changes must be submitted to the Change Control coordinator according to the Change Request Lead-time Matrix (Appendix B). This lead time will give the coordinator enough time to complete a schedule of proposed changes; distribute the schedule to all group leads and directors and allow the various groups time to communicate with each other regarding coordination or conflict prior to the Change Approval Committee meeting.

In some instances, changes may be submitted for “tentative” scheduling when they are pending campus approval, or when management is considering business impact prior to approval. These changes will be “tentatively” scheduled, presented at the Change Approval Committee meeting, and considered for any possible conflict with other scheduled changes, and must be clearly identified as ‘tentative’ by the requestor to avoid having the change rejected due to lack of appropriate approval. The original Change Request with all appropriate signatures must be received before the change will be definitely scheduled for implementation. If the necessary approvals are not received the change will be removed from the activity schedule.

Changes not received within the specified timeframes outlined in Appendix B will be considered “Late” and will be scheduled for the following weekly or daily change meeting as appropriate. “Late” requests that must be implemented during the current week will be classified as an “Emergency Request” and will require additional approvals; this information will be logged on a weekly and monthly basis and reported on through the Change Management status reports to all [MSC Client] Directors.

5.5 Tracking Requests

A ticket is required to be opened with the Help Desk for all Change Requests. Generation of a Change Request resulting from a campus initiated service request will be the responsibility of the [MSC Client] department to which the service request is assigned.

5.6 Disaster Recovery

Disaster Recovery processes are critical to the data center environment, and as such require visibility in the change process. Any change that will alter the current disaster recovery process or documentation must be identified on the change request, and updated documentation provided to the Disaster Recovery Coordinator.

Disaster Recovery Services will review all changes being implemented and copy those that have a potential to impact Disaster Recovery. They will perform a follow-up each Monday to determine if the change was successful, and if so, request that the update to the Disaster Recovery documentation be made within one week. A secondary follow-up the following week will be performed to ensure that the documentation has, in fact, been updated. A trouble ticket will be opened in the event no update has been made to the documentation within the timeframe requested.

6.0 The Approval Process

6.1 Preliminary Approval

Preliminary approval of a Change Request will be completed at three levels prior to being submitted to the Change Approval Committee for final approval and scheduling. It is the change author’s responsibility to obtain approval.

The responsible department’s director or delegate will complete the first approval. This first level of approval will consist of ensuring that testing was completed; modules are ready for production; implementation procedures are provided; the back out processes and procedures are provided; the appropriate date and time have been scheduled for the change; that the request has been filled out completely; and that the appropriate integrity assurance measures have been taken.

The responsible [MSC Client] Director will do the second preliminary approval and [MSC Client] Office Liaison who will review the request for any affect the change may have on Data Center resources; [MSC Client] Offices; or processes.

The Change Control Coordinator who will review the Change Request for compliance with procedures will do the third preliminary approval. A Change Request form will be returned to the author if it is not filled out correctly or completely, if it is unreadable, or if the proper approval signature(s) has not been obtained. The need to return a change request to the author can cause a delay in approving and implementing a change.

6.2 [MSC Client] Office Acknowledgment

In an effort to ensure that our users are aware of and prepared for changes to their environments hours of availability, client approval is requested for changes specific to [MSC Client] Offices operating environments. This approval is not always required and will be reviewed on a request-by-request basis. The requester may obtain an email approval or a faxed copy of the request with a client signature, which Change Control Coordinator will attach to the original request.

6.3 Final Approval

The final approval process is conducted at a Change Management Committee meeting comprised of the responsible representatives from [MSC Client], Network, Help Desk Manager, [MSC Client] Office Representative and Change Control Coordinator. The Committee will use documented approval criteria when making their decision (see section 4.2).

6.4 Change Schedule

A preliminary master change schedule will be distributed daily prior to the Change Review meeting to allow each representative time to review the schedule for any conflicts or unidentified impacts. A final master change schedule for the upcoming approved activities will be completed and distributed to all affected parties.

7.0 Emergency Changes

Emergency changes will be classified into one of two categories: “Priority” and “Out of Guidelines.” All Emergency changes require approval from: [MSC Client] Directors, Delivery Manager and [MSC Client] Office Project Director or PD delegate prior to implementation of the change. It is the responsibility of the Change Control Coordinator to escalate and procure all required approvals in an effort to implement the change as expeditiously as possible. Emergency changes do not require review in a Change Approval meeting prior to implementation. All Emergency changes must be fully documented; due to the nature of the request this will often occur after the fact.

Priority: Emergency changes required to fix production problems affecting campus critical business functions. These requests must be completed as soon as possible and cannot wait until the next scheduled change window.

Out of Guidelines: Changes which are necessary to support critical campus business needs which were not requested within the established lead-time (See Change Request Lead-time Matrix, Appendix B).

8.0 Review Process

As part of the Change Management process, all changes will be reviewed post-implementation. This process is necessary to ensure that:

The Change Coordinator is responsible for assessing the successful completion of the request. (Shouldn’t the change approval committee review all changes to facilitation of improvement in process?)

Past changes that were unsuccessful or canceled for any reason will be discussed at the weekly Change Approval Committee meetings, and a status report will be generated each week following the weekly meeting and distributed to management for review.

A monthly status report, which summarizes the entire month’s activity and status, will be generated during the first week of the following month and distributed to management for review. Year-to-date statistics will also be maintained each month and distributed with the monthly status report.

Skip Stein

Business & Technology Consultant

Business Information Technology ~ Intelligent Home Design & Automation

Table of Contents

1.0 Procedure Description