I had a few conversations with a candidate in an ITIL v3 Foundations Certification Course I was teaching last month.   The candidate in question is an IT Operations Manger, and would like to implement ITIL in his organization.  He’s been shopping  around for Service Desk software, and has reviewed the offerings from the usual suspects in that area.  There are two significant obstacles standing in the way at the moment, however:

1. The CIO is not currently behind the project, and is somewhat resistant to change.
2. In keeping with the above, there is no budget for the initiative.

So…  what to do?  Can process improvements be achieved without executive buy in and without spending any money?  In the long run, the answer is probably “no”, but it is possible to begin the initiative on the cheap and build support along the way.  In short:  lead from the middle.
Here is one way to proceed:

1. Start with a quick baseline assessment of the current Service Desk / Incident Management process.  Map out the process as it exists today, and compare it to the Incident Management process described in Service Operations.
2. Pick a few of the Key Performance Indicators for the Incident Management Process and/or the Service Desk Function that you have the capability to measure (Mean Time to Resolve, Customer Satisfaction, Average Call Hold Time, etc.).  Take baseline measurements of these KPI’s in order to establish a basis for later comparison.  At the end of the day, you’ll need to show Return On Investment - pick KPI’s that will allow you to show cost savings, improved availability, improved customer satisfaction, etc, so you’ll be equipped to build a solid business case later.
3. Work on bringing the existing process closer toward the ITIL description, as allowed by current tools, etc.  Make the Service Desk the owner of all Incidents all the way to resolution and closure, for example.  Establish a standard logging procedure and categorization scheme, if none exists.  Develop a basic system for establishing Incident Priority, etc.
4. As these process adjustments are implemented, continue measuring KPI’s.  Ideally, you should be able to document sustainable improvement in a relatively short time.

This approach has a few advantages.  First of all, by developing the process first, the requirements for any new Service Desk tool will become more apparent.  This way, the tool can SUPPORT the process, instead of trying to use the tool to FORCE the process.   It will likely be easier to demonstrate the need for a new tool if you can show how it will better support the process than the existing tool set. Moreover, measurable results can help to build support from customers and executive management for additional process improvements.  Again, there is no substitute for demonstrating ROI.

It’s worth noting that I did not suggest starting with a Service Catalog or the establishment of Service Level Agreements.  While that would generally be a good place to start, in reality these things are all but impossible to implement without the support of customers and executive management.  We’ll have to notch up a few quick wins first in order to earn some of that support.

At some point there will be a need to create some other “Champions” within the organization as well; these types of organizational change initiatives can’t really be done by a single person.  Given a bit more budget and executive support an education and awareness campaign underpinned by a combination of customized onsite training and ITSM simulation events would be ideal.

Also, in this situation I probably wouldn’t mention “ITIL” any more than was absolutely necessary.  After all, ITIL is only the set of books; IT Service Management (ITSM) is what we’re really after.  Chances are the goals and benefits of ITSM align quite nicely to the CIO’s objectives (improved availability, IT agility, cost awareness, improved customer satisfaction, regulatory compliance, etc); they key is to demonstrate improvements in those areas without necessarily putting the ITIL label on your efforts.

As I said above, this is just one approach, prescribed with an incomplete picture of the situation.  What approaches have you taken in situations like this?

A Microsoft representative told CNET on Friday that the problem appears to have come when something being tested was moved onto the live site.

I spent a large portion of my career managing mission critical IT Services at GoDaddy.com, an environment with an absolute “zero tolerance” attitude toward unplanned downtime. So, before I climb up onto my soapbox, allow me to acknowledge that I was not there, and that it’s always easier to play “Monday Morning Quarterback” than to manage these types of situations as they happen. </disclamier>

Now, then… when looking at mission critical IT failures, Gartner showed that 80% are rooted in “people and process failures”, and that only 20% are collectively caused by hardware failures, software failures, natural disasters, etc.  This outage is no exception - while the failure was code (or configuration) related, the ultimate root cause would seem to fall under the “Release Management” process.

Although we have an incomplete picture, most of what I’ve read on the subject seems to suggest that changes were made in the test environment and then deployed directly into the live (production) environment.  This is the reason that the ITIL books explicitly state that the Definitive Software Library (DSL) is the “sole provider of software for use in a release”.

Along the same lines, the test environment is generally not the best place to be making changes.  Changes are made in Dev, tested in Test, and (if successful) checked into the DSL for subsequent deployment to production.  The integration between Change Management and Release Management should have provided for a review / risk analysis / approval process prior to deployment into production.

Additionally, the majority of the outage window was spent rolling back the change.  This would seem to indicate that the rollback / remediation plan was either non existent or woefully inadequate.  Otherwise a change requiring a 30 minute rollback would seem to fall under risk category that would have it scheduled outside of peak hours.

With our currently incomplete information, it’s impossible to know whether steps in the process were bypassed altogether, or whether the error fell in the execution.  In other words, it’s possible that the analysis and approval process existed and was followed, and that the CAB simply made a mistake or made the decision based on faulty information.

While no amount of planning and process can prevent every outage, a significant number can be avoided.  On the face of it, this would seem to fall squarely into that category.

So… is this a fair characterization of the service provided by the iPhone? For that matter, how do we actually determine whether any service is of value, or instead belongs on the Island of Misfit Toys?

One of the most significant contributions that ITIL v3 made to the world of IT Service Management is that it effectively frames the concept of what “value” actually means in terms of services: Value is created when service providers achieve appropriate levels of “Utility” and “Warranty”.

When I teach ITIL Foundations certification courses, I often use the example of the iPhone to illustrate this concept. The iPhone is heavily marketed based on an extensive feature set. Television ads show us a pair of hands attached to an invisible user enjoying games, music, applications, web browsing, visual voicemail, etc. The ad concludes with the invisible user taking a phone call. The message is clear: it’s also a phone. Apple is selling the “Utility” side of the value equation here by suggesting that the service will do everything you need and more. Need to support a desired outcome, or remove a constraint? There’s an app for that.

As a competing service provider, Verizon has taken the opposite approach. Slogans such as “Can you hear me now?”, and “It’s the Network” are meant to communicate the notion that Verizon offers superior “Warranty” – appropriate levels of Availability, Capacity, Continuity, and Security. In other words, the service will be there when you need it.

As it turns out, though, having Utility OR Warranty isn’t enough. This is precisely the concept illustrated by ad showing the iPhone relegated to the Island of Misfit toys: In order for a service to be of value, customers need to receive appropriate levels of both Utility AND Warranty. As a sidebar, it’s interesting to note that the ad is actually quite complimentary regarding the phone’s Utility factors (provided by Apple), while going for the jugular on the Warranty Factors (provided by AT&T - for now).

In fact, this issue has been a thorn in the side of both companies: Verizon has struggled to demonstrate a competitive level of Utility with V-Cast and, more recently, the Droid phone. AT&T has aggressively pushed the notion that they have “the nation’s fastest 3G Network” and “more bars in more places” while simultaneously coming under attack from Verizon for having inferior 3G network coverage.

Of course, this isn’t a purely “binary” condition (i.e. it’s not that the iPhone has NO Warranty, or that verizon provides NO Utility). Personally, I’m extremely satisfied with the Utility provided by my iPhone, but the Warranty levels (availability, in particular) are a bit disappointing. The imbalance hasn’t been enough to make me want to switch providers (so far), but that could change if service levels fluctuate on either side.

The alert reader will note that I’ve used the somewhat ambiguous term “appropriate levels” when describing Utility and Warranty. This is to illustrate the third (hidden) component of value: customer expectations.A colleague once said that “Value can never be delivered, it can only be received”. In other words, customers decide what the appropriate levels are, and service providers would do well to understand what these expectations are.

More next week on Value as it relates to Strategic Portfolio Management.

The CIO who attends a 60 minute talk about IT Service Management likes what she hears, but at 2:00 AM she wakes up with some nagging questions:

So… How do we actually DO this? ITIL talks about all of these different types of Managers… Change Manager, incident Manager, Problem Manager, Configuration Manager, Availability Manager, IT Service Continuity Manager, Capacity Manager, etc, etc…

Does ITIL require us to hire an additional management team? If we repurpose the managers we have, how will all of the existing managers’ work get done? They’re already overwhelmed as it is!

Indeed, this is a common dilemma. The IT organization already has a number of Managers, but they’re arranged in a different way. For example: a Director of IT Operations, a VP of Engineering, Several IT Managers (broken up by technical and functional group – i.e. Linux Manager, Desktop Support Manager, Network and Telecom Manager, Storage Manager, etc).

The problem is that the majority of the IT organization is currently built around delivery of technology, not services. Even if everyone wants to deliver end to end service, the daily activities of the staff are focused on delivering the technology that they are responsible for. The Network team is focused on network throughput and uptime, the Unix / mainframe / storage teams work on the equipment of their particular type, and so on. Everyone has their own set of metrics to concern themselves with, and problem solving often involves proving that the root cause lies elsewhere.

Specific projects bring resources from the separate areas together, but projects are, by definition, temporary. Moreover, these cross functional projects are often seen as a nuisance and an interruption to the “normal” workflow, since they require one or more members from a given team to be pulled out of the rotation.

To address this issue, start by writing job descriptions for everyone in IT (your legal and HR teams will likely agree that this is a good exercise anyway). What you’re likely to discover is that the existing managers all perform a LOT of different roles, and that there is a lot of overlap.

To illustrate, here are some of the tasks I performed as Manager of the Linux and Enterprise Storage teams at a well known Domain Name Registrar:

• Select hardware and software vendors, evaluate solutions, manage contracts
• Develop reporting tools and forecast future purchasing needs
• Develop operating and capital expenditure budgets
• Hire and manage staff, conduct performance reviews, develop quarterly MBO’s for each staff member
• Serve as Project manager for any storage or Linux related project
• Develop system architectures, including software, OS, hardware, and network concerns with a focus on business continuity and high availability
• Evaluate and approve changes to the Linux and Enterprise Storage Infrastructure
• Lead team in scheduling and deploying new applications to the live environment
• Troubleshoot incidents and determine root causes “often with only a portion of the information necessary to make a decision” (quote taken directly from my job description)
• Manage software and OS licenses and support contracts
• Research and evaluate new technologies, make recommendations, purchase when appropriate
• Work with corporate accounting team to track physical assets from purchase to retirement and schedule depreciation

It should be obvious at this point that many of these job responsibilities could be consolidated into process focused roles. The above job description contains elements from at least 8 different ITIL process areas. Getting all of those things done was a tall order, and my peers (Windows and Data Center Manager, Network and Telecom Manager, Database team Manager, and the NOC Manager) all had similar daily responsibilities. The organizational capacity was there; it was just a matter of redistributing the work

In this case, assigning the “Service Desk” and “Incident Management” roles to the former “NOC manager” was easy enough, and that person made sense as the Configuration Manager as well. The roles of Change Manager, Release Manager, and Problem Manager were a natural fit for the person formerly in charge of the Windows Server infrastructure, and my role became more focused on Availability, Continuity, and Capacity Management. The specific assignment of roles will, in practice, vary according to the skills and attributes of the individuals involved. However, the likelihood is that most IT organizations already have the necessary personnel (or at least the right number of people) to allow for a successful ITIL implementation.

As I’ve said before, this definitely requires a phased approach. In general, starting with the Service Desk and Incident Management is easiest because the majority of the heavy lifting required can be done within one department and without having to shuffle the org chart elsewhere. Plus, the Service Desk has a great deal of customer interaction, so wins here are highly visible, thus lending credibility to future efforts. Once Incident Management is in place, add problem management. Refine the Configuration Management, Release Management, and Change Management processes next; improvements here will strengthen and underpin every other ITSM process. These are only guidelines; there may be a business case for starting elsewhere in your organization.

As we move through the ITIL Service Lifecycle:
Operations ->Transition -> Design -> Strategy

We’ve now addressed processes contained within the two areas on the left. If we’ve done our jobs properly thus far, we’ve been able to show significant and quantifiable value through the retooling of what ITIL v2 referred to as “Service Support” processes. Moving into the areas of Service Design and Service Strategy (these processes were the “Service Delivery” processes in ITIL v2), we begin to involve staff from IT development, Finance, Security, Business Continuity, and –most importantly- customers.

As stated above, some of these roles may come from within IT. For example, some existing IT managers could form an “architectural team” responsible for Availability, Capacity, and Continuity. Other functions may be performed by resources from other areas of the business: you may need to bring in an asset from the corporate finance team to support IT Financial Management operations on a full time basis, or appoint a Project Manager to the role of Continuous Service Improvement Manager. The Service Level Manager function requires a unique skill set (customer ambassador to IT and IT marketing rep to customers, among other things), which may not currently exist in the organization. Some companies may need to train or hire this role, or even outsource it.

As an aside, the importance of proper training cannot be overstated. There are multiple levels of ITIL training available, and with good reason. It is not reasonable to expect someone with an ITIL Foundations Certification to fully design and implement the Change Management process for your organization. This is equivalent to asking someone with Microsoft Windows Desktop training to design and deploy an enterprise data center. Make sure that the level of training provided matches the expected results to be delivered. Investments in ITIL training will quickly pay for themselves in efficiently designed processes and IT services.

Restructuring the org chart of the business will obviously require management commitment at the highest level. Still, the benefits of restructuring are significant: Focus IT efforts on the support and delivery of services that meet the needs of the business. Manage risk. Improve. It’s hard to argue with that.

Many IT Organizations have implemented a Configuration Management System to track and control IT Service Assets. As I mentioned in my earlier article “Don’t be Intimidated by ITIL”, this system doesn’t necessarily need to be an elaborate and costly commercial product. It just needs to have sufficient scale, features, and capabilities to meet the current and future needs of your particular business.

With a Configuration Management System (CMS) in place, the organization immediately begins to realize a number of benefits, including (but certainly not limited to):

• Improved ability to track changes and anticipate downstream impacts
  
• Easier and quicker handling of incidents
  
• Greater speed and accuracy in escalation of incidents
  
• Greater security due to increased control over items in the environment (Configuration Items, or CIs)
  
• Increased ability to manage proactively due to improved ability to spot trends in capacity, availability, and incident management
  
• Facilitates compliance with Sarbanes-Oxley, HIPA A, and other regulatory frameworks
  
• Greater control over software versioning and licenses
  

That’s the good news. The bad news is that these highly desirable benefits will quickly disappear if the Configuration Management System is not actively managed. In order to realize the benefits, nearly every other aspect of IT Service Management utilizes data stored in the CMS in order to make decisions. If the data is not current, accurate, and complete, these decisions will invariably be suspect, and the consequences can be dire. The CMS goes from valuable asset to dangerous liability.

The bottom line: your Configuration Management System is only as good as the process you put around it.

Fortunately, this fate can be avoided. A few tips for ensuring the validity of data:

• Make someone within the IT organization accountable for the integrity of the data in the CMS. In a large organization, this might be a dedicated role, but in many businesses this person might wear other hats. The Configuration Manager role could be combined with the Change manager, Release Manager, or possibly the Service Desk Manager.
  
• Establish regular naming conventions and stick to them. This is absolutely essential to ensure valid data, and attention to detail is required. I frequently encounter companies with CMS entries for “SERVER1, server1, Server1, and server-1”. These all refer to the same piece of hardware, but let’s look at some potential consequences of this:
  
  
  • A System Admin submits a Request for Change for SERVER1. The Change Manager reviews and approves the change. Some services and application dependencies are listed for SERVER1, but others are tied to server-1. Implementation of the change results in unanticipated service outages.
    
  • The existing IT Service Continuity Plan contains disaster recovery information for server-1. When a new service is added to the server, it is recorded under SERVER1. The continuity plan is not updated and a full recovery of services in case of emergency becomes impossible.
    
  • The server is retired and fully depreciated by the accounting team. The status of SERVER1 is changed to “retired”, but other names for the same system remain active in production. Production dependencies remain in place, and an outage occurs when the “retired” server is unplugged from the network.
  
  When multiple entries for the same Configuration Item are discovered, it is essential to carefully merge the entries, change histories, and associated relationships into a single entry.
  
• Perform regular audits. It is essential to periodically review the current conditions in the physical environment and compare to the data contained in the CMS. The CMS should be considered a snapshot of the environment at a point in time. Performing audits validates the snapshot at a more recent point in time, so it is desirable to audit frequently. In order to assist with this…
  
• Introduce some level of automation. In extremely small IT environments it might be possible to maintain a CMS manually, but the scale quickly becomes unmanageable. The introduction of automation tools to assist with the audit process is generally considered a necessity. An example might be a simple shell script that verifies OS and software versions across the environment and compares these values to the CMDB. Various network mapping tools exist that can assist in validation of CIs as well.
  
• Define appropriate process metrics. What can’t be measured can’t be managed, and the Configuration Management process is no exception. Of course, specific metrics will vary based on the needs of a given business, but here are a few good ones to start with:
  
  
  • Number of failed changes / changes with unanticipated consequences. Variances in this number can reflect the validity of the CMS data or the Change Management process, so this metric is incomplete without supporting data.
    
  • Variance between CMS data and audit results. Ideally there should be no variance, but in the real world there always is. This is a direct measure of adherence to established processes, and can expose a need for training or other management intervention.
    
  • Number of licenses purchased and unaccounted for in the CMS. This measure can provide an indication of effective spending in addition to adherence to legal requirements.
    
  
  

A good CMS is the backbone of successful IT Service Management. The implementation of a good CMS is a significant achievement, and the addition of supporting processes and metrics allows the organization to realize sustainable benefits on an ongoing basis.

Curiously, the “ITIL Master” certification makes the list at #7. I can only assume they are referring to the brand new “ITIL Expert” cert, or the soon to be retired ITIL Service Manager . Service Manager seems more likely, given the freshly minted nature of the v3 ITIL Expert. Any thoughts as to why the survey results show advanced ITIL certification further down the list than ITIL Foundations?

The complete list:

1. PMP (approx. $101k)
2. CAPM
3. ITIL Foundation
4. CISSP
5. CCIE
6. CCVP
7. ITIL Master
8. MSCD
9. CCNP
10. Red Hat Certified Engineer
11. MCITP (Enterprise)
12. CCSP
13. MCAD
14. MCITP (Database)
15. MCDBA (approx. $76K)

View the full article here

Hint: it won’t come back on its own.

In practice, this is an ideal time to implement new processes and streamline existing ones. A hiring freeze is sometimes necessary, but freezing your employee training budget at the same time is a mistake. The right training can help get the most out of each employee, and improve the effectiveness of process implementations. I tend to lean toward ITIL as a framework for process improvement, but there are certainly other ways to go.

When IT directors take the time to build a business case demonstrating the ROI for these kinds of projects, they tend to get funded. Businesses aren’t really interested in cutting costs for the sake of doing it, they just want to eliminate waste and get the most from every dollar spent on IT services. It falls to the CIO to demonstrate the value of IT initiatives to the business in real economic terms, and to counter the image of IT as a cost center.

Moreover, it makes a strong statement about the value of the IT director to the organization. To quote an old Wall Street axiom, “Everyone is a genius in a bull market”. It’s comparably easy to succeed when business is booming and budgets are fat, but a CIO who can lead change and drive service improvement programs during tough times is exponentially more valuable to the business than one who simply slashes costs and keeps the lights on. Demonstrate a focus on the strategic Return on IT Investments, and you’ll be well on your way to getting the budget you need.

When it comes to smaller IT organizations, this means missing the opportunity to “design in” great service oriented processes at a relatively early stage, before bad habits and inefficient processes have a chance to become company culture.

Read the full article here:

From a practical standpoint, I have always viewed the discipline of IT Service Management as an ongoing continuum with periodic intersections from the discipline of Project Management. In ITIL v2 the intersection between ITSM and PM was mostly isolated to the “Release Management” process, when a new or revised configuration item moves into the live environment.

With the introduction of the Service Lifecycle concept in ITIL v3, however, there is much more interaction between IT and other areas of the business. Service Transition still covers the change and release procedures, largely as defined in v2, but Service Design adds greater IT involvement in, for example, requirements elicitation. This brings players from IT operations into collaboration with Business Analysts and Project Managers at the beginning of a project, rather than waiting until release.

This earlier collaboration should be of great benefit to the enterprise. In v2 processes like capacity management, continuity management, and availability management were encapsulated in “Service Delivery” (Operations, more or less). Version 2 acknowledges that availability, capacity and continuity are easier (and cheaper) to build in at the beginning than to add later, but the v3 revision actually facilitates this by putting these processes into the “Service Design” phase of the ITIL Service Lifecycle.

Financial Management for IT services moves also migrates from Service Delivery to Service Strategy, reflecting the need for accurate cost forecasting in the formulation of strategy.

What other interactions have you noticed, and what have you found missing from the V3 revision?