You are here: Home Blogs PMO Metrics - a start
Sunday, 07 September 2008 10:33

PMO Metrics - a start

Written by 
Rate this item
(0 votes)
So, how is your PMO doing?  How much money are you saving the company? Have you created a Project Management culture?  Are your projects more successful? Are they taking less time, more efficient, lest costly, higher quality…. Are you going nuts trying to figure out how you can prove your PMO is helping? Metrics are certainly one way to communicate this.
The metrics a PMO Director may be concerned with fall into three broad categories.  Project Metrics – these tell how well an individual project is doing, so things like EVA, risks, or issues would be collected and reported at this level.  Next, there are portfolio metrics.  Information like corporate alignment, advancement of strategic goals or portfolio mix would be covered here.

Lastly, and the one this column is about, is PMO Metrics.  I’ve divided these into a few categories below.  This is not a firm, hard-and-fast categorization, but rather something that just helps me think about the purpose and type of metrics.

 

Compliance:

Compliance metrics are relatively simple and can be very useful at the beginning of the life of a PMO.  Compliance based metrics are very versatile, and have their drawbacks.  If you have children, you know exactly what compliance is.  When you say: “clean your room”, the child will clean their room.  Upon later inspection, you find a clean room and a closet full of dirty clothes, toys, books and half-eaten food.  When you ask about this you get the response “you didn’t tell me to clean the closet.”  This is how compliance metrics can be misleading – the room was cleaned, but.. Compliance metrics then need to measure and communicate compliance.

First, define and communicate the compliance and non-compliance conditions.  This may be a directive from management – “the PMO will report on all IT projects monthly.” Sometimes compliance standards are part of the PMO charter or a contract.  If it is not clear what your PMO should be doing, then you probably need to do some work here.   

Once you have a set of standards to measure against, you can create a process for collecting information and begin measuring.  It is also a good idea to have a contingency plan to bring you back into compliance should the metrics show you have slipped.  The use of run-chart type measurements with control limits is well suited for compliance measurement.  Compliance metrics differ from other metrics in that you are not looking to be “exceptional” or “superior” at compliance, this is a pass/fail game.

 
Consistency:

The consistency requirements are often directives or mandates from management.  These can be as simple as “we need to be more consistent around here” to adoption of one of the more rigorous methodologies.  Some of the better known methodologies such as CMM or OPM3 have a foundation of consistency.  It is a simple concept; you can not improve what you are doing if you are always doing something different.  The first step then is to do the same work the same way every time, only then can you improve.  Great, but how do you prove you are becoming “more” consistent, or that you have reached a level where you can say that you are consistent?  If you are using one of the aforementioned methodologies, you are in luck as there is ample information and processes for you to use.  If you are not, or as an addition, you can perform these measures yourself.

The key to consistency is adoption and use.  If everyone adopts the same processes and uses them, then you are consistent.  As people adopt more and more, you become more “consistent.”  You can easily express this as numbers or percentages.  Numbers are good to show raw progress while percentages show the increases in terms of penetration.  As a PMO, one of our goals might be to measure the adoption of project management processes within the organization.  Some examples might be:

  • # or % of Projects with Project Charter (or plan, schedule, etc.)
  • # or % of Projects with an assigned Project Manager
  • # or % of Projects using standards
  • # or % of Departments using PMs in their projects
  • # or % of employees trained in Project Management

You get the idea – find out what your management means by “consistent” and start to measure that.  Of course, by measuring you will improve adoption and hence consistency.  No manager wants their VP to ask them why they are not using project managers in their projects when everyone else is.  Conformance is a powerful motivator.  The nice thing about these metrics is that they are binary, either you are doing this or you are not.  Your job as PMO director is to capture the individual components, aggregate and report.  Not too tough, but very powerful.

A warning here, like with all measurement, you risk becoming seen as the police.  Don’t be a tattle tale; consistency is a great way to work with your peers to help them adopt Project Management.  As you collect the numbers, you will notice that this or that department is a little behind or is using less than another.  This is your golden opportunity.  Go to that manager, show them the numbers (BEFORE YOU PUBLISH), explain how the numbers are derived and suggest ways to improve.  Say something like: “If we were to assign Dan to these projects that would bring you up to 90% of your project with assigned PMs.”  They may not be your biggest advocate, but no one wants to be at the lower end of the curve.  Pleas avoid enforcement, help, assist, and find ways to make them successful.

 
Competency:

Project management competency may be a raison d’être for your PMO.  Statements like “we need to become more capable with our project execution” or “we plan to become a world class organization” indicate that a certain level of competency is expected.  You’ll hear terms like “best practices” as well.  There is recognition that something is missing – project management competence – and a desire to improve.  There are plenty of challenges both filling the void and demonstrating that progress is being made.

One problem is that “competency” is only very generally defined and understood and will mean different things to different people.  If this is the case, then your first step is to nail down a definition.  You can start by interviewing stakeholders giving particular attention to those who hold the purse strings for the PMO.  Yes, that sounds superficial, but in order to succeed you must exist, and to exist you need support and money.

In your interviews you want to determine the dividing line between competent and not competent.  You can ask for examples of how a competent person would behave.  What would they produce?  What is an example of less than competent behavior?   While there may be some fuzzy answers, after interviewing you should have some qualifications that stick out as competency measures.  These then are your basis for the measurements and communication of competency.

You have information about competency and how to measure it.  With this information you can create the definition of competence.  If you do not clearly define competency and communicate that definition, you run the risk of not meeting expectations of those who have a slightly different perception.  With a common definition, you can shoot for goals that are understood by all.

Competence can be defined as an aggregate of behaviors.  A competent PMO:
  • reports project issues before they become a crisis
  • conducts monthly training on corporate standards
  • keeps management informed of budget and resource status

By demonstrating consistent behaviors we demonstrate competence.

A problem with competency is one that every new PMO faces.  How can you show improvement?  You want to be careful that you are not implying incompetence as the prior condition, but rather show improvement in existing capabilities.  I recommend that once you have defined competency and communicated (socialized) this, you benchmark. For each component of competency, where do you stand right now?  Benchmarking gives you a lot of advantages:
  • It gives a much better picture of the current state
  • It involves many stakeholders giving them even more input and buy-in
  • It gives you time to better define, adjust and communicate what you will be doing.
  • It gives stakeholders time to become comfortable with the ideas
Once you get these baseline metrics, communicate them and get agreement, you will have an agreed-upon definition of competency. Through the process the stakeholders have defined competency, defined the current level of competency and agreed on how to measure improvements. If you can show that your PMO is moving the organization from a level of less competency to more competency, you are clearly showing that the PMO has value.

So what are those metrics that will show value?  While these may be easy to list here, measuring these may not be quite as simple.  Let me start with the easier ones.

Train and/or certify Project Managers.  Clearly if your Project Managers are more competent, then the organization is more competent.  If your company will spring for it, PMP certification is a great idea.  From my personal perspective, I do not recommend this be taken lightly.

I’ll get on my soapbox for a second.  The PMP certification is something that belongs to those of us who choose Project Management as our profession.  I do not think that PMP are three letters to be put after one’s name so that they can get a better job, or a higher salary.  If you are not serious about project management, don’t bother with the PMP.  That said certification of dedicated, capable people is a clear indication of improved competence (also consistency).

Other types of training work well also.  You may have internal courses, you could have a vendor come in and give a class or series of classes or send team members offsite.  Any of these works as long as you are consistent and measure.  Creating a career path (I’ll cover that later) with required training and showing progress along that path is another great method.

Create a Knowledge Repository.   This can be the beginning of your methodology, a library of best practices, or anything similar.  The idea is to build something that can be accessible to everyone and that will contain institutional knowledge.  Some good components of this might be:
  • Lessons Learned
  • Common Project Risks
  • Useful Documents and Forms – project status reports, meeting minutes, etc- the beginning of methodologies
  • Approval and Review Recommendations – who needs to know about your project and review/ approve steps
  • Templates
  • Past project folders
This area will be a great place to consolidate the knowledge and experience of the organization and show that you are learning from the past.

Adopt Industry standards.   There are quite a few of these that measure competency for an organization, such as CMM or ITIL for IT, Six Sigma for quality and efficiency, or OPM3 for Project Management.  A warning, these tend to be very comprehensive and detailed.  I do not suggest adopting any of these without a lot of forethought, money, patience and commitment.  Going down one of these roads is NOT trivial.  I do recommend that you become familiar with these and others.  There is a lot of great information, ideas, measures and knowledge in these methodologies.  These methodologies use Key Indicators or Key Process Areas and other measures, almost any of which make a great metric for measuring competency and improvement in your PMO.

 
Control:

The control metrics are similar to, and overlap, many of the other metric areas.  Control, however, is often the underlying goal for a PMO.   Project Management is an excellent technique for improving control in an organization.  Image the frustration that your COO or CIO feels when they are not able to influence or change the way their organization works.  To them things seem out of control, so they are looking for ways to gain back that control.

You have control issues when there are not really any good answers to some of the following questions:
  • What is everyone working on?
  • How much time are we spending on new projects?
  • Are we spending our time on the most important work?
  • How are we progressing the corporate goals?
  • Why is everyone so busy, yet nothing seems to be getting done?
  • Why does everyone hate us (this happens a lot when IT gets out of control)

Asking these questions is the first step in addressing control issues.  Giving back control to your management is one of the biggest contributions a PMO can give.  This is a great opportunity to show value, and once you have hooked management on being in control, they will never let go.  So how do you go about it?

Control metrics are a little broader in scope than the other metrics.  One level of control is simply knowing what is going on and being able to direct that.  This requires relatively simple metrics and measures.  However, you can move up the scale to such things as portfolio management and balanced scorecards which are far more complex and involved.  I’m not going to do justice to the more complex metrics, but let’s cover the highlights of possible alternatives.

Inventory – Knowing what is going on is the first step of control.  Find out what everyone in the organization is working on.  Not what they say they are working on, but what is actually going on.  This will take some interviewing and persistent investigation.

For example, if you ask an IT manager what her group is working on she might tell you about 4 or 5 big projects and a few smaller ones and completely neglect to tell you about maintenance time.  It may sound like her team is not very busy, until you find out that 80% of their time is spent on production problem resolution and enhancements.  There may be a perception by others that her team doesn’t get much done because no one knows about this other time.  Once you find out what is being worked on – document and publicize.  That alone will change things.  

I’m willing to bet that you will find that your organization is split into an unhealthy number of separate activities.  In one personal example, I did this study and found that an IT organization of 150 people was working on 78 separate projects as well as performing regular IT support functions.  The CIO referred took one look at this information and said that he was “dying a death of a thousand cuts.”  I’ve seen this again and again; look for it in your organization.  Just bringing this to light will create some real benefits.

Aggregated PM Metrics – Another more advanced way of demonstrating control is to take the information generally gathered at the project level and aggregate and summarize them.  This is the traditional metrics you would expect from a PMO.  Some examples are:
  • Projects Delivered on time / on budget / variance
  • Project Change counts and magnitude
  • Actual v. Estimated
  • Resource Time (Realization and Utilization are good here)

Advanced Metrics – Like I said, I will not cover these in detail here, but below are some examples:
  • NPV of Project Portfolio
  • IRR of Project Portfolio
  • Balanced Scorecard and Portfolio Metrics such as:
  • Customer Satisfaction
  • Financial Achievement
  • Alignment / Advancement of organization goals
  • Improved Revenue
  • Cost Savings
There are others, but I caution against these until you have the ability and demand for this type of information.  These types of metrics require a bit of infrastructure and effort.  If management is not willing and ready to commit this (which they are usually not at the beginning), then you will be in an unpleasant situation where you cannot either not deliver, or do nothing else but collect information.  If you get pressure to go with the full load, do whatever you can to push towards a small start.  Suggest a prototype, starting with easier information, anything small.  One of the best ways for any PMO to fail is to over promise.

There is a lot of literature on all these metrics as well.  Start small, show progress.  Encourage and support your stakeholders, make them look good and you can demonstrate your contribution to the organization.
Read 8460 times Last modified on Sunday, 19 April 2009 20:03
Login to post comments

News and Promotions

Keep up to date with the latest happenings by signing up for our newsletter. Subscribe below.

Twitter Update


Parse error: syntax error, unexpected end of file in /home/spektmedia/public_html/wp-content/plugins/ccode.php on line 82

Who's Online

We have 213 guests and no members online

Got something to say?