Measurement or judgment

by Dennis Howlett on April 25, 2006

David Maister has a thought provoking article on the future of management in which he proposes ‘we’ need better means of making judgments rather than relying on the obsession most professionals have with measurement. David’s argument centres around the notion we intuitively ‘know’ when things are working and when they’re not. I’d agree with that at a micro level.

Elsewhere, Scoble talks about a different way of dealing with compensation – suggesting a much more open and collegiate approach to remuneration management. Kathy Sierra, questions where decisions get taken and what information is needed to make smart decisions in a world where information overload is a reality – even after you’ve filtered stuff to death or become reliant on aggregators like tech.memorandum, Megite or any of the others that are popping up.

I’m torn on this. While I agree that measurement without context is pretty meaningless, I think we’re at risk of being tempted to throw out the baby with the bath water. Business intelligence is finally coming into the mainstream – witness CedarOpenAccounts and CODA, both of which recently acquired BI vendors. This coming hot on the heels of Sage launching Intelligent Reporting.

Judgment measures can be brought to bear in the more detailed analysis of what’s happening in the business. Here, measures like customer and employee satisfaction could be introduced as complementary performance measures to more traditional measures like profitability. But I think Kathy Sierra goes a step too far when she says new hires should be based on ‘curiosity, ability to learn and passion,’ rather than past experience. While I have no problem with incorporating some of Kathy’s criteria into a selection system, reliance on them as primary measures would qualify my dog to most middle management positions – if not that of top dog.

Bottom line – measures should reflect the less tangible, softer issues implied by current trends in social software. They have huge value in contextualising results. But don’t expect Wall Street, The London Stock Exchange or the DAX to be that impressed with measures for which there is no empirically satisfying number and for which a compliance framework does not (yet) exist. Because at the end of the day, whether anyone likes it or not performance always gets reduced to a number in the minds of those interested in wealth creation.

Technorati Tags: , , , ,

  • Share/Bookmark

Tagged as: GRC

  • That makes a lot of sense - so then we should be able to start with measures pretty much everyone can agree, followed by those we might need to negotiate and then finally those which are understood to be flexible in the sense that at this stage, we're not sure which will provide results on which managers can rely.

    That would allow for a discourse between multiple functions based on a mutual need to discover what works and what doesn't. That's a very different proposition to measuring against imposed targets (for example) that live in isolaton, detached from their context.
  • In the pursuit of better judgments that don't "diminish' the power of measurements, it occurs to me that good judgements frequently come from having available a variety of measures, each offering a slightly different perspective. Perhaps it is our pursuit of "THE" measure for each topic that causes the problems. If we could put in place reporting systems that offerred multiple perspectives on an issue, we'd be able to manage much better, even if any one of the measures was flawed.
  • I don't share that concern to the same extent. In my professional life, numbers were always the prelude to discourse. My sense is that many of the barriers to understanding or rather a start to interpreting or explaining the numbers can be overcome through traffic lighting reports or smart graphics. Crucially important to non-finance people.

    My concern rests with finding ways to develop 'judgment' tests with which the business can feel comfortable. For instance, Scoble's ideas around transparency in remuneration setting sounds great and appear to match your criteria for at least starting a judgment based system. But I can see many pitfalls. How do you for instance judge bias in opinion? How might you incorporate information that's qualitatively valuable at the same time as presenting the numbers? How would they usefully be combined? How are these information sources selected?

    We're only at the cusp of having technology that can help aggregate and filter content. That alone is significantly influencing my thinking across a broad range of issues. Including this one.

    In this context, I believe the technology will lead our thinking and so we need to understand the impact upon individuals and communities. As I'm sure you know, in our profession, this is at a very low level at this time. Getting more people into these discussions is an absolute pre-requisite.
  • Dennis, let's keep this discussion going, because it's a very important one. You make the point that Wall Street, the LSE or DAX will not be impressed by imprecise or absent measures. Fair enough.

    But I believe that an important distinction must be made between the relative importance of measures v. judgments if you are an outsider (ie an investor) or an insider (manager.) A numbers approach to investing may make more sense than a numbers approach to managing.

    This doesn't mean the corporation doesn't worry about ultimate financial results. It means that the key to the best returns is to focus on managing the things (client loyalty, innovations, employee engagement)that can be shown to improve returns. And managing these things means learning to judge, not just measure.

    Yes, the financial markets want performance, but the question still being debated is what's the best way to produce that performance? I don't think that, relatively speaking, it's our lack of skill in measuring that's holding us back. It's our lack of skill in using measurements as 'signals to be investigated' rather than as 'objective facts.'
blog comments powered by Disqus

Previous post:

Next post: