Suppose your sister has five children (mine does), and suppose she lines them up in size order (I have seen her do it) and then she awards a prize to the tallest (ok - now that seems a little unfair). But - if that was all that happened all you might say is "Hey I have a weird kid sister"- (sorry Maggs couldn't resist:)
But...suppose she tells all the parents at the next parent teacher evening that her oldest child was winning tallness awards - positively self-congratulatory right?
So do businesses practice auto-congratulation - I'm afraid so!
OK - So it sounds a little far-fetched, but I have seen an international hotel chain (One of the biggest) get excited that they had an extraordinarilty large number of hotels in the top decile of energy performance in the UK.
When I looked at the source of data (I really could not credit the results without digging a little deeper) I found that this same hotel chain was the source of over 85% of the source data in the national benchmark.
Suppose they had 85 hotels and suppose there were only 15 other hotels (not accurate). Then the probability of having one or more hotels in the top decile could be calculated as follows:
1-(15/100*14/99*13/98*12/97*11/96*10/95*9/94*8/93*7/92*6/91) = 99.9999999827 %
or put pretty simply - certain (or less than 2 : 10 Billion would they fail on a pure chance basis)
Now that was an auto-congratulatory benchmark - the sort of award that is so safe you would let it look after your children !
Why is this a problem... Well it extends to whole industries...
It expains why fridges are rated this way in the EU - So that it becomes hard NOT to be green or even dark green - look !
When you see an unbelievable performance - you'd better believe that's exactly what it is !
at kWIQly - we prefer a different way of looking at performance that avoids these problems