Compensation Surveys Are Biased
|
(dated September-October 1994) |
Fred Cook, Chair, Frederic W. Cook & Co., Inc.
After years of watching companies conduct and
use compensation surveys, I have become convinced that there is
bias in the way surveys are constructed, interpreted, and
usedparticularly at the management level.
Multiplied throughout the industry, the
cumulative effect of this bias has driven pay levels upwardabove
the "real" marketthereby contributing to layoffs and downsizing.
The latter occurs when companies realize they cannot and need
not sustain survey-inflated pay levels.
There are 12 common causes of this upward
bias. My purpose is to alert compensation professionals and
other users of surveys to this bias so they will be able to use
survey data more effectively in the future.
1. User Bias. Companies that
sponsor surveys often do so with an implicit (albeit unstated)
objective: to show the company as paying either competitively or
somewhat below the market so as to justify positive corrective
action. Those who are contracted to conduct the survey and
interpret its results on the company's behalf subconsciously
take on these same objectives. This user bias does not by itself
cause salary inflation, but it does create the climate in which
upward bias occurs and is accepted.
2. Sample Bias. Companies like
to compare themselves against well-regarded, high-paying, and
high-performing companies. Those firms that participate in
surveys drawing data from a large number of other organizations
often have the ability, through computer technology, to create a
subset of participants with whom they wish to compare
themselves. This typically results in a higher competitive pay
line than the general survey, thereby showing the user company
in a more favorable (i.e., less competitive) light in terms of
pay rates.
Alternatively, the survey may be custom
designed, in which case the sponsoring company will tend to
select a high-paying, high-performing group of companies against
which to compare itself. The survey designers will exclude
low-paying (and less well-regarded) companies. It is perfectly
natural to want to compare oneself against the star performers
in an industry; this reflects the company's goal to become more
like the leaders.
This would be fine if surveys compare
relative performance as well as relative pay. Most do
not. Actually, paying less than others may be quite appropriate
if the company's performance is low relative to the survey
sample. Likewise, high pay may be justified by high performance.
What happens, however, is that the low-paying
company will downplay its relative performance and use the
survey to justify pay increases, while the high-paying company
justifiably maintains its high position based on its relative
performance. The net effect, over time, is upward movement in
competitive pay levels.
3. Survey Selectivity. Most
companies have access to several surveys covering the same
population. In cases where different surveys show the company in
different competitive positions (some more favorable than
others), compensation professionals tend to disregard,
challenge, or downplay those surveys that do not show the
company in the desired competitive position.
4. Scope Bias. Survey
professionals accept the idea that the relative size of the
organization should influence its pay, particularly at upper
management levels. Yet "size" can be measured in any number of
ways: revenues, equity, assets, market capitalization, net
income, etc. Sponsoring organizations tend to select size
variables that let them compare themselves favorably to the
survey companies. Thus, if revenues are high but market
capitalization is low, they will select revenues as the
variable.
If every company in the survey has the
ability to select the size or performance variable that favors
itself, then it is technically feasible for all companies
to show themselves as paying below the market. When this
happens, what is the real market?
5. Compensation Selectivity. A
total compensation package is composed of many elements, whereas
most surveys tend to focus on a few, such as salary and bonuses.
It is natural for companies that have a competitive total
compensation package to be light on some pay elements and heavy
on others. Companies with very generous benefit packages,
supplemental executive retirement plans (SERPs), or large equity
grants tend to disregard or downplay those pay elements in
conducting surveys. If a company surveys only those areas where
it is light (cash compensation, for example), it should not
interpret or use those findings in isolation. If it does, it
will be raising its total compensation levels above the market.
6. Benchmark Bias. In
submitting survey data and interpreting the results, companies
must match their positions against positions in the survey. In
doing so, they tend to match their positions against those that
have higher responsibilities and hence higher compensation. One
company's positions may not have the full scope of
responsibilities typical of that position in other companies.
But the interpreter will tend to disregard this when comparing
pay levels. Conversely, if a company's position shows up as
highly paid relative to the survey, the interpreter will explain
that away by saying that the position has more responsibilities,
a different reporting relationship, or other factors that
justify higher pay. Since very few benchmark positions are
perfect matches, it may be quite appropriate to adjust survey
data for differences in position responsibility. But, if this is
done to explain and justify a higher-paid position, then equal
efforts should be made to explain and justify those that are
paid below the market.
7. Statistical Bias. Survey
professionals use a variety of statistical techniques to
interpret the survey results and determine where a particular
company stands against the survey population. Single and
multiple regression techniques are often used to supplement
straight statistical averages, medians, and quartiles. A sponsor
motivated by a desire to have the company appear in a good light
can select, from among the statistical techniques available,
those that will support this purpose. This problem will grow
worse as sponsoring companies become more sophisticated in using
option-pricing models to compare the values of equity-based,
long-term incentive grants. By manipulating the variables and
assumptions in ways favorable to the company, the interpreter
may be able to make an above-competitive grant practice appear
less generous and the grant practices of others appear more
generous.
8. Converting Actual Bonuses into
Target Bonuses. Many companies use surveys to assess and
reset their target bonus practices as a percent of salary.
However, most surveys collect data on actual bonuses, not target
bonuses. In times of strong economic expansion and performance,
it is common for actual bonuses to be above the target level.
Companies that rely on high actual bonuses reported in the
survey to justify raising their target bonuses lead the way in
escalating total pay levels over time.
9. Results Bias. Having a
strategy of paying competitively based on performance means a
company is obligated to offer a competitive pay opportunity,
not guarantee a competitive result. For example, a position may
have a target bonus payout of 30% of salary, set so that
the base salary plus the target bonus would result in
competitive total annual pay when performance standards are met.
However, there is no guarantee that the target bonus will be
paid or that total pay will be competitive.
Companies may overlook this obvious truism
when they have low or no payouts under annual bonus or long-term
incentive plans in comparison with others that have made higher
payouts, perhaps because of better performance. Low payouts
relative to the competition may be perfectly justifiable and not
a cause for corrective action. Surveys that include payouts from
variable pay plans should be interpreted with reference to
relative performance data. In the absence of such data, surveys
should focus on competitive opportunities, not results.
10. Misinterpretation and Normalization
of Special Equity Grants. Increasingly, companies make
special equity incentive grants to their key people. For
example, instead of making smaller stock option grants every
year, a company may accelerate option grants into a single,
jumbo grant. Or it may give executives a special, one-time
equity grant to signal a significant event, such as a new
strategy or a merger. Finally, when hiring a senior executive
from outside, the employer may use special equity grants to
induce the executive to take the job or to make up for benefits
lost in the job change.
These special grants more often than not are
included in the survey collection process. Without knowing the
reasons
for the grants, the survey interpreter may
assume that they are part of a normal granting practice. Even
knowing a grant is a one-time special event, the interpreter may
normalize it by assuming it will be repeated every three or five
years. The net effect is to raise the survey averages to new
levels and to create a situation where special grants by one
company become built into normalized practices by the survey
population.
11. Value Bias. Some employees
are true "value builders" and rewarded as such. Others are
"value maintainers," and some are, in fact, "value destroyers"
because they are paid more than they are worth. Surveys,
however, do not distinguish among these categories. When a
survey includes pay data from across the value spectrum, survey
averages are pulled up by the justifiably high pay of the value
creators. These averages, however, are used to rationalize
higher pay levels for value maintainers and destroyers as well
in order to be "competitive."
12. No-Decline Bias.
Compensation professionals have grown to expect that survey data
will show pay rates rising every year. A conceptual framework
does not even exist for handling marketplace declines in pay.
However, with all the layoffs, downsizings, and early
retirements of higher-paid workers, one would think surveys
would show declines in market pay rates.
Even if a survey shows a decline in pay
rates, the user will tend either to disregard or reinterpret the
data to reflect an increase. Also, survey purveyors know that
steady increases in competitive pay levels are good for
business, and declines are not. This, combined with user bias,
may explain why surveys continue to show increasing pay levels.
Correcting the Bias
If there were no compensation surveys, there
probably would be wider disparities among company pay practices
than exist today. These disparities would tend to be more
economically justifiable, both on the high and the low side.
Natural economic forces would operate to create a market in
which people would be paid according to their relative worth.
Surveys tend to narrow the disparities, particularly on the low
side, because low-paying companies use them to justify pay
increases that otherwise would not be justified. They tend to
have less effect on the high side because high-performing
companies can justify high pay and will do so rather than using
the survey to depress their pay levels. By pulling up the
bottom, surveys raise the average for everyone, thereby causing
an upward escalation in overall pay levels.
How can you avoid the negative effect of
bias? I propose three solutions. First, acknowledge that upward
bias exists in surveys and act to correct it where possible.
Second, downplay the importance of surveys use them as a check
on where you are, not as a rationale for doing something
different. And third, use relative size and performance
comparisons in a justifiable and consistent manner in
interpreting any survey results.
|