Be wary of service comparisons between major local telephone companies

Various news accounts have cited a Federal Communications Commission survey of the quality of local telephone service, and have drawn conclusions about the relative quality and long-term trends in the service of Bell Atlantic and NYNEX.

Some opponents of the Bell Atlantic -- NYNEX merger have cited this study as well.

News editors should be aware of a basic problem with drawing comparisons from the FCC survey results:

The FCC itself notes that the findings should not be used to compare the service of one company to another's because the companies use different standards when they make their reports to the Commission. Furthermore, the FCC warns that the data are not even reliable for a single company over time because the companies have changed survey techniques and the questions themselves.

Here are some of the disclaimers from the preface of the FCC's "Update on Quality of Service for the Local Operating Companies" (March 1996):

• Maximum and minimum values ... suggest possible internal differences in data collection and processing (page 4).
• In evaluating the data one should first note that the FCC itself does not impose standards ... one should be cautious in making premature judgments (page 1O).
• One should be aware of the potential pitfalls in using of (sic) the quality-of-service data presented here (page 13).

Although the Commission has attempted to standardize the data requirements among reporting companies, one should not be lulled into the assumption that comparable data items for different companies are exactly the same. Different companies may have different procedures for collecting and presenting the data that may affect the quality and meaning of the data provided to the Commission... caution should be exercised when attempting to make direct comparisons ....

Relating to this is the problem of continuity of measurement ... the companies themselves periodically wish to change their internal measurement procedures from which regulatory data are drawn, adding difficulty to long-term measurement (page 15).

If the FCC cautions against the comparability of the data in its own service quality report, what's a journalist to do? There are other, independently conducted surveys that go directly to customers to measure their perceptions of products and services, asking them a standard set of questions.

"Fortune" reported on one such survey in its December 11, 1995 issue.

Referring to "one of the most comprehensive customer satisfaction studies ever done," the article, Americans Can't Get No Satisfaction, discussed the findings of the American Customer Satisfaction Index, a joint project of the University of Michigan business school and the American Society for Quality Control. According to Fortune "It is based on a random sample of the actual users of 3,900 products and services ... statistically significant and representative of national trends ... Roughly 30,000 people participated in the telephone surveys in each of two years; every company was assessed on its products and services around 250 times ... Each score is a weighted average derived from an econometric model of consumers' responses to 17 questions rated on a scale of one to ten ...."

For all products and services, the survey found that the 1995 national satisfaction average was 73.7, down 1.1 percent from 1994. For the record, these were the results for local telecommunications service: