What’s in a Norm? Food for Thought

Ad research pioneers George Gallup and Horace Schwerin had mixed opinions about the value of norms. They understood the theoretical importance of benchmarking, but they also understood the knowable and unknowable error that all benchmarks have. Indeed, Horace Schwerin once commented that he believed copy tests should be conducted without norms. It was his view that norms could be a crutch that helped marketers avoid thinking in absolute terms about what their advertisement should achieve.

Although an unconventional view today, it is still an interesting and important proposition. Even in the early days of copy testing, when the development of norms was a relatively straightforward undertaking, norms were not always the useful benchmarks they seem. A main idea communication norm, for example, is likely to be based on both advertisements that have a single key selling message, as well as ones that have more complex, multi-faceted communications. Moreover, the norm will usually be based on what the respondents of each test perceive the selling message to be, which may or may not have been the same as the communication objective. Or it may include ideas that are inherently easier or more difficult than yours to convey. Depending on your own communication objectives, it can be preferable to set an absolute criterion that the message must be played back by a certain proportion of respondents rather than compare to a norm based on commercials with message content and strategies.

But questions about norms are even more relevant today. Today’s advertising world is more complex than in Gallup and Schwerin’s time, when advertisers had a mass market orientation. Copy testing companies simply provided extensive, broadly-based, category-specific norms based on methodologies that could accumulate extensive databases based upon samples of men and/or women between the ages of 18 and 49 or 65. Since then, media has evolved from providing mass appeal programming and publications toward a diverse industry replete with a vast landscape of special interest offerings. Meanwhile marketers, armed with segmentation research that facilitate honing communications in on target markets with the greatest leveragability, take advantage of the expanding media opportunities with initiatives targeted to more specifically designated audiences. Thus the advertiser’s need for norms has changed with implications for the quality of norm that is often available.

• Today, advertisers often specify that samples meet specific target definitions designed to match their demographic or psychographic targets. Competitors within the same category may specify different sampling targets or more have different screening criteria for identifying like samples. A norm for travel and leisure services, as an example, may consequently be comprised of a variety of sample definitions, some based on advertising targeted to the premium traveler, others to the budget minded; some to family vacationers, others to business travelers; some to the youthful and adventurous, others to older vacationers with a more relaxed mindset. As a result, when a category norm may exist, it may not reflect the specific target in which the marketer is interested.
• The increased use of pretesting has given advertisers greater flexibility and agencies greater creative freedom. A downside to this positive development, though, is that tested concepts are often included into a normative database even though not all will have been taken into final production nor received media support. When a norm is based on finished commercials, as were typical erstwhile syndicated services, it represents, by definition, all executions that were judged “airworthy.” If pretested concepts are included, today’s normative databases may present a lower than desirable hurdle if they include concepts that failed to pass muster.
• Even highly effective advertising will not have been developed to be effective on all performance measures. Some commercials are developed to be strong in some aspects of communication and other commercials are developed to “move the needle” in other areas. For example, sometimes ad liking will be an important criterion and sometimes it is not. The same can be said, at times, about recall, persuasion and ad advocacy. Since most ads are tested within a standardized system for the company, they are tested against measures that are not always relevant to the objectives of the advertising. Since norms are typically compiled based on complete test results, they may be based on cases where the measure itself was not relevant to the test commercial’s communication objectives.
• Advertisers also have a wide variety of techniques and vendors from which to choose. Consequently, any given vendor may have experience in a category but only with a limited repertoire of clientele. Unless a program is in place to add non-client readings to their database, a prospective new client may be comparing their results to a “norm” based upon a non-representative sampling of category advertising.

The consequence of all this is that any particular norm today is based on a data set that is likely shallower and less comprehensive than they have been, making their use less straightforward than in earlier already problematic times. Schwerin’s proposition may even begin to have greater appeal under these circumstances, but solutions exist for ensuring that useful benchmarks are also considered. Knowledgeable firms will usually have addressed these issues and be in a position to provide workable answers through means like the following:

1. Using their experience and breadth of testing to craft a norm specific to your target audience;
2. Establishing that performance within subgroups does not to show much variation; and/or
3. Devising a test design that will provide its own benchmarks.

As a result you can be more confident that you are comparing the specific target audience reactions of your advertising to that of your competition, giving you a strong perspective for assessing the likely effectiveness of your messages.

It was mentioned above that it has been customary since copy testing’s early days for most companies to evaluate their advertising against category-specific norms. However, there is a school of thought that all advertising norms provide a better benchmark because each ad must compete against the totality of all advertising in order to break through. While there is merit to maintaining perspective on how all advertising tends to perform in a given testing system, losing sight of how well your advertising is doing relative to your competition among your target audience is risky business. A company in a high interest category that accepts an ad that is above the all advertising average without determining that it may nevertheless be below industry norms will be at a disadvantage.

As with other aspects of research, experienced professionals at your vendor will be able to provide guidance on how to best establish action standards for your test and will be best able to use these capabilities to get the most out of your results and provide strong direction for your communications.