To determine the top 10 advertising campaigns the following options were available. The first one was to go directly to the consumers and get their feedback. This method necessitated going to the media target group (TG) and was not practical, particularly when we had to test campaigns across 50 different product categories with different TG definitions.

The second option was to create or approach a sample of professionals in advertising, market research, and marketing and expose them to the details of the campaigns. This could be done either by using video films or story-boards, or printed descriptions of the same.

We chose to go with the advertising fraternity, because they were the most aware and knowledgeable about the campaigns of all the groups under consideration.

HOW WE DID IT

Viraag Agnihotri, GM (Delhi), NFO-MBL India Deepak Singh, Group Project Manager, NFO-MBL India Fifty-six television ad campaigns were shortlisted, based on perceptual ranking and scores on eight other parameters. Survey sample was made up of ad professionals in five cities with over six years experience in four categories: creative, account planning, media planning, and client servicing. Respondent were asked to rank to 10 best campaigns, on overall opinion and eight other parameters, on a 10-point scale. Only those campaigns rated among the top 10 by at least 20 per cent of the respondents were considered. The campaigns were then segregated using the mean ranking (of the top 10). The lower its mean ranking, the higher up it ended on our charts.

The research process involved two phases. The first phase included identification and selection of campaigns to be evaluated and to determine the parameters for evaluating a campaign. For this purpose, an exhaustive list of ad campaigns run in the last year was generated from industry sources. It was then vetted by advertising industry professionals and Business Today and NFO-MBL. Parameters for evaluation too were finalised at this stage, namely likeability, relevance, credibility, meeting brand and marketing objectives, uniqueness, creativity, media effectiveness, and fit with brand or company image

Given the staggering number of campaigns launched in a year, the scope of the study was limited to national campaigns (a national campaign being defined as one that was run across more than one zonal boundary). Finally, 56 campaigns were selected for evaluation. A semi-structured questionnaire was designed and run through few advertising professionals, before a final draft was arrived at.

In the second phase, campaigns were evaluated by 50 creative, account planning, media planning, and client servicing professionals hand-picked from leading ad agencies with over six years of experience. The randomised survey sample was made up of professionals in four categories: creative, account planning, media planning and client servicing. The survey, conducted in December 2001, included a structured interview with the target respondent by a researcher to elicit responses. The sample was distributed across zones and ad agencies to ensure a proper spread. Leading ad agencies of Delhi (including Gurgaon), Mumbai, Chennai, Bangalore, and Kolkata formed the base of the survey.

For the ranking process, the respondents were first checked for familiarity with the campaigns. The respondents were asked to evaluate all the campaigns they were familiar with, on overall opinion and research parameters, on a 10-point scale. Post this, they were asked to rank the 10 campaigns that they considered the best. The respondents were aided with the description of each campaign. The details included brandname, number of advertisements in the campaign, the length of each ad in the campaign, period during 2001 it ran, the language of the films, and a brief description of each ad in the campaign.

For any campaign to feature in the top 10, the first criteria was that it had to be rated among the top 10 by at least 20 per cent of the respondents, ensuring that only the campaigns liked by a sizeable lot made it into the second phase. This also ensured at least a 20 per cent familiarity base for all campaigns in the top 10. The highly-liked campaigns were segregated using mean ranking (of the first top 10 ranks), the lower the better. A tie in mean ranks was resolved in favour of the campaign that had better percentage ranking in the top 10.