POLLING IN RUSSIA: UNDERSTANDING THE INFORMATION

C O N F I D E N T I A L MOSCOW 001608 SIPDIS E.O. 12958: DECL: 06/05/2018 TAGS: PGOV PHUM SOCI RS SUBJECT: POLLING IN RUSSIA: UNDERSTANDING THE INFORMATION Classified By: Political M/C Alice G. Wells. Reason: 1.4 (d). ¶1. (SBU) Summary: Russia has three home-grown stalwarts in the field of public opinion polling with another trying hard to break into the exclusive club. These three all share a history leading back to the godfather of public opinion surveys in Russia, Yuriy Levada. They, along with new-entry Bashkirova and Partners, produce important analyses on the political and social opinions of Russians. They have in common very similar methods for conducting the surveys, methods that reflect the peculiarities of Russia. This methodology includes complex sampling schemes and the use of quota sampling. Despite the difficulties of extracting conclusions from such methodologies, polling results should not be discarded, but rather taken with a grain of salt. End Summary. Public Opinion Polling in Russia -------------- ¶2. (SBU) Three national organizations dominate the field of public opinion polling in Russia with another at the margins attempting to be recognized as a player. The three most prominent organizations are the Levada Center, The All-Russia Center for Public Opinion (VTsIOM),and the Foundation for Public Opinion (FOM). These organizations routinely publish analyses of data each collects from regular omnibus national surveys whose topics run the gamut from politics to entertainment to religion. Bashkirova and Partners (B&P) also releases public use survey results from time to time, but does not maintain the constant output of the "big" three. ¶3. (SBU) Although set up by Yuriy Levada (founder of the Levada Center),VTsIOM has always been a wholly-owned government enterprise, which has led to claims of political influence in its analysis and results. FOM has similarly come under fire because of its apparent close relationships with the government agencies that fund its national surveys. FOM and VTsIOM have developed a broad government client base, while B&P and the Levada Center rely mainly on private commercial entities and other non-government clients. B&P has worked extensively with the International Republican Institute (IRI) and Levada Center has worked on a corruption project with Information Science for Democracy (INDEM),an NGO that focuses on rule of law and corruption issues. B&P also maintains a considerable level of independence from the GOR by receiving the lion's share of its revenues from private sources. Independent analysts give greater weight to Levada Center data because they consider its surveys less influenced by the Kremlin. ¶4. (SBU) All four polling organizations told us that they use orders for marketing surveys and orders from government agencies to subsidize the costs of their regular general use surveys. When compiling each survey, additional political monitoring or topical questions can be added as the costs of the survey are borne by other paying customers. These routine national surveys cover a broad range of topics and have a sample size of 1,500 to 3,000. VTsIOM, FOM and the Levada Center conduct a survey every week while B&P conducts at least two surveys a month. Each survey includes standard questions that the polling organizations use to track the popularity of political figures (i.e., Putin and Medvedev). The surveys also cover topics the particular organization thought interesting although at times the questions are driven by popular topics rather than sociological questions. For example the Levada Center and VTsIOM asked questions about the Champions League Soccer match recently held in Moscow. As with any company, they hoped to garner some free publicity if news outlets used their polling results in their stories. The Inverse of the Probability of Selection -------------- ¶5. (SBU) These four polling organizations described for us remarkably similar methods for conducting national opinion polls. While in the United States pollsters use telephone numbers to draw a random sample and conduct the survey, in Russia, a large number of households do not have telephones, requiring significantly different sampling and data collection methods. Pollsters here use a complicated mix of multi-stage random sampling and quota sampling to delineate a geographic block of housing units. Survey takers must cover the territory on foot to administer the surveys. The multi-stage sample design used by all four firms groups the entire Russian population into larger and larger groups and then randomly samples at each grouping level starting with the largest. In Russia, the polling agencies first sample the main subdivisions of the Russian Federation (the oblasts and republics) usually selecting 15 to 20 of the 85 regions. From each oblast, they then select a sample of villages, towns, and cities. The next stage selects smaller geographic areas that include housing units (apartments or houses). In each of these population centers they then select apartments or houses based on predetermined criteria (e.g., second building on the right, third floor, second apartment). ¶6. (SBU) At each level, however, the sample is complicated by certain requirements. For example, at the national level, oblasts are selected to assure sufficient coverage of urban and rural areas. At the oblast level, population centers are selected to assure, for example, sufficient coverage of small agricultural settlements and large urban areas. At the lowest sampling level, the individual living unit or apartment, the sampling moves to quota sampling. Once a household has been selected, the survey taker selects from that household individuals that meet certain criteria usually by age and gender. Should the sample not result in enough respondents of a certain category, e.g. males under 25, the survey taker goes from apartment to apartment until the quota for each age and sex category has been met. Understanding the Data -------------- ¶7. (SBU) The complex sampling methods make creating national estimates difficult. Because of varying population sizes in each grouping from oblast to apartment level, any person in the national population has a different probability of being selected for the survey. (Note: In the national telephone surveys in the US, each person has an almost equal chance to be selected.) Because of the different probabilities of selection, each respondent's answers must be given different weights. More specifically, the sampling designs result in a larger proportion of respondents from less populated regions than actually is present in the national population. These responses ought to be given less weight than respondents from large cities that make up a smaller proportion in the sample than in the national population. ¶8. (SBU) No polling organization told us of any weighting procedures used to devise national estimates or for estimating confidence intervals (i.e., the plus/minus) of each proportion calculated. Even among statisticians, calculating such intervals for complex sample designs has proven an obstinate problem. Bashkirova complained of another vexing theoretical statistical problem, namely quota sampling. She pointed out that the math that produces the confidence intervals for an estimate only works with random sampling. Quota sampling makes the assumptions of the math invalid. ¶9. (C) Georgiy Satarov of INDEM gave us his sociologist's views on some public opinion polling data, views that could help understanding what the data actually say. He examined FOM's report on everyday corruption published March 20 on the internet. FOM's data showed that about half of all Russians feel corruption is endemic and cannot be successfully combated. The survey also explored which government agencies respondents felt were most corrupt and how these perceptions changed over the past ten years. While he dismissed the data as mere propaganda, he described in detail some of the drawbacks of asking only a few questions in a large survey. When FOM asked about corruption, for example, the survey taker did not provide a definition of corruption. Because corruption covers a world of sins, end users of the data cannot be sure that their conception of corruption (bribery, kick-backs, etc.) corresponds to the respondents' conception. ¶10. (C) Satarov also pointed out that the time frame for FOM's questions was not fully explained to the respondents or in the data, meaning different respondents might consider events from two or three years ago while others would think back no further than six months. Finally, Satarov complained that for many personal, illegal or socially sanctioned events (such as paying a bribe),respondents are generally hesitant to answer a stranger's questions. Satarov lead an INDEM research project examining the incidence of corruption. He found that respondents needed specific cues and preparation questions before he would trust their answers. Because FOM did not include such cues in its data collection, Satarov dismissed the results as useless. Predicting Elections: Two Case Studies -------------- ¶11. (SBU) The Duma elections of December 2007 and the Presidential elections of March 2008 provide a worthwhile test of the four polling organizations. By comparing how these organizations predictions square with actual results, a clearer picture emerges of how well each firm does in estimating public opinion. Contacts in political parties indicated that they understand the utility of polling data for crafting a popular message or for maintaining realistic assessments of a campaign's success. Representatives from Just Russia and Civic Force told us during the Duma elections that they were handicapped because they could not afford polling. Smaller, non-Duma parties such as Yabloko or Union of Right Forces (SPS) did not have their own polling units and routinely predicted wildly optimistic election results (often predicting their party would get upwards of 15 percent of official returns). In the end, these parties actually garnered proportions of the vote very close to what the four polling agencies had predicted for them. ¶12. (SBU) The table below provides the estimates for the four organizations in the week just prior to the Duma elections. The bottom line provides the actual results. Each pre-election estimate had a plus or minus three percentage point margin of error. United Just Russia KPRF LDPR Russia Other -------------- -------------- VTsIOM 62 12 8 7 11 Levada Center 66 12 8 6 8 FOM 62 12 9 7 10 Bashkirova 57 13 10 9 11 Election Results 64 12 8 8 8 Given the margin of error the pre-election poll numbers for these four firms did not show any statistical differences. The actual result also fell with the margin of error in each case (with the possible exception of B&P which appears to have predicted a lower showing for United Russia than in fact it received.) ¶13. (SBU) The various pre-elections polls from the presidential campaign did not show much unanimity. The table below indicates the predicted returns for the presidential elections in the week preceding the March 2 elections. Again, the estimates came with a standard plus or minus three percentage point margin of error, and the bottom line indicates the final results as reported by the Central Election Commission. Medvedev Zyuganov Zhirinovskiy Bogdanov -------------- -------------- VTsIOM 78 12 9 1 Levada Center 80 11 9 1 FOM 72 13 13 1 Bashkirova 76 13 9 2 Election Results 70 18 9 1 Only FOM's estimate included the actual returns for Medvedev while all four agencies underestimated Zyuganov's support. ¶14. (C) Andrey Mukhin, director of the Center for Political Technology, many times dismissed the results of national polling, saying that the 1,500 respondents in the sample could not portray all the people of Russia. He even claimed that Kremlin officials had a direct hand in assuring that Levada Center, VTsIOM and FOM published the "right" data (i.e., showing large wins for United Russia). On the other hand, he claimed that the Kremlin had its own semi-secret polling that only select members of Putin's Presidential Administration could view. He mentioned to us that Putin decided to lead the United Russia list for the Duma elections after one of these polls indicated a significant decline in public support for United Russia. (Although Mukhin discounted the published results of opinion polls, the idea that even Putin tracked public opinion and made a momentous decision based on it indicates the level of political importance he now attached to polling, if not the actual published results.) Polling Data -- User Beware -------------- ¶15. (C) Russian public opinion polling firms and their staffs exhibited a thorough knowledge of current survey methods, and the staff we spoke with demonstrated high standards of professionalism. Presnyakova felt that, all other issues aside, the well educated analysts working at the four organizations maintained a high level of professional ethics. She said they would not "massage" data to achieve a particular result. Nothing we found contradicted this sentiment. However, perceptions matter and the common knowledge that VTsIOM and FOM rely heavily on Kremlin contracts for work colors many commentators' perceptions. The opposition news magazine The New Times recently highlighted some cases of leading questions in some VTsIOM surveys. While such questions are cardinal sins among survey methodologists, VTsIOM published the exact wording of its questions along with the results. As with any scientific venture, the truth lies in reproducibility, and the polling results before the Duma elections from all four polling agencies indicate strong correlations of results. On the other hand, the results from the pre-presidential election polling show wide convergence. ¶16. (C) Skepticism, then, ought to be the key word in using data from any of the polling firms in Russia. The sampling methods, the lack of statistical weights and the fact that only 1,500 to 3,000 Russians are surveyed mean that precision is at best illusory. Even the plus or minus three percentage point margin of error cannot be take at face value. The presidential election results showed that all polls missed the mark on Zyuganov's election returns. As Satarov further explained, for fairly complex concepts such as corruption, the data only go so far. For his purposes, the particular FOM data may not suit him; however, for giving broad indications of opinions and experiences, they data may indeed suffice. While there is a danger in reading too much into the data, such a danger exists in any survey (including INDEM's in which the sample size was 3,000 -- not much larger than the opinion polls). ¶17. (C) Satarov, The New Times, and countless other interlocutors have raised concerns about pro-Kremlin bias on the part of VTsIOM and FOM. Satarov even said that he only needs to know who paid for a particular data collection to know how the data are biased. On the other hand, the fact that the Levada Center was "more wrong" in favor of Medvedev than VTsIOM in the final presidential election polls, raises questions about the extent of such bias. While some opinion polls may serve a broader political interest (a serious problem in all developed countries as well),it appears at least that Russia has its own home-grown band of skeptics ready to critically analyze questionable conclusions. RUSSELL