Are the numbers right? Reliability of Political Polls in Venezuela

By Juan Diego Aguilera • Social Sciences, Cycle 3, 2012
 

 

 

Introduction

During President Hugo Chávez Frías’s governmental mandate, Venezuela has found itself immersed in a constant political debate caused by the strong difference in interests existing between the officialism/chavism (faithful supporters of Chavez’s ‘socialist’ regime) and those who stand firmly against it (a.k.a. the opposition). When the time of electoral processes approaches, many want to know what tendencies are ruling the political climate, and possible outcomes of upcoming elections. Diverse poll companies –such as Datanalisis– conduct nationwide opinion surveys that yield varying projections of what the election’s results could be. Every time a new electoral poll is released, the aforementioned debate tends to heat up, generating controversy and doubt among the people, who find themselves confused not knowing which poll to rely upon. With my paper, I seek to provide the common citizen with a factual and analytical base showing the reliability of Datanalisis’s political surveys by providing the data necessary to support my findings.

Every time the latest poll results are announced or leaked, a wave of rumors floods the media. Like in any competition, the party that benefits most from newly released results tends to use them to their advantage in the form of electoral and political propaganda; often “bending” the data towards their benefit, in order to move the voting public in a desired direction. Naturally, those seen disadvantaged by the results tend to criticize them, repeatedly accusing poll companies of fraudulent methods or pay-offs by their opponent forces. Such manipulation of information leaves the regular citizen uncertain on what rumors or information they should believe. Many wonder which of the polls are reliable and, if determined to be “effective”, what makes the polls effective?

In order to answer this question, a theoretical framework regarding statistics and public opinion research will be established; thoroughly observing the different components that form the base for political surveys. Later, poll projections of selected electoral encounters will be compared with the real results proclaimed by the Venezuelan National Electoral Council (CNE). Products yielded by the comparisons shall determine the effectiveness of Datanalisis’s political polls (if projections and results fall within established margins, DT’s polls can be defined as accurate).

I will study a set of polls created by Datanalisis, a market research company with more than 25 years of experience in the Andean area, Latin America and the Caribbean. Throughout the past couple of decades Datanalisis has presented poll results close to electoral reality, which has given the company its current renown as one of the most important survey and market research companies in Venezuela. However, many still doubt Datanalisis’s reliability when it comes to “predicting” tentative National Electoral Council’s real results. It is important to clarify that Datanalisis works as an independent firm, meaning it is completely financed by private funds with no connection to the Venezuelan government or the National Electoral Council (CNE), which is the “maximum electoral authority, responsible for electoral transparency and assurer of the efficient organization of all electoral acts carried out in the country.” (CNE, 2012)

Historical Background and Origins of Political Polls

Political Polls emerged after the appearance of radio and market research studies generated governmental interest in public opinion (Girard, Alain and Jean Stoetzel, 1973). By acquiring a notion of the variations in public feedback regarding governmental duties, politicians could adjust strategies as to work more effectively and/or attract more voters. Since then, polling and surveying methods have evolved, allowing closer approximations to the opinions of the population within the studied samples (Abreu, 1999). Individuals such as Elmo Roper and George Gallup have developed effective mechanisms that correct common sampling mistakes that can lead to misleading results like those obtained in the first public opinion surveys by firms such as Literary Digest (Fundación Futuro Chile, 2010).

Since the polling company “Datos” appeared in Venezuela in the 1960s, political polls have been widely used there in order to anticipate possible electoral outcomes (Abreu, 1999). The expectation for firms conducting studies of the same population is that they should yield similar results when trying to determine the same parameters. However, predictions from different polling companies often present noticeable gaps between one another. This causes the people to question the methods employed in the realization of said surveys. In order to answer those questions, the theory behind the polls will be clarified in this paper, with the goal of determining if Datanalisis’s polls are effective.

The Value and Use of Public Opinion Surveys

Ideally, the government’s role consists of working for the people’s best interests while steering its country in the direction leaders consider appropriate through a set of policies and legislations. Public opinion polls serve the purpose of providing feedback to those who conduct it, and this information can then be sold to political parties, governments, and others who wish to know the tendencies and opinions regarding the population’s preferences within the political atmosphere (León, 2012). Through results yielded by political polls, governments can the redirect their course of action as to better serve the people (Abreu, 1999). Individuals value notions of the country’s general opinion, as they become informed of their political surroundings (Molina, 2011).

The Cons of Political Polls

One of the functions of political polls is to aid individuals in decision-making matters related to the political future of their country. However, this feature tends to be misused to guide and influence individuals towards a desired goal or vote (Abreu, 1999). There has been discussion on sociopolitical repercussions of public opinion polls such as “stimulus of the antidemocratic principles” or “encouragement of populism” among other negative aspects (Dader, 1990). Another problem that often emerges –at least in Venezuela– is the variation of data presented by different polling companies. Contradictory predictions of electoral possibilities yield uncertainty to the common citizen, who cannot use the presented information of public opinion to aid in decision-making (León, 2012). This collision of opposing data causes debate on political polls to heat up, creating conflicts between the different parties who do know on which poll to trust (Noticias 24, 2006).

Theoretical Frame

Polls and surveys are based in concepts of statistics, which according to Judith M. Tanur (1978) in her book “Statistics: A Guide to the Unknown” consist in “The science of data collection, analysis and interpretation, either to aid in decision making or explain some phenomenon or applied study of a random occurrence in regular or irregular conditions.” For the better comprehension of surveys, some key statistical principles need to be explained.

A political poll or survey is an instrument used to determine the distribution of a population’s political preferences during an established time period, with the purpose of inferring possible results of upcoming elections. For Datanalisis and the CNE, people legally capable of voting (18+, either gender, any social class, registered to vote) constitute the statistical population being studied.

It is impractical (perhaps impossible) to reach an entire nation’s population through a single survey – except for the actual national elections. Given this limitation, only a selected subset of the population is surveyed, known as the sample (Datanalisis, 2007). The process of selecting what parts of the population will be subject of the polls is known as sampling, and forms part of the survey’s phase of design. Ideally, the optimal sample for a political poll presents a homogeneous distribution of individuals proportional to the population’s characteristics, including but not limited to, gender, age, race, and socio-economical status; this method is known as “Sampling by quotas” (Datanalisis, 2007, Document “Clase de Muestreo”).

Sampling processes are one of the most important parts of a survey’s design phase; its correct realization will be directly reflected in the accuracy of final predictions when compared to the real results. By using appropriate sampling methods, sampling errors can be minimized, and its exact size obtained through application of statistical formulas (Everitt, 1999). The size of the sampling error determines a margin of “maximum admissible error (ee)”, which consists of the maximum acceptable difference between the values obtained in the surveys and the real results. At the time of comparison between the real results with the poll’s predictions, the survey’s projections can be considered statistically accurate if the results fall within the ee margin.

Once a suitable sample is determined, qualified political and market research experts define the questionnaire that will be used by field agents during the surveying process as well as its reach within the population being studied, concluding the phase of design. It is of utmost importance for the sample’s characteristics to be proportional to those of the population being considered, if the sample does not effectively represent the proportions of the population, results will differ from the optimal ones.

After the stage of design, the operative phase begins. At this point, the individuals who will be conducting the surveys go through a rigorous training process conducted by personnel with previous field experience. Surveyors are provided with induction sufficient to achieve the best possible results (standard procedures, “dos/don’ts”, etc.). After this, the previously determined questionnaire goes through a pilot test; when passed, operatives proceed to do the interviews in the field. The collected data is sent back to the central offices, for the phase of analysis. Specialized personnel then conduct quality controls and verify the results, their distribution, variations, and coherence.

Analysis of Results

I will examine Datanalisis’s results obtained under the following indexes: “Level of Governmental Approval”, “Political Self-Determination” and “Vote Intention”. The company’s effectiveness will be analyzed taking into account predetermined statistical formulas, CNE’s official results and the company’s results under the aforementioned indexes for the following events: Legitimation of Powers (2000), Recall Referendum (2004), Presidential Election (2006), Constitutional Reform (2007), and Constitutional Amendment (2009).

Level of Governmental Approval

“How do you evaluate President Hugo Chavez’s labor towards the country’s well-being?”

This parameter measures the population’s agreement with governmental performance during an established period, using a modified Likert Scale. According to Jonathan Brill (2008), this scale is “a special type of the more general class of summated rating scales constructed from multiple ordered-category rating items”. The scale’s modified version presents changing levels of agreement or disagreement, (Strongly Agree/Agree/Disagree/Strongly Disagree) with no midpoint (“Regular”). This prevents surveyees of “taking cover” in the benefit of the doubt, thus allowing a closer approximation to possible electoral results. Generally, the amount of individuals who agree with the government is proportional to the votes it (in this case, Chávez) attains. (Datanalisis, 2006)

On the following chart, DT’s projections of governmental approval are shown, followed by the margin of error determined by statistical formulas (ee), the inferior and superior limits of the standard deviation bell (% Hugo Chávez Approval -/+ ee, respectively), and ultimately, the real results proclaimed by the National Electoral Council. (All data extracted from Datanalisis’s and CNE’s Databases)

From the chart, we can see that in 80% of the studied events, Datanalisis’s projections statistically match the real results within the established margins of error (they fell within the established boundaries). The only result that falls out of the ‘bearable’ boundaries is 2006’s Presidential Elections, which was 0.88 percentile points under the inferior limit. Even though it can’t be considered statistically accurate, such small differences are usually considered negligible in the world of political polls, according to León (2012).

Intention of Vote

“If the presidential/state/municipal/parliamentary elections were to take place on the upcoming Sunday, Who would you vote for?”

This parameter measures the intended vote of the individuals being interviewed at the time of the survey. Theoretically, this question provides direct evidence of what the possible results of the election would be, delivering one of the firmest bases for determining a candidate’s popularity. In reality, many surveyees tend to abstain from answering this question, claiming they “don’t know/won’t answer” as a response, since in many cases they may feel intimidated by such a direct question. In order to use this parameter for predicting possible results, new percentages are recalculated taking into account just the individuals that had a firm intention of voting –for whichever candidate. “That recalculation is the one we consider as the real index” (León, 2012). As the time of the election approaches, the amount of surveyees that abstain from answering the question tends to decrease, allowing for a more direct inference of the final predictions.

On the following chart, DT’s projections of Intention of Vote are shown, followed by the margin of error determined by statistical formulas (ee), the inferior and superior limits of the standard deviation bell (% HCh Approval -/+ ee, respectively), and ultimately, the real results proclaimed by the National Electoral Council. (All data extracted from Datanalisis’s and CNE’s Databases)

Under the “Intention of Vote” index, 80% of the studied results fall within the determined standard deviation bells and can be considered as statistically accurate. Again, the only result that fell out of the prediction’s acceptable boundaries is only 1.1 percentage points away from the superior limit, a greater difference than with the previous Level of Governmental Approval parameter, but still considered close in the world of political polls (León, 2012).

Political Self-Determination

“How do you define yourself according to your political standings?”

When interviewers present this question, they provide the following set of answers: Chavist/Pro-Government, Anti-Chavist/Opposed to government, Neither Chavist or Anti-Chavist, Don’t Know/Won’t Answer. This parameter is one of the main tools for the determination of the population’s political preferences in a clear and concise manner.

On the following chart, Datanalisis’s projections of the distribution of political opinions in the country are established. Even though this is not a direct indicator of the final vote to be casted by the surveyees, it can be inferred that an individual self-declared as Chavist tends to vote for the president, and so on. Note: Datanalisis did not study the index Political Self-Determination for the Relegitimation of Powers in the year 2000; thus, this data is unavailable. (All data extracted from CNE’s and Datanalisis databases)

From the chart we can see that half of the results fall within the acceptable errors margins when compared with the election results. It is important to clarify that this parameter seeks to give a notion on the distribution of political opinions across the country’s population, not directly determine the possible outcomes of the electoral processes, and consequently does not serve said purpose with full efficiency- even though the results do not fall too far from the margins.

Other than the three mentioned indexes, the questionnaire established in the phase of design comprises a group of other questions that help narrow down the survey’s results in order to attain more accurate predictions. Most of the predictions’ variations are caused by individuals who abstain to answer at the moment of the survey. These individuals are the ones that really determine the final election results, since they can be convinced by declarations or actions of either candidate at the last minute, actions that can quickly shift the volatile electoral climate of any country, not just Venezuela.

Conclusions

Development and perfection of surveying mechanisms have allowed polling processes to increase their results’ accuracy. However, variations of these mechanisms can cause sampling errors and may lead to conflictive results between the polls of different companies. Political polls are intended to be informative tools for those who gather and acquire the information. They have great potential if appropriately utilized, but can also be subject of misuse or manipulation by parties trying to influence opinions.

After thoroughly observing the results of the surveys subject to this study, it can be concluded that Datanalisis’s political polls work as a feasible mechanism to portray Venezuela’s electoral reality. Yet, as all other polling firms existent in the country, Datanalisis is not 100% effective in predicting all electoral results within the acceptable error boundaries, but allows for a close approximation.

From the events studied, 66.6% of the results fell within the margins of DT’s predictions, and the rest were considerably close to the calculated boundaries. Even when they are to be pondered as statistically inaccurate, “such small errors (in the case of Level of Governmental Approval and Vote Intention) can be considered negligible in the world of political polls” (León, 2012), thus serving as viable mechanisms for the inference of possible electoral outcomes. It is important to restate that in the case of the Political Self-Determination parameter, the direct objective of the index is not to predict the outcome of the vote, thus it does not do so with full efficiency.

I believe the common citizen can find the theoretical framework and analysis presented as a well-founded base for relying on Datanalisis’s polls and avoid possible influence and manipulations from external parties.