National Catholic Reporter

The Independent News Source

When Samples Don't Tell the Story

 |  NCR Today

A few days ago I expressed the wish that a Sister Survey similar to those conducted by the distinguish sister-sociologist, Marie Augusta Neal, in the years following Vatican II, could be done now to find attitudes of sisters independently in the midst of the investigation crisis.

Meanwhile, word comes that the estimable periodical, U.S. Catholic, is surveying sisters about the investigation and separately inviting readers to post their views of the probe. The general reader survey is described as a "poll." The first wave of responses showed 55 percent sharply critical of the Vatican's initiative, but after a priest filed a entry on his blog highly in favor of the process, together with a link to the magazine's site, the results tilted heavily in the other direction, 80 percent approving the probe.

Be careful what you believe. The methods and results can be terribly misleading. I worked for a brief time at the Gallup Poll. An occupuational hazzard there and in similar operations is that the public often placed their methods of scientific random sampling on a par with undisciplined surveys. To qualify as a valid poll means to use the scientific procedure that usually leaves a very small margin of error.

By contrast, efforts to gain opinion by making no attempt to sample a cross section of the public but instead just issues a general invitation has little to no credibility as a measure of attitudes. When the local television news asks viewers to call in to say whether or not they favor installing new swings at the playground, the result will indicate something about who was watching and who was motivated to vote, but it won't capture what the public at large thinks.

U.S. Catholic will come up with something interesting in its survey but without a colossal coincidence the sampling won't represent sisters at large. And obviously the "poll" of readers is subject to factors like the priest's advocacy that swing interest groups one way or another.

07-04-2014.jpgThere's more to NCR than what you read online. Subscribe today!

Scientific surveys have their flaws, of course. They're limited by the quality of the questions and the ubiquitous lack of context. But they do accurately reflect what they look for.

Collecting opinions and thoughts of people on a non-scientific basis has its own value. Much can be derived by giving subjects a chance to nuance their ideas and feelings. It's just that these projects shouldn't be confused with the actual polling process.

In other words, unscientific samplings have real benefits, but they cannot carry the weight of the rigorous Sister Surveys that Sister Neal guided. That would provide a sweeping portrait of what the array of sisters are thinking in the midst of an inquisition they did not choose.

NCR Comment code: (Comments can be found below)

Before you can post a comment, you must verify your email address at Disqus.com/verify.
Comments from unverified email addresses will be deleted.

  • Be respectful. Do not attack the writer. Take on the idea, not the messenger.
  • Don't use obscene, profane or vulgar language.
  • Stay on point. Comments that stray from the original idea will be deleted. NCR reserves the right to close comment threads when discussions are no longer productive.

We are not able to monitor every comment that comes through. If you see something objectionable, please click the "Report abuse" button. Once a comment has been flagged, an NCR staff member will investigate.

For more detailed guidelines, visit our User Guidelines page.

For help on how to post a comment, visit our reference page.

 

Feature-flag_GSR_start-reading.jpg

NCR Email Alerts

 

In This Issue

July 4-17, 2014

07-04-2014.jpg

Not all of our content is online. Subscribe to receive all the news and features you won't find anywhere else.