CARL SAGAN'S BALONEY DETECTION KIT

Based on the book The Demon Haunted World by Carl Sagan

The following are suggested as tools for testing arguments and detecting fallacious or fraudulent arguments:

 
• Wherever possible there must be independent confirmation of the facts
• Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.
• Arguments from authority carry little weight (in science there are no "authorities").
• Spin more than one hypothesis - don't simply run with the first idea that caught your fancy.
• Try not to get overly attached to a hypothesis just because it's yours.
• Quantify, wherever possible.
• If there is a chain of argument every link in the chain must work.
• "Occam's razor" - if there are two hypothesis that explain the data equally well choose the simpler.
• Ask whether the hypothesis can, at least in principle, be falsified (shown to be false by some unambiguous test). In other words, it is testable? Can others duplicate the experiment and get the same result?

• Conduct control experiments - especially "double blind" experiments where the person taking measurements is not aware of the test and control subjects.
• Check for confounding factors - separate the variables.

• Ad hominem - attacking the arguer and not the argument.
• Argument from "authority".
• Argument from adverse consequences (putting pressure on the decision maker by pointing out dire consequences of an "unfavourable" decision).
• Appeal to ignorance (absence of evidence is not evidence of absence).
• Special pleading (typically referring to god's will).
• Begging the question (assuming an answer in the way the question is phrased).
• Observational selection (counting the hits and forgetting the misses).
• Statistics of small numbers (such as drawing conclusions from inadequate sample sizes).
• Misunderstanding the nature of statistics (President Eisenhower expressing astonishment and alarm on discovering that fully half of all Americans have below average intelligence!)
• Inconsistency (e.g. military expenditures based on worst case scenarios but scientific projections on environmental dangers thriftily ignored because they are not "proved").
• Non sequitur - "it does not follow" - the logic falls down.
• Post hoc, ergo propter hoc - "it happened after so it was caused by" - confusion of cause and effect.
• Meaningless question ("what happens when an irresistible force meets an immovable object?).
• Excluded middle - considering only the two extremes in a range of possibilities (making the "other side" look worse than it really is).
• Short-term v. long-term - a subset of excluded middle ("why pursue fundamental science when we have so huge a budget deficit?").
• Slippery slope - a subset of excluded middle - unwarranted extrapolation of the effects (give an inch and they will take a mile).
• Confusion of correlation and causation.
• Straw man - caricaturing (or stereotyping) a position to make it easier to attack..
• Suppressed evidence or half-truths.
• Weasel words - for example, use of euphemisms for war such as "police action" to get around limitations on Presidential powers. "An important art of politicians is to find new names for institutions which under old names have become odious to the public"

Above all - read the book!

• The Critical Thinking Community
• CSICOP/Skeptical Inquirer
• Australian Skeptics
• Quackwatch
• Carl Sagan Productions Ann Druyan's comment on this web page:
  "I have no problems whatsoever with your efforts to spread the word on critical thinking.  It was Carl's dream and mine that each and everyone of us would have that baloney detection kit inside our heads.  I salute your efforts in this direction.
  With best wishes,
  Ann Druyan"
• 12 Nov 2000 Project Voyager: OneCosmos represents the realization of a shared lifelong hope to organize and lead a team that will deliver the vision of Cosmos through every available screen: spectacular living Internet, engaging television and cinematic works of art.
• Examining the role of think tanks by Sharon Beder, Engineers Australia, November 1999.
• Innumeracy.com.
• Faith-Based Reasoning - Scientific American June 2001. In one case [global warming], the president invokes uncertainty; in the other [missile defence], he ignores it. In both, he has come down against the scientific consensus.
• InConcept.
• Scientific American, Nov 01: Baloney Detection: How to draw boundaries between science and pseudoscience, Part I. Part II.
• Scientific American May 02: The Exquisite Balance - It seems to me what is called for is an exquisite balance between two conflicting needs: the most skeptical scrutiny of all hypotheses that are served up to us and at the same time a great openness to new ideas.... If you are only skeptical, then no new ideas make it through to you.... On the other hand, if you are open to the point of gullibility and have not an ounce of skeptical sense in you, then you cannot distinguish the useful ideas from the worthless ones - Carl Sagan, 1987.
• Skeptic.com
   

  
   
   
   
   
   
   
   
   

  Less serious sites:

• Journal of Irreproducible Results
• The Annals of Improbable Research. (with the Ignobel Awards)