From:
RRC
Date: 2001-06-21 13:21:00 UTC
Subject: Bob C: When is Evidence "Evidence"?
Q: "You've gone on and on about evidence...if anecdotal evidence is the
only thing wrong with something, why not just accept it? ...It can't hurt...."
"You may have heard the story of Procrustes, who stretched or shortened
his guests to fit his bed. What you may not know is that he measured
them up before they left the next morning and wrote a paper 'On the
Uniformity of Stature of Travellers' for the Philosophical Society of
Attica" A. S. Eddington.
The point of the above quote is that data often reflects the biases of
the observer and are often an epiphenomenon of personal belief systems,
rather than reflecting actual or significant differences. In this
example, Procrustes has PRECISE data to suggest his travelers are of a
uniform stature, BUT the data has been biased by the painful elimination
of the tall and short values, so is not very ACCURATE. This happens all
the time; perhaps the hardest thing that a scientist can do is to create
a research design which does not have some sort of observer bias. But as
often as someone can point out problems with scientific studies, they
are a minor compared to the problems of studies backed with nonspecific,
negative, uncorrelated and anecdotal evidence.
Nonspecific evidence is especially effective to the right group of
people. Have you seen some of the latest psychic programs? My favorite
one is that guy on the Sci Fi channel who talks to dead people. He faces
a small group of people and starts talking about someone, and invariably
one of the group recognizes the person. Perhaps this guy can actually
speak to dead people, but I suspect he is simply playing the ancient
parlor trick that was exposed by Harry Houdini nearly a century ago. I
can play it too; want to see? Based on the size of the list, I should
get at least 2 or more hits. Ready?
"I see an elderly gentleman. He is wearing glasses, but they may be
sunglasses. He has a horrible secret he is ashamed of that still bothers
him from the grave. He wanted to make things better with his family, but
died suddenly in a painful manner. He enjoyed his collection
tremendously and thought your gift was wonderful. I believe his first or
middle name begins with T but it could be a B. Something happened to his
leg before he died."
This is nonspecific evidence, and it is typically "pigeonholed" by
individuals to fit a particular person or event (pigeonholing is the
attempt by the victim to find a family member to fit the scenario; they
will scan their mental list of people until they find one who fits. In a
room of dedicated believers, the pigeonholing is so intense that the con
man can list detailed nonspecific attributes and someone will find a
match. In those cases where a match is not found, the victim is told to
go home and ask other family members, who then do their own
pigeonholing, usually with positive results because they are trying so
hard to find a fit). So a person thinking of their father Tom (who was
ashamed of smoking, who loved his stamp collection and your gift of a
first day cover, wanted to change his will to include his estranged
brother, but died of a heart attack after breaking his leg) gets the
same response as someone else thinking of their Uncle Bert (who was
ashamed of being gay and wanted to tell his family he was sorry about
always fighting with his deeply religious brother, who loved his
collection of ceramic hummingbirds, loved your gift of a silk tie, but
died in an auto accident which crushed his legs). You make YOUR OWN
specifics fit the nonspecific evidence offered by the con artist.
Pigeonholing is also the basis of a good number of the positive
attributes for many alternative treatments. You are given generalized
symptoms to treat, and are told to watch for generalized benefits.
Because the symptoms and benefits are nonspecific, anything can be
claimed as a positive result. For example, I sell you a mixture which I
claim will make your ferret feel better. If FOR ANY REASON, your ferret
feels better (this is also not tested, but is generally a "feeling"
expressed by a pet owner), then the reason can be (and usually IS)
attributed to the mixture. The problem is, what if your ferret started
to feel better simply because they got over a mild illness and would
have felt better regardless of treatment? Or if nothing was wrong with
them to begin with? If you look at the packaging slips provided by drug
companies, they are quite specific about what the drug will or will not
do, give an expected time-frame for benefits, offer a list of
side-effects, and will even tell you the placebo response. In other
words, they offer SPECIFIC evidence which can be tested, falsified or
confirmed, and the results are reproducible by outside agencies. Sure
they could be lying, BUT IT IS TESTABLE! The test is to ask SPECIFIC
questions and only accept specific answers (What was my uncle's name?
How old was he? Where did he die? What city was he born in? How does
colloidal silver work? How do you know? What are the side effects? What
are the specific benefits? What is a safe dose and range? What happens
if you give at the same time as an antibiotic? A steroid?) You guessed
it, these are only discovered by testing the substance.
Negative evidence is when you use the lack of an event occurring as
evidence of the effectiveness of a treatment. For example, suppose I
gave 100 ferrets a mixture I said prevented ECE if used correctly. If
the hundred ferrets who used the mixture never got ECE, then they become
negative evidence of the effectiveness of the medicine (it must work;
they took it and never got sick). But what if they were never exposed to
ECE, or had been exposed as kits and are now immune? Even if the mixture
was 100% effective, how would you know? The California Fish and Game
extensively uses negative evidence to support their position ferrets
will go feral if allowed as pets (just because you can't find them
doesn't mean they aren't there). This type of evidence is also used
effectively in snake oil remedies where the lack of disease is
considered "prevention." The test is to ask how many individuals had the
disease, how many took the treatment, and compare those numbers to those
"cured". In other words, scientific tests.
Uncorrelated evidence is sometimes very hard to recognize. It is
essentially assigning the attributes of one material to another based on
perceived or real relationships. You see this a lot with "families" of
substances, like between onions and garlic, elemental substances and
their chemical compounds, or drugs like aspirin and Advil. The trouble
is, many substances can have very close relationships but not have
similar responses. For example, even though there is a very close
relationship exists between aspirin and Advil, I tolerate aspirin but am
very allergic to Advil. I can eat peanuts but am allergic to soy. Some
silver compounds may kill bacteria at water treatment plants, but how
does that prove colloidal silver is effective inside the bowels of
carnivores? BTW, colloidal silver is NOT a silver compound, but rather
metallic (elemental) silver ground to microscopic dust and suspended in
demineralised water. This is another tactic used by the California Fish
and Game to prove ferrets can go feral (If they can go feral in New
Zealand, they can go feral in California). Just because something works
in people doesn't mean it is safe in ferrets, as anyone who had a ferret
eat a Tylenol can testify. There are plenty of drugs which you can use
on dogs or cats which have horrific or no results in ferrets. Did you
ever try those WOW potato chips? They were fried in an oil that is
chemically identical to vegetable oil (to fool your taste buds), EXCEPT
for a reversed portion of the molecule which renders it indigestible to
human intestinal enzymes. But the two oils are the same, right? They are
at least closer in structure than a silver compound is to elemental
silver. NEVER assume evidence is correlated without a study to back up
the claim. In the case of colloidal silver, THERE IS NO STUDY WHICH
CORRELATES ANY PERCEIVED EFFECTIVENESS OF ANY SILVER COMPOUND TO
MICROSCOPIC BITS OF ELEMENTAL SILVER IN A FERRET'S BOWEL! If a proponent
of colloidal silver compounds has such a study, published in a
mainstream, reviewed journal, now is the time to present it and make me
at crow. Evidence currently offered is uncorrelated, which means there
is NO proof it works as advertised. The test of correlation is proof of
the link, which requires testing.
Anecdotal evidence is simply someone saying it works for them, so it
will work for you. Humans are particularly prone to accepting anecdotal
evidence as factual because it is a human attribute to believe those you
trust. I believed my dad when he said Ford makes the best trucks, I
believed my photojournalist mentor when he said Canon made better
cameras, and I believed my best friend when he said Apple made better
computers; they were all correct. But when does a belief constitute
factual evidence? Anecdotal evidence usually "becomes" accepted evidence
when beliefs are SHARED. So among Canon camera owners, Canon makes the
best cameras, and among Ford pickup owners, Chevy is crap. This is
essentially a belief system no different than found in religious or
political systems, and is frequently associated with phrases such as
"faith," "my ferret is proof," "conspiracy", and "if it works, why
argue?" These sort of statements are a type of emotional coercion used
to kept members from questioning group authority; they have little
effectiveness against nonmembers. The California Fish and Game uses
anecdotal evidence to great extent (sightings of ferrets equal colonies
of ferrets, or the ever popular "ferrets are vicious animals because
they bite when frightened"). Anecdotal evidence is not blind tested; it
is not even reported in a manner which CAN be tested! Is Ford a better
pickup? Are Canon cameras superior to Nikon or Minolta? Does colloidal
silver actually work? The key to knowing the truth is in the manner in
which they were tested. And if it has never been tested?...then it is
not evidence. That's the bottom line, like it or not.
I am a skeptic, proud and true. I need information before I will trust
ANY substance. I need to know expected interactions the substance will
have with other medicines. I need to know the mechanism by which the
substance works (don't just claim "an enzyme"; which enzyme? How does
metallic silver prevent it from working? Why is colloidal silver
specific to bad germs only? How does it work on viruses, bacteria,
parasites and other "bad" inhabitants of infected bowels?). I need to
know the range of safe doses. I need to know the possible side-effects
and the long term problems. You get these answers when a formal study is
done, as well as a true idea of the effectiveness. You don't get it from
nonspecific, negative, uncorrelated and anecdotal evidence.
Let me put it another way. Forgetting for the moment all the ingredients
in a colloidal silver concoction are "natural" (cyanide is natural, as
is formaldehyde, rattlesnake venom, poison mushrooms, opium, botulism,
and anthrax). Suppose an effective colloidal silver product was marketed
by Dow Chemical. Who would use the stuff without published testing? What
if Dow Chemical said, "trust me..." and offered NO evidence? Would you
trust them then?
Put it another way. Assume all the bad reports about those greedy,
uncaring drug companies are true. Then why isn't colloidal silver
marketed to the mainstream public and prescribed by medical doctors?
Silver is very cheap, and silver nitrate even cheaper; imagine the
profits of making just an over-the-counter drug! We know silver nitrate
works; it was long used to kill gonorrhea germs in the eyes of newborns.
Why not use it to kill AIDS? Or that nasty flesh-eating variant of E.
coli? E. coli is found in the bowel; surely if elemental silver can kill
that bacteria there, it can work injected into a nasty wound on the leg.
Come on! Companies only interested in profit would have jumped on this
one a long time ago. Why not? I'm sorry, but I think silver bullets only
kill fantasy monsters, and I simply do not believe in werewolves.
I reject the use of ANY substance that hasn't been tested in a manner
which ties claims to results. Does colloidal silver work? Is it a magic
bullet which can cleanse the bowel or prevent death from systemic
cancer? You simply do not know and cannot know without controlled
testing. The throw-away phrase, "it can't hurt," is ONLY true if a
substance 1) can actually do no harm (what is its effectiveness then?),
2) it isn't a substitute for treatments which are known to work, and 3)
the use doesn't change the effectiveness of other treatments. The only
way I know of to figure out the 1st and 3rd requirements is by
standardized testing. Thus, regardless of the claims of effectiveness
for colloidal silver products (or ANY product), I will not use them
until I know they are safe and effective. My ferrets deserve no less.
The challenge isn't for the skeptics to answer (which is also a tactic
of the con artist); the burden of proof lies with the ones making the
claim. Nor is my challenge to prove a claim ANY suggestion that a
particular substance will not work. Maybe they do. So PROVE it!
Bob C