Circular logic

Tuesday, August 15, 2006

Example #1: Theresa Imanishi-Kari's Notebook

Teresa Imanishi-Kari was alleged to have falsified experiments, and data, in her laboratory notebooks. She was exonerated of all charges in 1996, but only after a Kafka-esque decade in which the charges against her continually metamorphosed, during which she could not actually examine the specific charges or evidence against her, and during which she was characterized as an example of corrupt science by government oversight committees. One of the principal charges against her was that she had cobbled together a laboratory notebook to bolster her claim of having performed experiments on immune response -- experiments which lead to an acclaimed paper in the journal Cell.
Imanishi-Kari's notebooks were apparently very messy and difficult to decipher. They contained paper strips of instrument readings that were taped to pages, and which were in varying colors and degrees of fadedness, printed with varying ribbons on a variety of printers. The pages themselves appeared out of order, dates were scratched through and corrected, there was white-out correction fluid used back-to-back on both sides of some sheets, and mechanical impressions suggested that pages were, in fact, dated out of the order they were written. The question naturally arose whether this messiness was the product of a clumsy, hurried attempt to deceive. Thus, the U. S. Secret Service was asked to perform forensic analysis on the notebooks. Part of the forensics was to compare Imanishi-Kari's notebooks against a sample of the notebooks of other scientists.
Secret Service investigators were unable to examine more than a tiny fraction (an estimate was perhaps 1%) of the notebooks produced by researchers in the same laboratory during the same time period. In the forum of the hearing, Imanishi-Kari's counsel cross examined one of the Secret Service investigators about why he had not included for examiniation the laboratory notebook of Moema Reis, at one time a research collaborator of Imanishi-Kari. The agent testified that they excluded this notebook because it looked a lot like that of Imanishi-Kari so they were "...unwilling to use that as an example of what a normal or a usual notebook would be."

Since an explicit goal of the comparison was to establish the normalcy of Imanishi-Kari's notebook, the decision to exclude other notebooks that looked like hers becomes an implicit definition of hers as being abnormal. The neat circular logic is...
Collect a sample of notebooks to establish a norm.
Exclude from the sample notebooks that appear similar to that of the subject.
Conclude that the subject notebook is unusual and abnormal.