Most rebuttal reports start with the phrase “This research is fatally flawed.” Whether that is true or not, experts tend to use that phrase quite liberally without paying too much attention to the supporting evidence.
When rebutting a survey that has been submitted in evidence, one should ask two questions: (1) has the researcher adhered to the fundamental scientific principles governing the design and execution of survey research; and (2) has he or she gone beyond acceptable and customary design choices? If the rebutter can demonstrate that the researcher has violated fundamental principles, then the “fatally flawed” conclusion is definitely appropriate.
When it comes to design choices, the rebutter must be careful to distinguish between unacceptable choices—which would automatically fall into the “fundamental principles” category—and choices that might differ from the choices the rebutter would have made, but which certainly do not qualify as “fatal flaws.” What can the researcher whose work was wrongly accused of having committed “fatal flaws” do?
In a case in which I was involved five years ago in the Eastern District of Michigan I was fortunate that the client agreed to fund a remake of my original research in which I substituted my original design choices with the rebutter’s recommendations. The rebutting expert in this case was one of the most prominent and well-published members of the profession who had to dig very deeply to find something wrong with my research and ultimately resorted to making indefensible assertions.
As one would expect, the results of the “remake” were identical to the original study proving to the court that design choices that are clearly not intended to sway the results one way or the other are not worthy of the “fatally flawed” label.
When litigating a case in which a rebuttal is proffered, attorneys should be aware of the “fundamental principles vs. design latitude” dichotomy and pursue it at deposition and also, if possible, by replicating the original research while incorporating the rebutter’s design recommendations. Yes, that adds to the overall expense, but it saves the agony of having to convince the finder of fact or juries that the points raised by the rebutter are trivial and inconsequential.