Scientific Skepticism vs. Courtroom Skepticism

[Antipositor]

The spate of wrongful convictions in recent years overturned by DNA testing exposes the difference between the scientific method and courtroom methods of discovery.

Science is based on utilizing willful uncertainty to evaluate and test assertions, hypotheses and theories. The premise is that truth can not be falsified, and fallacies can never be proved.

Courtrooms seek to create certainties where evidence and logic might fail to deliver a reasonable level of confidence in a scientific realm. For this reason the apparent error rate is much higher in courtroom verdicts than in scientific journals.

Skeptics seem divided along these lines as well. Some of the most famous skeptics such as James Randi, Michael Shermer and Paul Kurtz have made a name for themselves by being able to give a quick verdict to unexplained questions. Uncertainty it seems is not their forte, and they are often sought for their "final word" on stories which seem to defy easy explanation.

Other skeptics follow a more scientific discipline, employing uncertainty and refraining from employing front-loaded conclusions and negative certitudes. For them, challenging a belief is more than creating a negative belief to counter it; it's a proper investigation of evidence. If a mundane explanation becomes apparent, so be it, but it's not necessarily always the outcome. Absence of evidence is not proof of absence of evidence, therefore some phenomena remain unexplained. An uncomfortable position for some, but nonetheless a correct state of understanding for such cases.

[Lord Muck oGentry]

Absence of evidence is not proof of absence of evidence [...]

I'm pretty sure that it is. :smiley: Was that a typo?

On the assumption that it was: are you saying that there is no burden of proof on non-mundane explanations?

[Antipositor]

I'm pretty sure that it is. :smiley: Was that a typo?

On the assumption that it was: are you saying that there is no burden of proof on non-mundane explanations?
Yes, that should read:"absence of evidence is not proof of evidence of absence."

[Lord Muck oGentry]

Yes, that should read:"absence of evidence is not proof of evidence of absence."

Fair enough.

Now what about burden of proof?

[Trinoc]

Now what about burden of proof?
Burden of proof is a courtroom concept. Science doesn't (and scientists shouldn't) care which "side" of a debate comes up with the clinching (for now) evidence.

[Pebble]

Burden of proof is a courtroom concept. Science doesn't (and scientists shouldn't) care which "side" of a debate comes up with the clinching (for now) evidence.

Not of course the same as saying that science should treat all possible outcomes as equally likely in all circumstances. A purely statistical exploration of the data is fine when there is no hypothesis to be tested. Where hypotheses exist, some are congruent with what we know to be more probably correct, others require if correct a complete re-evaluation of current knowledge. The evidence required to adopt the first type of hypothesis is less than for the second.

[Trinoc]

Not of course the same as saying that science should treat all possible outcomes as equally likely in all circumstances.
Of course not ... if you told me something that required major changes to existing theories to accommodate it then I would insist on the evidence being very good, but it would be immaterial whether you provided that evidence, or me, or someone else. The idea that someone has personally to prove a "case" should have no place in science.

[Pebble]

The point being that 'famous' skeptics generally are simply dismissing nonsense rather than being incapable of dealing with uncertainty as suggested by Antipositor.

[Trinoc]

The point being that 'famous' skeptics generally are simply dismissing nonsense rather than being incapable of dealing with uncertainty as suggested by Antipositor.
True, but I'm sure we have all been tempted simply to shout "Bullshit!" when we hear something that doesn't fit in with existing theories, rather than thinking the idea through. 90% of everything is rubbish (Sturgeon's Law), but we can't know which 90% if we dismiss everything which seems fishy out of hand.

[Pebble]

Pebble's Law: 90% of "new insights" are recycled bullshit. (perhaps in deference to my avatar - recycled slurry)

[Trinoc]

Pebble's Law: 90% of "new insights" are recycled bullshit. (perhaps in deference to my avatar - recycled slurry)
It's true. Original bullshit is so hard to come by these days.

[polomint38]

It's true. Original bullshit is so hard to come by these days.


Oh! there are sites out there, too polite to mention :cheesy:

[Jack of Kent]

The spate of wrongful convictions in recent years overturned by DNA testing exposes the difference between the scientific method and courtroom methods of discovery.

I find this statement puzzling.

There is a difference between how evidence is used in science and law, of course. But I am not sure there has been any "exposure".

Criminal law usually requires "beyond reasonable doubt"; civil law usually requires "balance of probability".

The evidence of DNA (or, eg, Meadows' misconceived use of statistics) can be put as part of the prosecution (criminal) or claimant (civil) case. It is then for the tribunal of fact (judge or jury) to decide whether the evidence meets the required evidential standard.

But it appears you are contrasting pre-DNA cases with post-DNA cases. Before there was access to DNA, the court still did the job of assessing the available evidence.

I am not sure how DNA now being available (and so showing certain convictions to have been untenable) demonstrates how the legal approach to evidence to be incorrect; after all, the appeal decisions based on new DNA evidence are also legal decisions.


Science is based on utilizing willful uncertainty to evaluate and test assertions, hypotheses and theories. The premise is that truth can not be falsified, and fallacies can never be proved.

If you say so; IANAS.


Courtrooms seek to create certainties where evidence and logic might fail to deliver a reasonable level of confidence in a scientific realm. For this reason the apparent error rate is much higher in courtroom verdicts than in scientific journals.

This is a strawman argument.

A criminal court is not seeking certainty; it is putting the prosecution to test on whether their evidence can meet the evidential standard of "beyond reasonable doubt".

The criminal court affords the defence a full opportunity to challenge the prosecution in this exercise.

And only when the prosecution has set out the case, and having heard the defence, will the court decide whether the evidential standard has been reached.

At no point is any participant in the criminal litigation process seeking certainty, still less "create" certainty.

In view of this, your second sentence here makes little sense to me.


Skeptics seem divided along these lines as well. Some of the most famous skeptics such as James Randi, Michael Shermer and Paul Kurtz have made a name for themselves by being able to give a quick verdict to unexplained questions. Uncertainty it seems is not their forte, and they are often sought for their "final word" on stories which seem to defy easy explanation.

I fear this is a strawman argument too.


Other skeptics follow a more scientific discipline, employing uncertainty and refraining from employing front-loaded conclusions and negative certitudes. For them, challenging a belief is more than creating a negative belief to counter it; it's a proper investigation of evidence. If a mundane explanation becomes apparent, so be it, but it's not necessarily always the outcome. Absence of evidence is not proof of absence of evidence, therefore some phenomena remain unexplained. An uncomfortable position for some, but nonetheless a correct state of understanding for such cases.

I suspect these are the skeptics which you prefer. But as you prefer this position instead of two undoubted strawmen, I am not sure you can take that much comfort from your analysis.

For me, you do raise - indirectly - an interesting point about purpose: a litigation process is intended to be discrete. A dispute or matter needs to be resolved, and the rules of court and the burden of proof means that there will be an answer (unless you can get your lawyers to draw it out!).

If the prosecution or claimant does not prove their case to the relevant standard then they lose. The case ends. The scientific enterprise is (or should be) less time restrained.

But where I think you have misunderstood the legal process is that this is because the legal process seeks certainty; finality (with the prospect of appeal or re-trial) I will concede, but not certainty.

[Antipositor]

I find this statement puzzling.

There is a difference between how evidence is used in science and law, of course. But I am not sure there has been any "exposure".

Criminal law usually requires "beyond reasonable doubt"; civil law usually requires "balance of probability".

The evidence of DNA (or, eg, Meadows' misconceived use of statistics) can be put as part of the prosecution (criminal) or claimant (civil) case. It is then for the tribunal of fact (judge or jury) to decide whether the evidence meets the required evidential standard.

But it appears you are contrasting pre-DNA cases with post-DNA cases. Before there was access to DNA, the court still did the job of assessing the available evidence.

I am not sure how DNA now being available (and so showing certain convictions to have been untenable) demonstrates how the legal approach to evidence to be incorrect; after all, the appeal decisions based on new DNA evidence are also legal decisions.


The "legal approach" managed to ruin the lives of many innocent people, not to mention allowing the actual perpetrators to scoff at the law and remain free to commit further crimes. This is not good for society, nor is it good for any us to lose confidence in a system we trust to deliver justice.

I agree it's sometimes difficult to achieve a high degree of certainty in such things as criminal cases, but locking up or executing innocent people doesn't help. A recently-released wrongfully convicted man who spent many years on death row, awaiting execution in the US seems to have hit the nail on the head; when he said, "the more serious the crime, the more likely the accused will be convicted." I believe this underlines a basic flaw in the concept of courtroom justice. You also seem to favor this outcome-based approach in your defense of the legal system(s).

I am sure that a more scientific approach to criminal cases would be a large improvement over the adversarial game of "proving" one's case "beyond a reasonable doubt." For jury trials, the fallacy of "truth being conferred by consensus" seems also to be ignored. No reputable scientist should lend his/her name to such a flawed approach to discovery.

If nothing else, remove the premise that you can "prove" a theory. This concept went out 400 years ago when it was disproved that Jan Von Helmont "proved" that mice were "spontaneously generated" in tubs of grain and dirty shirts.

[Jack of Kent]

The "legal approach" managed to ruin the lives of many innocent people, not to mention allowing the actual perpetrators to scoff at the law and remain free to commit further crimes. This is not good for society, nor is it good for any us to lose confidence in a system we trust to deliver justice.

What is your alternative? No criminal justice system at all?

Your "legal approach" is actually a process of assessing evidence in court, tested by a defence, and subject to an independent tribunal of fact, with the opportunity of appeal.

What exactly is wrong with that? What should have courts done before DNA testing? Should they have just waited until something was developed of which they could not be aware?

And as for people "getting off" - that just means that the prosecution has not proved their case beyond reasonable doubt. Perhaps they should be imprisoned anyway because (like the West Midlands Serious Crime Squad in the 1970s) we - er - know they're guilty?


I agree it's sometimes difficult to achieve a high degree of certainty in such things as criminal cases, but locking up or executing innocent people doesn't help. A recently-released wrongfully convicted man who spent many years on death row, awaiting execution in the US seems to have hit the nail on the head; when he said, "the more serious the crime, the more likely the accused will be convicted." I believe this underlines a basic flaw in the concept of courtroom justice. You also seem to favor this outcome-based approach in your defense of the legal system(s).

Very interesting point.

When there is a serious crime it is certainly more likely that there will be pressure for a prosecution.

Those on Death Row certainly need properly funded and competent defence and appeal lawyers.

But I am not sure that a different approach to evidence itself would work.


I am sure that a more scientific approach to criminal cases would be a large improvement over the adversarial game of "proving" one's case "beyond a reasonable doubt." For jury trials, the fallacy of "truth being conferred by consensus" seems also to be ignored. No reputable scientist should lend his/her name to such a flawed approach to discovery.

I could be perhaps unfair and point out that those, like you, who favoured the "inquisitorial" single-expert model included those in the Spanish Inquisition and a number of fine Witchfinder Generals.

The adversarial and jury-based approach is seen by many as being part of a liberal state.

If I was accused of a serious crime, I certainly would prefer to have a jury trial rather than a single expert :-)


If nothing else, remove the premise that you can "prove" a theory. This concept went out 400 years ago when it was disproved that Jan Von Helmont "proved" that mice were "spontaneously generated" in tubs of grain and dirty shirts.

I have no idea about this and fail to see its relevance...

[Antipositor]

What is your alternative? No criminal justice system at all?

Your "legal approach" is actually a process of assessing evidence in court, tested by a defence, and subject to an independent tribunal of fact, with the opportunity of appeal.

What exactly is wrong with that? What should have courts done before DNA testing? Should they have just waited until something was developed of which they could not be aware?

And as for people "getting off" - that just means that the prosecution has not proved their case beyond reasonable doubt. Perhaps they should be imprisoned anyway because (like the West Midlands Serious Crime Squad in the 1970s) we - er - know they're guilty?



Very interesting point.

When there is a serious crime it is certainly more likely that there will be pressure for a prosecution.

Those on Death Row certainly need properly funded and competent defence and appeal lawyers.

But I am not sure that a different approach to evidence itself would work.



I could be perhaps unfair and point out that those, like you, who favoured the "inquisitorial" single-expert model included those in the Spanish Inquisition and a number of fine Witchfinder Generals.

The adversarial and jury-based approach is seen by many as being part of a liberal state.

If I was accused of a serious crime, I certainly would prefer to have a jury trial rather than a single expert :-)



I have no idea about this and fail to see its relevance...
Thank you for sharing that.

Legal reform is not the same as instating anarchy; of course I'm not in favor of "nothing at all."

I happen to have been a bit disturbed by the growing ranks of the wrongly convicted, and it threatens my security a bit to learn that the system is so badly flawed.

I don't have all the cures, but looking at the scientific method, it seems to have a fairly good track record, therefore any move away from the adversarial, "prove your case" courtroom methods and towards a more scientific "test the evidence" method gets my vote. A hallmark of pseudoscience is often a mission to "prove a case" with regard to a given theory which is often posited in front, and rest of the journey is a search for evidence which seems to support that conclusion. This is not science.

You bring up a lot of nasty history of wrongful prosecution, such as the Spanish Inquisition, but that doesn't validate the modern system, which is centuries old, and is still largely designed to create a sort of "oubliette" for those it deems guilty. Appeals are difficult for those who didn't have the benefit of new evidence or the advent of a new technology such as DNA testing, and many have rotted away in prison or were executed without chance of an appeal. Some were guilty of course, but how many? Was it a pretty-good accuracy rate? How can you know? Science is better at estimating probabilities than judges or juries.

[DrS]

looking at the scientific method, it seems to have a fairly good track record, therefore any move away from the adversarial, "prove your case" courtroom methods and towards a more scientific "test the evidence" method gets my vote.The opposition is surely not scientific method versus adversarial legal system, but common law versus Roman or Napoleonic law. It seems to me that neither system is fundamentally concerned with science per se but is concerned with evidence of whatever sort it might be.

[Antipositor]

The opposition is surely not scientific method versus adversarial legal system, but common law versus Roman or Napoleonic law. It seems to me that neither system is fundamentally concerned with science per se but is concerned with evidence of whatever sort it might be.
Courtrooms use science where it serves their ends, but verdicts are often not reached scientifically. It's more of a political process than a scientific one, (you vote for the truth).

[DrS]

Courtrooms use science where it serves their ends, but verdicts are often not reached scientifically. It's more of a political process than a scientific one, (you vote for the truth).
You are talking about common-law systems as opposed to the Roman/Napoleonic law form of an investigative magistrate or magistrates. I personally think that the latter form is preferable, but the vast majority of those used to the common-law system would, I suspect, opt for trial by jury. This requires evidence to be interpreted from two different perspectives. It is adversarial by nature, and recent history of law research tends to see this as involving the construction of opposing narratives.

Evidence is presented or disputed as part of the forming of the case for either sides, and each side's narrative is considered by the jury. How do you conceive of reaching a verdict scientifically in such a system?

[Jack of Kent]

Can Antipositor kindly state exactly what he/she thinks should be changed?

[MischiefMonkey]

My only critism of jury trials would be that those interpreting the evidence can not directly question it, which is why I wonder if the inquisitorial method might have an advantage.

However neither system can follow a purely scientific model. As I understand, in science a hypothesis is open to a continuous process of testing, resulting in evidence that supports it or in the hypothesis being disproven.

To me, applying this to law would reverse the burden of proof. The hypothesis would be that Person A is guilty, thus the defendant would have to disprove the hypothesis by conclusively proving their innocence.

[Croydon Bob]

My only critism of jury trials would be that those interpreting the evidence can not directly question it, which is why I wonder if the inquisitorial method might have an advantage.

When I was on a jury we were allowed (encouraged) to ask questions, via a note to the judge, if there was anything we were unclear about during the trial. Once in the little room making our decision we were allowed to ask questions or ask to review evidence via a note (mine didn't, a friend was on a jury last month that spent 17 hours reviewing the forensic evidence before going for 'guilty'). So we weren't allowed to directly question evidence, but we were able to do so indirectly. And if, at the end of trial, the evidence is too complicated or confusing then the jury should find the accused 'not guilty'.

[Antipositor]

Can Antipositor kindly state exactly what he/she thinks should be changed?
This could get repetitive, but I don't have the answers; I merely want the high number of wrongful convictions to decrease, and I'm sure that any step towards the scientific method and away from a political-style process would be an improvement.

[Matt]

This could get repetitive, but I don't have the answers; I merely want the high number of wrongful convictions to decrease, and I'm sure that any step towards the scientific method and away from a political-style process would be an improvement.

So lets set up a control society a differnt rule of law so we have something to compare to in our experiments. :cheesy:

Oh hang on that's what we use Scotland for >:D

[DrS]

any step towards the scientific method and away from a political-style process would be an improvement.

But how do you see that happening within the context of either civil law (the majority of European systems) or common law (UK)?

I think, too, that your definition of the process as political is geared for confusion. Neither system is political in the generally understood meaning of the word. The fact that there are votes in the jury system (which is not normally found in the majority of European countries anyway) does not make it some sort of democratic process.

[MischiefMonkey]

When I was on a jury we were allowed (encouraged) to ask questions, via a note to the judge, if there was anything we were unclear about during the trial. Once in the little room making our decision we were allowed to ask questions or ask to review evidence via a note (mine didn't, a friend was on a jury last month that spent 17 hours reviewing the forensic evidence before going for 'guilty'). So we weren't allowed to directly question evidence, but we were able to do so indirectly. And if, at the end of trial, the evidence is too complicated or confusing then the jury should find the accused 'not guilty'.

Same at the trial I sat on the Jury for. However, there were questions we would have liked the prosecution/defence to have asked the witnesses. Those questions weren't asked so the answers were not part of the evidence we were reviewing.

Wonder how the trial by judge that was mentioned on the news today will work out.