On Megrahi and evidence

[Tony Williams]

I'm not convinced that identification of someone you know is much better. I sometimes see someone I know in the street. Until I get a proper look and realise I'm wrong, of course. I've also been known to walk straight past people I know really well in the street (and I never hear the end of it!). Context and suggestion play an important part in what we see. We see what we expect to see and miss what we don't.

That's true - context makes a huge difference. I have often failed to recognise people I know but have met in unexpected circumstances - I know that I know them, but I can't work out where, or who they are.

[Tony Williams]

Traditionally the gold standard?

Depends on how far back you look. Many decades ago, all it took was for a witness to point at a defendent and say "I saw him do it" and he would almost certainly be found guilty.

Also, generally speaking, I'd have thought that even people who might be convinced by a confident eyewitness if that was the only evidence present would normally tend to give more weight )possibly *much* more weight) to forensic evidence, not merely from an acceptance that eyewitnesses may be dishonest, but also from an acceptance that they may be mistaken.

Ironically, sometimes people give too much credence to forensic evidence. This is not usually absolute, but has varying confidence levels. For instance, DNA matches are often made by comparing only a very small part of the genome, and it is increasingly being realised that this may be misleading (especially when DNA from different people is mixed together). And of course there's the notorious "cot death" probabilities, plus the "shaken baby syndrome" which is now being questioned.

There is a willingness on the part of juries to believe expert witnesses who state their views with (often misplaced) confidence.

[tolman]

Depends on how far back you look. Many decades ago, all it took was for a witness to point at a defendent and say "I saw him do it" and he would almost certainly be found guilty.

I think you'd probably have to go a fair way back to find a time when eyewitness evidence typically trumped all other kinds of evidence.

If you go back far enough, you might find a time when there wasn't much *apart* from eyewitness evidence, pretty circumstantial evidence (the victim was stabbed and this guy owns a knife), and more solid evidence (we found easily identifiable stolen goods in this guy's room, and in his pockets). Even in those situations, it's possible that the stolen goods would be seen as as reliable an indicator of guilt as an identification.

Of course, if you go back far enough, you could also find that with people being less mobile in the past, eyewitness evidence would more frequently have been given by people who actually knew the person they were talking about, where it could have been rather more reliable than identification of strangers from among a rather wider population.

Ironically, sometimes people give too much credence to forensic evidence. This is not usually absolute, but has varying confidence levels.

But still generally rather better than things like eyewitness evidence.

[QUOTE=Tony Williams;90794]For instance, DNA matches are often made by comparing only a very small part of the genome, and it is increasingly being realised that this may be misleading (especially when DNA from different people is mixed together).

And of course there's the notorious "cot death" probabilities, plus the "shaken baby syndrome" which is now being questioned.

Generally, things that apply to relatively few cases might take a while to be properly questioned (though the simplistic cot death probabilities didn't take much time or many cases to fall apart).

There is a willingness on the part of juries to believe expert witnesses who state their views with (often misplaced) confidence.

Though in general, that is at least founded on a history of generally being right, possibly rather more often than contested eyewitnesses.
It's also maybe related to an assumption that the justice system does exercise some kind of quality control on expert witnesses, which if imperfect in practice, has at least the potential to be pretty good in principle.

[Tony Williams]

It's also maybe related to an assumption that the justice system does exercise some kind of quality control on expert witnesses, which if imperfect in practice, has at least the potential to be pretty good in principle.

Not much comfort to those on the wrong side of the imperfections...

The general point I am making is that there are many instances of juries being convinced by evidence which is less rock-solid than they are led to believe. What is really needed is a general appreciation of the different confidence levels which apply to different forms of evidence. This is increasingly widely understood where eye-witness testimony is concerned (although not widely enough, as the Megrahi verdict demonstrates), but the limitations - and sometimes misuse - of forensic evidence of all kinds are not generally known. This applies to DNA, fingerprints and even (I recall reading a while back) the way in which fires spread.

Sometimes the wish for certainty in evidence leads to practices which are obviously flawed. Many years ago I recall reading of a case of a fatal accident in which the police stated firmly that a vehicle was travelling at a certain speed before braking - they knew this because of the length of the skid marks (this was before the widespread use of anti-lock brakes, obviously). It was clear from the evidence that the police had a simple scale which said X feet of skid mark equalled Y mph speed. I thought what??? That's ridiculous - it will vary a lot depending on the car, how it is loaded, the tyres, the friction coefficient of the road surface and the weather. Such a statement could only be made if the same car with the same load on board had actually been tested on the same stretch of road in the same weather conditions - which of course it hadn't been. But this evidence was accepted in court.

[tolman]

Not much comfort to those on the wrong side of the imperfections...

But we weren't talking about perfection, just relative confidence, and relative reliability.

Sometimes the wish for certainty in evidence leads to practices which are obviously flawed. Many years ago I recall reading of a case of a fatal accident in which the police stated firmly that a vehicle was travelling at a certain speed before braking - they knew this because of the length of the skid marks (this was before the widespread use of anti-lock brakes, obviously). It was clear from the evidence that the police had a simple scale which said X feet of skid mark equalled Y mph speed. I thought what??? That's ridiculous - it will vary a lot depending on the car, how it is loaded, the tyres, the friction coefficient of the road surface and the weather. Such a statement could only be made if the same car with the same load on board had actually been tested on the same stretch of road in the same weather conditions - which of course it hadn't been. But this evidence was accepted in court.

Is that a case of a wish for certainty, or just an attempt to have some kind of objectivity (measuring skid marks rather than just saying 'long' or 'short').
IIRC, there were people in the 70s doing work on skid patterns, and being used as expert witnesses, and I'm sure the guy I read about then was doing reasonable research.

Was the evidence actually challenged in court?
If something's obviously flawed, isn't it up to the defence to point that out, maybe calling their own witnesses?
Assuming, of course, that the evidence is not merely unreliable, but that its conclusions are significantly wrong, which are not necessarily the same thing.
I'd suggest that sometimes, things can fail to be challenged not because they are assumed to be right, but because challenging them successfully isn't likely to make much difference to the outcome for the client.

Anyway, that's not an argument against expert evidence as a whole, just an argument for quality control on experts, and competence in defence lawyers.

[Tony Williams]

But we weren't talking about perfection, just relative confidence, and relative reliability..............Anyway, that's not an argument against expert evidence as a whole, just an argument for quality control on experts, and competence in defence lawyers.

I agree entirely. As I've said before, my concern is that juries may be misled by expert evidence which is presented without any information about confidence levels.

The New Scientist has just run a two-part study on the uncertainties around DNA evidence, particularly where the sample is incomplete and two or more are mixed together. In one actual court case the odds against an observed DNA match were stated to be 95,000:1 by one professional DNA analyst and 48:1 by another. Further research by other analysts after the trial resulted in figures of 13:1 and 2:1, depending on how the sample was analysed statistically. Now there is one hell of a difference between 95,000:1 and 2:1, but the jury was not given this information.

In another test carried out as an experiment, a number of DNA labs were sent two samples and asked if there was a match. Some said yes, others no, others maybe. But what normally happens is that one analyst will go to a trial as an expert witness and give his opinion which is often taken as gospel by the jury. Neither judges nor lawyers usually know enough about the technicalities of DNA analysis to challenge the expert.

Which brings me back to the point I have been making, which is that evidence (whether eyewitness or forensic) should not be presented in court without the confidence levels being explained.

[tolman]

I agree entirely. As I've said before, my concern is that juries may be misled by expert evidence which is presented without any information about confidence levels.

Surely there's as much of a problem when information about confidence is presented which itself isn't necessarily accurate, whether through incompetence of the expert, or simply a lack of knowledge at the current time.

There's also the issue of precisely how confidence and probability are presented and explained, which can require a preexisting understanding of probability in general, or an attempt to educate jury members (where the odds are reasonably good that some of them aren't particularly bright) in probability.

For example, if there's a true 1 in 100,000 probability that a clean crime scene sample would match a random citizen by chance, and there is a match to a given defendant, the correct interpretation of that 1 in 100,000 probability depends totally on how the defendant came to be identified - were they already a suspect for quite seperate reasons who then had a DNA sample taken, or were they found from a DNA database?

In one actual court case the odds against an observed DNA match were stated to be 95,000:1 by one professional DNA analyst and 48:1 by another. Further research by other analysts after the trial resulted in figures of 13:1 and 2:1, depending on how the sample was analysed statistically. Now there is one hell of a difference between 95,000:1 and 2:1, but the jury was not given this information.

At least if the figures are quantified, there's the chance that successful appeals could be launched (assuming that there wasn't sufficient other evidence to ensure conviction).
In the case of eyewitness evidence, where someone says 'I'm sure it was him', there's relatively little that could be done later to correct errors unless maybe if the eyewitness made a compelling retraction.

Neither judges nor lawyers usually know enough about the technicalities of DNA analysis to challenge the expert.

If you and I know about it, that rather seems to be as much of a failing on the part of lawyers as on the part of experts.