I know this should be in the 'testing' forum, but it seems dead in there, so I'll post it here first...
I'm part of an online networking group who are generally skeptical of woo stuff. However, they do love their personality tests. Someone has now mentioned that they are 'scientific'.
Red rag.
So, anyone got any suggestions on a testing protocol? Generally these tests are a graph of some sort and then some text explaining the person's traits so I thought we could get five people to answer the questions, then get the testers to do the documents.
First test would be to pass the documents about to all the people and see if they can recognise themselves.
Secondly, I thought we could ask two testers to analyse the exact same documents and see if they came out identical.
Thirdly, just give the same set of results to the five people and see if they rang true - rather like a Barnum statement thing.
Needs work, but I could be close to getting these people to agree to it.
I've always viewed them with suspicion as well, mainly because none of the results are ever negative. I realises that I'm a little lazy and easily distracted but that never seems to be an option in these tests
Anyway, try the ever useful Skeptics Dictionary for more information about the MB tests, most of the others are based on these. As for testing perhaps a time lapse? Get people to do the tests, leave it a few months then get them to do it again - compare the results. Stress can also skew the results of the test so if you can engineer that for a comparison test as well (maybe half of the test normal, half stressed. Or two different sets of test questions)
Witness the fact that you're posting on here when you should be workingOriginally Posted by mongrel
Oops, me too
[forces himself back on topic]
Sarge, I agree this would be happier in the testing forum - it'd make it less dead. Would you like me to move it?
More thoughts:
* The problem with a time lapse approach is that subjects would probably answer the questions differently, and the testers could simply say "Well, people change over time and the test is just reflecting that".
* I reckon we'd need a lot more than five to give significant results.
* How about handing out genuine results to a test group and dummy results to a control group (double blinded, of course) and see if the rate of subjective acceptance is the same between groups?
* I agree that different testers blindly analysing the same test results is a very good check - but aren't these things marked by computer, which would guarantee identical results?
This is what really bugs me about them as well. No matter how hard I try I can never get them to admit I'm a lazy, anti-social bastard who doesn't care about anyone else and would rather spend all day in bed. Instead it's all "prefers to draw their energy from internal sources" and "takes a more relaxed attitude and likes to realise their ambitions at their own pace".
I've always viewed them with suspicion as well, mainly because none of the results are ever negative. I realises that I'm a little lazy and easily distracted but that never seems to be an option in these tests
I have nothing against the idea of personality tests, but everything I've seen seems far too close to the vague feel-good crap that psychics spout, rather than anything actually approaching science. Nobody is perfect (present company excepted of course) and pretending that they are will prevent a test from ever being valid.
From some of the discussions on the JREF forum I get the impression that there are some tests around that show significant correlation to various things, such as career success. Unfortunately these always seem to be a case of screenshots from a version you didn't buy.
I'm not sure to what extent you are joking, but the latter examples are far more factual than the former, which are merely subjective interpretation seen from a traditional social viewpoint.
This doesn't prove that such tests are scientific, but I would much prefer they adopted a non-judgemental, objective perspective than a "You're not as good as us!" approach.
That's very politically correct of you, Araneus. Much more fun just to call Mongrel and Cuddles the lazy bastards they freely admit they are
Seriously though, if you were an employer using something like this for recruiting, wouldn't you find a "warts and all" analysis more valuable - even if that meant being judgemental?
The latter language is not necessarily more objective. It is quite possible for somebody to be genuinely lazy - "relaxed" is not a synonym for "lazy", it is something different, and I don't see why it is inherently any more objective.
This sort of mealy-mouthed wording seems indicative of a non-scientific approach to me. It makes it sound like the output might 'smoothed' to make it more palatable.
I've always regarded that sort of language as part of the huggy-feely, can't say anything negative about people school. IMO most people have undesirable traits, most people though recognise this within themselves and work to overcome it in situations that require social interaction. Some people don't manage to work this out and are labelled slackers, lazy, loud mouthed, gossip etc. and whilst it may be 'hurtful' doesn't make it any less true.
Sounds like you've been stuck in meetings too long :-\
I agree that there is almost certainly an amount of "punch-pulling" going on in order to avoid offending people, but I don't agree that terms such as "lazy" are just as objective as "relaxed".
What you might consider "lazy" I might consider "efficient"; what you might consider "anti-social" I might consider "independent"; what you might consider "loudmouthed" I might consider "honest". It is not simply a case of being mealy-mouthed, but recognising that such terms just do not have any objective meaning.
Unless the test is designed for a very specific purpose, such as recruiting somebody into a particular role which required a certain set of traits, it would be inappropriate to use subjective and judgemental terms in the results of a test which is supposed to be descriptive, not prescriptive.
Indeed, I agree. But why does the same argument not apply to the word "relaxed"?What you might consider "lazy" I might consider "efficient"; what you might consider "anti-social" I might consider "independent" ... such terms just do not have any objective meaning
Again, I agree. However, this is the kind of use to which such tests are being put, I believe. The question is, are they fit for purpose? It all depends on the algorithm being used to generate the words ...it would [generally] be inappropriate to use subjective and judgemental terms in the results of a test which is supposed to be descriptive, not prescriptive
Because "relaxed" is purely a descriptive term, it does not encapsulate any prescription or judgement the way "lazy" does. In theory, it would be possible to objectively measure relaxation through blood pressure, pulse rate etc, which you could never do for "laziness" even with unlimited resources and equipment.
To put it another way, the test result needs to be restricted to "is propositions" rather than allowing "ought propositions". Terms like "lazy" and "antisocial" are veering very strongly towards the ought, not the is.
It is my impression that the tests are fairly standardised, but the results are then interpreted in the context of the test's purpose. This is exactly why non-judgemental terms are required, because it is up to the test administrator to make his own judgement based on the requirements of the situation. A test which pre-empts the recruiter's preferences by using subjective terms is not going to be as widely useful as one which provides a neutral assessment.Again, I agree. However, this is the kind of use to which such tests are being put, I believe. The question is, are they fit for purpose? It all depends on the algorithm being used to generate the words ...
Those are actually extracts from a BUPA employee assistance e-mail about personality types that was supposed to get us to recognise that some people are [strike]lazier[/strike] more relaxed than others and that we should all try to get along. Or something like that.
"Draws their energy from inner reflection"? That sounds objective and factual to you? That's a direct quote. I agree that tests need to stick to what can actually be measured objectively, but there's a huge difference between that and what actually happens. To take another example, I apparently have a choice between being "highly competitive and driven" or "equally ambitious but more relaxed". There is no option of not being ambitious, which is a shame because I'm not. It's not a case of what can be actually measured, it's a case of spouting some feel-good, PC nonsense to avoid offending anyone. The whole point of personality tests is they test your personality, warts and all, it is not to make up stuff that sounds good but ends up twisting the meaning so far that it is no longer just PC, it is just plain wrong.
Edit: Why doesn't strike work? I'm sure that's standard PhP.
Right, update.
I challenged them because they said it's 'scientific'. I said if it's 'scientific' it can be tested, so let's sort a test out somehow.
When challenged, I did the wrong thing. Rather than say "Hey, you're the one making the claim, how about we stick to the subject", I said "It's like a lie detector test" etc. This started a whole new argument.
The person making the original claim never responded to anything I said, someone else picked up the thread (they also did the tests) and challenged stuff.
In the meantime I started reading the Six Thinking Hats thing by Edward DeBono. What a great book! Only read the white hat so far, but he analyses and kicks the bejesus out of what people 'believe' and what are actually 'facts'. Armed with this new insight, the second person defending MB tests actually admitted there was truth in what I said (i.e. it's interpretation, not facts).
But still no hint of a test.
Slippery buggers.
Oh, and we've just had sales training, which on the whole is marvellous stuff and can be measured. I've got figures.
Anyway, it does go a bit woo sometimes. Today we do 'transactional analysis' and my business partner as a 'first born' and having a high 'A' reading or some stuff is an 'organiser', always being able to schedule things, stuff that isn't organised or planned properly doesn't fit with his 'type'.
On his way into the office, he phones me in a panic, his road tax is a week overdue and his MOT expired yesterday.
I laughed so much a little bit of wee came out.
I think it would be pretty easy to test Myers-Briggs scientifically, using the same protocol as used for astrology charts: a number of subjects undergo a test, all of the generated results are anonymised, and each subject has to choose the result which they believe applies to them. If a significant number of participants choose correctly, the test is a success, whereas if the result is no better than chance (as astrology almost always is), the test is a failure.
Posting Permissions
Reply With Quote
Bookmarks