The problem of scales, self report and attributional style/psychological types

I remember seeing a study into distress or disorder in the UK armed forces. Using the same techniques as were used on civilians soldiers seemed to be equally distressed as the general population. This didn't surprise me much. Their mentality demands they be used to high levels of distress and their self report would be skewed against accurately describing their levels of distress, perhaps.

Put it this way. Take two people and ask them to rate their favourite flavour of ice cream.

Person A says the love icecream. On a scale of 1 to 10 they rate everything as 8 and chocolate as a 10.

Person B says they like icecream. On the same scale they mark everything as 5 but vanilla gets a 6.

Person C says the don't like icecream. They rate all flavours as 3 but rate vanilla and chocolate as 2.

Person A and person B have shown their favourite icecream but the absolute scores are very different. They've expressed the same preference for liking all icecream the same apart from their favourite. Person A has rated their favourite 2 points above all the others. Person b has rated their favourite 1 point above.

From this example I hope its possible to see the two different styles. Person a and b are expressing the same thing but with different strengths of expression.

Person C is a different type. They don't like ice cream. Their expression of dislike is strong but their expression of strong dislike isn't significant like person A's expression of strong like. Person c's most hated flavours are 1 point less than the rest. Person a's favourite is 2 points higher.

Though externally the scales are supposed to be measuring the same thing they're not because there are lots of factors - lots of variables - which aren't the same between the three people self reporting on their taste in ice cream.

This is a big problem of science and it gets worse. Let's take a mean average to work out which is the most popular ice cream based on the numbers given (not the expressed preference as a binary value). Person A gave their favourite ice cream 2 more points than the rest whereas person b only gave 1 point because of their personal style. Person C gave vanilla and chocolate one negative point because they were equally disliked. The science could be used to misappropriately say that chocolate is the favourite on average but this is not because it is the favourite. It is because the person who liked vanilla didn't have their expression fully understood by the measurement system.

I'm simplifying a couple of very important problem.s in science. The lack of consideration of these is why psychiatry is often considered a pseudoscience.

There is the problem of the simplicity of self report scales.

There is also the problem of how to use statistics to find the facts.