The video above describes something known in psychology as the Implicit Association Test (IAT). Keon, the dude in the video, does a good job of explaining what the test is and why it is interesting so watch and gain in understanding.
The reason why the IAT is interesting (for me at least) is that it can have quite startling results. The example Keon has chosen for his video is a racism test. Effectively, the test will determine whether the person taking the test has an implicit racism. This is not a conscious, directed, racism that might motivate someone to join a hate-group, or take some deliberate racist action, but instead a sub-conscious, implicit, racism that might direct behaviour that is more instinctive or reactive. In other words, the IAT detects racism that the person being tested won't necessarily even know they have.
The point is, you could be as tolerant and open minded as you want to be, but you live in a society that has racist undertones in it and this will have influenced you. Watch the video, take the test and see how much.
Be prepared to be surprised by the results though (whatever your own race or view on racism happens to be)...
.
.
.
.
.
.
.
I happen to be friends with Keon and he posted this on Facebook a few weeks ago. I took the test in the video and had a couple of immediate objections. Firstly I found both of the final stages of the test difficult and, while I suspected I found the second one harder, I wasn't sure. But secondly, and what I thought was far more importantly I suspected strongly that the ordering of the tests conditioned me. In the process of the test I got used to associating good with white and left and bad with black and right. So I suspected that my difficulty in the final stage had more to do with the difficulty in changing than a genuine difficulty in associating good with black faces.
I mentioned this to Keon and he suggested I try either taking the test again, reversing the steps in the video, or even better, going here and taking a randomised version of the test. I chose the second option and took that test multiple times, with different orderings. I also took different versions of the test. Unfortunately, I repeatedly came out of the test having shown either a slight or a moderate preference for white faces, rather than as I would have hoped, no preference for either. This was entirely independent of the ordering of the test. If you have doubts as well then I strongly recommend that you check out the link above.
I don't rationally associate white with good and black with bad. But sub-consciously, I clearly find it easier to think white=good than I do black=good. I'm glad this test made me aware of this. Some people (especially around my realm of natural sciences) like to view the likes of psychology somewhat disparagingly because of its supposed lack of testability. Well, the people who came up with the IAT have done a stellar job of overcome this supposed boundary and have revealed very interesting things about human psychology.
Your thoughts, as always, are welcomed... (are you sub-consciously racist? were you surprised with the result? Do you doubt the effectiveness of the test?)
Keon has made lots of other videos on psychology. His YouTube channel can be found here.
[Edit: Sesh points out in this comment that even if the test does prove the existence of implicit preferences, it is not accurate to use the word "racism" to describe these associations. I have to admit that, despite my use of the word in this post, I do agree. Is this a far call?]
Follow @just_shaun
I took the test twice because the first time I did it I think I was allowing myself too much time to think about my response (even half a second is enough) and so didn't find the final stage difficult. The second time, however, I tried to respond as quickly as possible and I did find the black=good step a bit tougher than the white=good. I guess I'm as racist as everyone else, but hopefully not so much that half a second's reflection can't overcome it!
ReplyDeleteAlso, I was partially thrown in the third section by the fact that a lot of the white faces looked Kiwi and so linking them to 'good' seemed counter-intuitive...
A very interesting video, though!
I was completely thrown as soon as I had to try and associate left with bad and right with good.
DeleteIt's such a preposterous idea that the test should be thrown out on that basis alone.
I've tried this test four times. Every time I did it, it gave me "African-American" and "Bad" on the same side to begin with (though it switched whether that side was right or left). And every time it first made me associate "European-American" with "Good" and "African-American" with "Bad", before doing it the other way around. It also did the same with the gender test - i.e. it first made me associate "Female" with "Liberal Arts" and "Male" with "Sciences".
ReplyDeleteLife is not infinitely long, so I can't be bothered giving up any more time testing the randomisation procedure they use. But let's just say I am very far from convinced.
In addition, the one time that it didn't say "there were too many errors to give a result", it told me I was "slightly" racist because I was "slightly" slower at associating black faces with positive words than white faces. What sense should I make of this? How much slower? What sort of error bars have they assigned to my speed of association? I understand that the website interface is not designed for real science but just to provoke people a little, but come on. Your initial sceptical reaction was spot on as far as I can tell.
If you're not getting told you have much of a preference for either type of face, perhaps you don't have an implicit preference. *Every* time I've done it I've either had a slight preference for white faces or a moderate preference for them. I can also tell that I'm having to take more time with the good and black option than the other, no matter how hard I concentrate.
DeleteI also just went to check the randomisation and the first one I was given had good and black first, so maybe you were just the one out of sixteen. I hadn't done the gender one, but I just had a play and it was twice male-science first and once female-science first in the three times I looked. I didn't finish any of the tests so I don't know whether I'm implicitly sexist. It does state explicitly at the website that it is a 50/50 randomisation.
I find your doubts as to whether the researchers would analyse the statistics of this correctly to be disappointing. Even a cursory look at the website attached to the demo test would make it very clear that this has been well tested and well thought out. When I wrote about the ISW this week I didn't go into heavy statistical detail either. The four possibilities are no preference, weak preference, moderate preference and strong preference. I would assume that which category one gets put into relates to the distance between the two reaction time means in units of reaction time standard deviations.
As I said, life is too short for me to keep testing out the randomisation. Since that is quite an easy thing to get right, let's assume it is correctly done and I have merely had an unusual experience.
DeleteI can't see anywhere on the website a discussion of how error bars are actually calculated. I didn't do the test enough times for them to use appropriate individual statistics, so if anything they must be using error bars derived from other people's responses. Is it valid to estimate the standard deviation in my response times based on the response times of the population in general?
In fact I have several questions which are not answered. Do they account for the fact that the sample of people taking the online test is subject to a strong self-selection bias? All of the test categories on the website are also ones where I start the test with a strong preference for what I'd like my result to be, and a clear intuition that they're going to say the usual result is the opposite. Can the fact that I'm to some extent consciously trying to achieve one particular result affect my performance? Do they have control tests with less controversial subjects, like a preference between red objects and blue objects?
Do the measured response times for the population in general account for the possibility for evolution of response time as the test goes on? For instance every time I got to the 4th round my concentration was wavering and in an attempt to speed up I often classified black faces as white and vice versa (also young faces as old etc.). So if my time slows because of errors like that, does it say anything about my preferences? Do they account for errors of that type?
I have to say I find it strange that you are disappointed by my scepticism! I admit, my Bayesian prior is modified by the number of pseudo-scientific studies using bogus statistics that I see in the media every week, so I don't approach it thinking "they don't tell me how they did it, but they must have done it right". But I thought scientists should be sceptical about everything (yes, including ISW results - ours and others'!)?
*My* biggest remaining concern is that what the test measures is not preference but mere familiarity. I live in Finland; I never see non-white people! Familiar is good, foreign is confusing. But I also think that, even if that is true, it doesn't make the test itself any less interesting.
DeleteI was disappointed because I guess I had an implicit assumption that your scepticism arose because psychology is not a hard science. If you're just preparing your equally sceptical comments to add to James' Biochemistry posts then I'm not disappointed at all. I don't see why I should just trust that James has properly tested for *all* the possible problems in his field either.
Regarding your first paragraph of objections, it states explicitly at the website in the F.A.Q that the test, administered on one person is not incredibly accurate, especially if the preference/association is small. When a full sample of people is considered, these objections don't seem relevant to me.
And I have to admit that, even though all of your questions in your second paragraph of objections are interesting, they all seem likely to push my preference more towards my desired outcome, not towards the undesired. In fact, the times I only got slight preference for white faces were when tried to pretend I was an angry black man who hated white people (somehow I still had a slight preference for white people though).
All of the objections in you final paragraph of objections are fixed by randomisation (at least with respect to possibly causing a skewness one way or another in which way the population's preference lies).
I just did the age one and the sexuality one. I got my first strong preference so far... for young people (... I'm not sure that one is entirely implicit though...). And I recorded a slight preference for gay people. Apparently the sexuality one is against the trend, but then words like "fabulous", "marvellous", "lovely", etc, do sound kind of gay!
I had already asked Keon if he wanted to write a post about this. I will raise my efforts. I'm sure he would know more about the statistics used and/or not used.
I don't think it is necessarily the case that wanting to get a certain result in this test makes it easier to get that result. I think it is quite possible to have the opposite effect - precisely because when you come to the round you know is important (e.g. associating black with good), you try harder to make sure you get your decisions right, and so take slightly LONGER over every choice because you wait for your conscious brain to make the decision.
DeleteIf I deliberately slow ALL rounds to the speed of my conscious brain, and thus level the playing fields, I get no preference between black and white, and also a slight preference for old over young, etc.
I don't think the connection between the full sample error bars and the classification of an individual is at all as simple as you suggest, but that might require more elaboration than I can provide right now (also I've had three beers).
PS: It's harder for me to try to poke holes in James' posts, but yeah, I don't trust him either. :)
I had intended to imply (if I ignore all your other objections unrelated to error bars) that you can't really classify an individual with any strong confidence at all (unless they are extreme - i.e. me with old people) - which I thought agreed with you *and* the website's F.A.Q... But, this doesn't stop one from deriving meaningful statistics about the population.
DeleteYour other objections seem now to come down to "if someone taking the test consciously does something that messes the test up, the test gets messed up". It does advise you *not* to take extra time to answer the questions. I'm pretty sure I could rig the test to get whatever answer I want by deliberately stalling on certain questions.
I do have a hunch that if you'd encountered the website that administers the test before my blog post or Keon's video that you wouldn't be quite so willing to oppose it, mostly because the website is much more careful about what it does and doesn't claim (and much more honest about what the test implies), whereas Keon and I threw the word racism around like it was a toy. I guess, the post and video would get less attention if written more carefully, but given that we want this blog to be about the actual science, I guess I've learned a lesson
.
.
.
.
(i.e. have a provocative title and then *immediately* backtrack from it in the first paragraph... then you get the readers *and* intellectual honesty).
If you can't reliably classify any single individual, how can you produce meaningful statistics on the classification of the population as a whole?
DeleteTo do so you must at a minimum assume that the standard deviation in response times is independent of the classification of the individual's automatic preferences. This may or may not be true. I can think of plausible arguments why it should not be true that genuine racists and genuinely unbiased people have the same statistical distribution of response times! I cannot however think of a way of verifying this assumption that does not require prior information on the individual's classification from an external source.
I'm going to have to bow out of this conversation, but I don't understand this objection at all. I can think of many ways to extract statistically meaningful information about the population as a whole. Not all of them would require assumptions to be made about the standard deviation in response times. Not all of them would even need to use the individual's standard deviation in response time.
DeleteAnd, if the effect you're expecting is real I could even measure it by plotting the difference in the mean response time for each individual (difference between white=good/black=bad and white=bad/black=good) against the standard deviation and looking for a statistically significant trend. There will of course be a scatter around whatever the true trend is but this doesn't stop me measuring the trend with some sort of statistical significance.
I can't reliably measure the ISW effect in a single pixel of CMB, either. But I can reliably measure the ISW effect in a population of pixels.
Clearly I'm not good at explaining myself in the limited comment space under a blog. I'll have one more go at clarifying the problem I'm referring to here and then we'll have to leave it (or discuss it in person).
DeleteSuppose you have a single random variable X, which takes value in the range (x, x+dx) with probability P(x)dx. Then by measuring values of X in several different realisations, you can hope to reconstruct P(x).
But if you have two different random variables X and Y taking values x and y ("different" random variables means P(x) is not equal to P(y)), and your data consists of realisations of the random variable Z = (X or Y), with an additional probability p that Z=X and (1-p) that Z=Y, then how can you reconstruct P(x) or P(y)? As far as I can tell, you need external knowledge about whether each value is a realisation of X or Y.
Also, P(z) is certainly not gaussian even if P(x) and P(y) are, except in special cases.
The analogy with an ISW measurement doesn't hold, because there your random variable is (X+Y) not (X or Y), plus you also have a pretty good idea of P(y) beforehand because it comes from shot noise. (Actually you also assume P(x) rather than attempting to reconstruct it from the data.)
In your example, P(z) (and its mean, standard deviation, quantiles, median, skewness, kurtosis, etc) is the interesting thing that can be extracted.
DeleteZ is interesting because the world is not made up of those people with preference and those people without, it is made up of people sitting somewhere in a spectrum of preference and I want to know things like what the mean preference is (and yes, I'm assuming the people are doing the test honestly and not deliberately trying to manipulate it to give their desired result). I *can* then look at sub-populations made up of just black people, just white people, just people living in Alabama, just people living in Vladivostok and meaningfully compare each one of those sub-populations - I can tell them apart.
The mention of the ISW effect wasn't meant to be a tight analogy, merely an obvious counterexample to a statement like "If you can't reliably classify any single individual, how can you produce meaningful statistics on the classification of the population as a whole?" which seemed to me to be a general truth you were claiming.
Also, even if somehow the test was measuring something that is not conditioned by the earlier rounds, I have serious problems with the idea of telling people they are "racist" based on the results.
ReplyDeleteThe test repeatedly tells you to go as fast as you can, not to stop to think about anything and so on. They argue that this way they are uncovering some sort of instinctive reaction because we are not allowing our conscious thought processes to take over. But "racism" as I understand it is precisely about conscious actions, or at least actions which take place over a long enough time period that the possibility for conscious thought exists (you can't make people use their brains if they don't want to).
What I'm getting at is that even if somehow society has conditioned me somehow (and perhaps we can think of conditioning mechanisms other than the obvious one of making me do this particular test in this particular order) to be fractionally quicker at associating white faces with positive words, this doesn't mean that I would ever act in any manner that could sensibly be termed "racist". After all, Nature has conditioned me to notice pretty girls, but that doesn't mean I spend my time wolf-whistling at them.
If the test is accurate, I can see it having relevance in one of those imagined scenarios where you have a split second to decide whether to save the white person or the black person from a runaway train or something ... but other than that I'm not sure what it tells us. And I'm far from convinced it is accurate.
The website that has the test seems to be very careful to avoid the words racism/sexism, etc. It only ever speaks of implicit associations and preferences.
DeleteHowever, neither Keon nor myself were so careful and you do have a good point. Calling an implicit preference "racism" is somewhat over the top. I've added a link to your comment in the main text.
But the test itself does have more interest and relevance than your final paragraph allows for. Not every decision I make allows me the time to be careful and completely conscious about what led me to the decision. Fine, if I were deciding who gets a job after a day's careful thought I would hope and imagine that it was 99% conscious and rational. But not all split-second decisions need to be life or death. What if I were a taxi driver choosing passengers, or a police officer breaking up a riot, or a ticket officer checking tickets, etc? It seems there that most of my decisions would be made in a couple of seconds because, either I only have a few seconds, or I need to make many of these decisions every day. If I were in such a position I think it would be useful to know what my implicit (and irrational) preferences are.
Completely unrelatedly, something I found interesting in this video was the fact that white people felt guilty to have an implicit white preferences and black people felt proud to have an implicit black preference.
Or at least a few people in the small selection of people being interviewed felt those feelings of pride and shame (just to pre-empt the obvious and correct claim that this isn't at all statistically significant).
DeleteFinally got around to it and, unfortunately, definitely found white=good slightly easier than black=good.
ReplyDeleteBut I've been aware of my possible sub-conscious 'racism' for a while. For example, I tend to *notice* when somebody I meet is black (please excuse the lazy black/white dichotomy for now; in this case it just makes exposition easier), but not when they're white. I think this comes from growing up in a town where almost everybody was/is of white European/British ancestry. I don't think this actually has any bearing on my actions or decisions though, because these are guided by my rational, conscious mind (Sesh made this point above). As a silly example, if I have to ask for directions, or ask for a chair in a pub, or something like that, I'm no less likely to approach a black person than a white person. So I don't quite know what to think about this test.
Perhaps I just fall into the category of person that Keon discussed at the end, who is aware that their background and environment may suggest certain associations, and can therefore actively correct them. I always feel uneasy when a news story ends with "...is being investigated by members of Operation Trident, who deal with gun crime in London's black community." Even if that is true, couldn't it just be left unsaid?
Anyway, interesting video!
It's not something the test itself can answer (or claims to answer), but I'm not entirely convinced that my conscious mind can/does override every decision I make. Of course being aware of any implicit preference definitely helps, even if it isn't perfect.
DeleteThe dichotomy is excused, for now.
That dichotomy point was one of the reasons I actually first went to the website to do the test. When I tried to do Keon's version I wasn't sure which faces were "white" and which "black" and spent most of my time confused about that. Then, when I did it at the website (where it would tell me if I got it wrong), it turned out my immediate best guess was right 100% of the time. I suppose the dichotomy needs to be made to have a working test. Another interesting amendment to the test, along with Sesh's many suggested ones, would be to gradually whiten/darken the faces and see how the average implicit preferences of the population changed.
Thanks for the comment.