The consequences of sampling bias

By Angus "Andrea" Grieve-Smith

I wanted to go into a bit more detail about something I've mentioned before: that the use of non-representative samples can cause problems down the line. To illustrate this, I want to examine the claims of health disparities that Emilia Dunham lists in her Bay Windows article.

1. Transgender people take more hormones and have have more surgeries than average.


2. Transgender people smoke at a 30% prevalence rate, and use other substances to cope with the stress from discrimination.


3. We’re more likely to suffer from depression and anxiety, and more likely to live with HIV.


4. 61 - 64% of transgender people have been physically or sexually assaulted.


5. 41% of transgender people have attempted suicide.


6. All these percentages skyrocket for transgender people of color and low-income folks.


7. A startling 1 in 5 transgender people have experienced complete refusal of services from healthcare providers.


8. If transgender people aren’t referred to with correct names or pronouns or are treated with coldness, they may avoid the office.

Of these statements, only the last one is an existential statement. All the others are statements of prevalence or likelihood that are not generalizable without a representative sample. In my impression, some of them are more likely to be true of the entire transgender population than others. There are chains of causation from transgender actions to these disparities, and the chains are not all the same. Here are some possible causal chains. They are not the only possible ones, but they are the ones that seem likely to me.

First there are the inherent consequences of transgender actions: more hormones and surgery. If you're only concerned with transpeople who choose to take hormones and undergo surgery, then of course this is true. But if you believe that not all transpeople choose hormones or surgery, and you don't know how many do, then you have no way of knowing how great these disparities are.

Then there is harassment based on perceptible differences: physical and sexual assault. A lot of this has to do with passing - as one gender or another, not necessarily the one you prefer. The passing does not have to be total: a transperson can avoid a lot of harassment simply by avoiding being noticed. However, note that there is a feedback loop here regarding socioeconomic status: wealthier transpeople can afford higher quality hormones, surgery, hair removal or attachment, clothes, padding, cosmetics and training that can give them (us) a better chance of passing as the target gender.

There is also discrimination based on records or perceptible differences: refusal of healthcare service. There can also be housing, consumer and job discrimination, which can affect some of the factors below.

A transgender person has a number of potential reactions to the harassment or discrimination described above, including: avoidance of healthcare providers, depression, anxiety, substance abuse, suicide attempts. Out of fear of discovery, many transpeople engage in hidden sexual activities, where there is a greater risk of HIV infection.

Completing the vicious cycle I described above are the consequences of poverty, which may in turn result from discrimination: there is greater likelihood of harassment and discrimination (and the consequences that follow from that harassment and discrimination) and sex work (which increases the likelihood of HIV infection).

I know from personal experience, from friends' anecdotes and from online reading that these disparities do not affect all transgender people. Some people do not choose hormones, some do not choose surgery. Some never take publicly visible transgender actions, and others pass well enough, so they are never harassed or discriminated against. Some are able to deal with the harassment or discrimination they experience without resorting to depression, anxiety or substance abuse, or attempting suicide (which is not a judgment against those who are unable). Some are able to avoid unprotected sex. Some are wealthy enough to avoid the consequences of poverty.

Here's the problem with sampling: Dunham and other researchers have no way of knowing for sure whether they've oversampled from those who choose hormones and/or surgery; those who take publicly visible transgender actions; those who don't pass enough of the time to avoid harassment or discrimination; those who already have tendencies towards depression, anxiety, substance abuse, suicide or casual sex, for unrelated reasons; and those who have lower incomes. After all, these are precisely the populations that public health researchers are more likely to come into contact with. Without representative samples, they can never prove that these disparities exist to the extent that they claim.

Now I want you to imagine that these researchers actually have been oversampling these higher-risk populations. On one level the consequences are minimal: if these are the populations with the greatest need, then it's just another way to spend public health dollars on the people who need them the most. But on the image level and the credibility level, there are problems.

I've seen on the Web and on television that some people have a stereotype of "tranny" that combines all these factors: a drug-addicted, unpassable, mentally ill hooker with bad plastic surgery. Some people use that stereotype to justify harassment and discrimination against transgender people, and some family members fight against accepting their relative's transgender feelings because they fear that this will be their fate. These kinds of unsupported survey results feed into those stereotypes.

What if at some point someone does succeed in doing a representative survey, and finds that the drug-addicted, cigarette-smoking sex workers are a small portion of the transgender population, and that the average transgender person is a drug- and disease-free, well-adjusted, successful computer technician making $60,000 a year? What if all the transgender health money was actually better spent on overlapping programs that would serve the needy population just as well? I think someone might feel cheated, and I think there might be a backlash.

There's also the possibility that we might be missing out on some valuable information. What if we found that there were people who had the exact same background, and the exact same transgender feelings, but one group became drug-addicted HIV-positive sex workers and the other became successful computer technicians? We could examine the populations and see what made the difference between health and sickness. It might not be the obvious solution.

This is why we need representative sampling, and this is why you need to comment on the proposal and tell that to Secretary Sebelius.