Differential misclassification is probably the most complex topic we teach in introductory epidemiology, and it seems to be the one that students struggle with most. Most of the resources that we encounter don’t explain it very clearly for beginning students, so I thought I would make an attempt. This post is aimed at public health students starting out in epi, rather than laymen – there is a bit of terminology which may be hard for people with no public health experience to follow.
Misclassification is one type of measurement error. It refers to the incorrect identification of a binary variable (e.g. exposed / unexposed or outcome / no outcome). You can also have measurement error with regard to continuous variables, for example weight, age, height, blood pressure etc, but I will not be discussing that here.
In this post I’ll often be using the terms “case” to refer to someone with the outcome, and “control” to refer to someone without the outcome, in a reference to case-control studies. These studies have particular risks of differential misclassification, and this will also help me simplify the language in the post a bit. When I say “first variable” I mean the variable being misclassified (which could be either the exposure or the outcome). If we are talking about misclassification of the outcome, then the “other” variable is the exposure, and vice versa.
What is differential misclassification?
Misclassification comes in two types – differential misclassification and non-differential misclassification. “Differential” in this case refers to differential with regard to the other variable, that is
- misclassification of the exposure that is different for people with and without the outcome, or
- misclassification of the outcome that is different for people with and without the exposure
By “different” in this context we mean more or less likely to occur, since we are talking about binary variables. If you’re like most of my students, those sentences made your eyes glaze over a bit. So let’s look at some examples:
- Doctors might investigate overweight (exposed) patients more thoroughly for cardiovascular disease (outcome) than non-overweight patients, leading to under-detection of heart disease in the non-overweight. In this case the measurement of the outcome (cardiovascular disease) depends on the exposure (being overweight).
- People who know that they have lung cancer (outcome) might report their past smoking habits (exposure) differently to healthy people. See the 2×2 table below – 100 smokers without lung cancer have falsely claimed to be non-smokers, but all of the smokers with lung cancer have been honest.
There’s an important thing to note here – some textbooks say that non-differential misclassification occurs when “all groups” are equally likely to be misclassified, and this can be confusing for students because there are four subgroups in any study (unexposed controls, exposed controls, unexposed cases and exposed cases). So some students think that equal proportions of people from each of these four subgroups have to be misclassified – this isn’t correct. What has to be equal is the proportion of either cases or controls, or exposed and unexposed people, who are misclassified. For example, people with and without lung cancer (outcome) must be equally likely to falsely tell you that they do not smoke (exposure). By “groups”, these authors mean the two large groups according the other variable, not all four of the subgroups.
Non-differential misclassification occurs when the likelihood of incorrectly measuring the first variable is the same with regard to the other variable. For example:
- If everybody is equally likely to be diagnosed with lung cancer (outcome), regardless of whether or not they smoke. Doctors investigate patients for the outcome (lung cancer) equally thoroughly, regardless of their exposure status (smoking or not smoking)
- If everybody under-reports their alcohol consumption (exposure), regardless of whether or not they have colon cancer (outcome). See the 2×2 table below – half of the heavy drinkers have claimed to be low consumers of alcohol, and the proportion is the same in people with and without colon cancer.
Differential misclassification is most likely when the outcome and the exposure have both already occurred, and when the value of one is known to the person measuring the other. For example, I smoked for years, but I do not yet have any smoking-related illnesses – so how I report my smoking history to you today can’t depend on whether or not I will develop lung cancer in future. But if you were to ask me about my smoking after I got sick, I might report it differently – because I would know that I were ill, and this might make me more honest about my smoking history than I would have been when I were healthy.
Some students also get confused on this point – they think because the exposure is associated with the outcome, that people who get the outcome in future will be more likely to over- or under-report their exposure, even though it hasn’t happened yet. But the issue here isn’t whether smokers (who are all at high risk of lung cancer) are likely to report differently to non-smokers (who are mostly at low risk of lung cancer) – it’s whether smokers with lung cancer report differently to smokers without lung cancer. And they can’t possibly, if they don’t have lung cancer at the time that they report their smoking status. At that point in time, before they get sick, they’re all just smokers.
Why is differential misclassification especially bad?
Epidemiologists worry about differential misclassification more than non-differential misclassification because it can cause bias in either direction. Non-differential misclassification can (in almost all circumstances) only ever make an association look weaker than it truly is. This is because non-differential misclassification is essentially random mixing of the two study groups (exposed and unexposed, or cases and controls), which will always make them look more similar than they really are, bringing measures of association down. You can see this in the table below.
In the scenario above, the true odds ratio is 4. In the non-differential example, 10% of both the exposed cases and the exposed controls have said that they were unexposed, bringing the observed odds ratio down to 3.5. In the differential scenario, 50% of the unexposed controls have falsely said that they were exposed, while the cases have all reported their exposure status accurately, bringing the odds ratio up to 10! (Note that you could also have a scenario where differential misclassification resulted in an odds ratio that was nearer to the null value of 1.0 – it just depends which category of participants incorrectly report their status.)
Misunderstanding 1: Mistaking systematic error for differential error
Some students get confused between error that is systematic (that is, always in the same direction) and error that is differential. For example, some students see a scenario where all participants under-report their weight or their alcohol consumption, and they think that this is differential because everybody is under-reporting rather than over-reporting. The important thing for distinguishing differential and non-differential misclassification isn’t the direction of the error, but whether it occurs equally for one variable regardless of the other.
Misunderstanding 2: Non-differential misclassification has no impact on study results because “everything balances out”
I think we often give students this impression since we are so much more worried about differential misclassification. I also suspect that some people get this idea from introductory statistics, if they know that random error does not affect means or proportions in single groups. But here we are discussing comparisons between groups, and that changes things. As can be seen in the 2×2 table above, non-differential misclassification does have an impact – it biases ratio measures towards the null value, because you’re mixing the two groups together and making them look more similar.
Misunderstanding 3: “Bias towards the null” means “The odds ratio gets smaller”
The null value for ratios is 1, not 0, so bias towards the null for ratios means that the observed ratio is closer to 1 than the true value. This distinction doesn’t matter for ORs above 1 (provided they don’t get so small that they’re below 1), but if the true odds ratio were 0.4, bias towards the null would actually result in a larger number (e.g. 0.6 or 0.8), closer to the number 1.
How to tell the difference between differential and non-differential error
Many students eventually feel that they understand the two types of misclassification in theory, only to get stumped by an example on an assignment or an exam. When assessing an example, step back and ask the following questions:
- What is the exposure, when was it measured, and by whom?
- What is the outcome, when was it measured, and by whom?
- Given these facts, could the accuracy of the measurement of the exposure be affected by the outcome (or vice versa)?
If the exposure was measured before the outcome, then differential misclassification of the exposure is very unlikely, since the outcome isn’t known when the exposure is measured – whether I develop lung cancer in 2030 can’t affect what I tell you today about my smoking history. However, differential misclassification of the outcome is still possible, since the exposure happened first and was known when the outcome was measured, e.g. if my doctor investigates me more carefully for lung disease in future because she knows that I used to smoke.
If the people who measured the outcome don’t know the exposure status of the participant (or vice versa), differential misclassification is less likely. This is why we use blinding when possible – to stop researchers or patients reporting better outcomes if they know they are receiving a real drug (the exposure) rather than a placebo, for example.
If the exposure and the outcome are measured at the same time and the people doing the measurements are aware of them both, then differential misclassification is a risk. The next step to try to assess what impact it would have on the results of the study.
You can do this by thinking about whether the misclassification would make the exposure look more or less common than it really is among people with the outcome (or vice versa), and whether this would exaggerate or reduce the apparent difference between the two groups (with and without the outcome). This would then send the odds ratio away from one (more different) or towards one (more similar). Most students find this very hard at first, but it gets easier with practice. Use practice exam questions if you can, and try to think about this issue whenever you are reading a study on an area that interests you.