Achilles, greatest of the Greek Heroes, is on an early-morning run along the coast of Thessaly when he sees his old friend the Tortoise rubbing his beak against a piece of floss tied between two posts of an old fence.
ACHILLES: Hey, Torty! Ready for a rematch?
TORTOISE: Not today, I’m afraid.Today’s my flossing day.
ACHILLES: But haven’t you heard? You don’t need to floss any more.
TORTOISE: I’m afraid I do. I had some strawberries this morning.
ACHILLES: Well, not you personally, you’re not even a mammal. But people don’t. The new dietary guidelines no longer tell you to floss. The Associated Press looked at all the evidence, and it turns out flossing doesn’t really reduce plaque much.
TORTOISE: Well, flossing doesn’t reduce plaque much in the context of randomized controlled trials encouraging people to floss. It’s not the same thing.
ACHILLES: Isn’t it?
TORTOISE: Look, do you floss?
ACHILLES: I…rinse my mouth in sea water and then rub a piece of pumice against my teeth. I don’t like flossing.
TORTOISE: And if you were in a medical study where they told you to start flossing, would you suddenly start?
ACHILLES: I might.
TORTOISE: You might, it’s true. Or maybe Diomedes wakes you up early one morning, says “me and the boys were thinking of a raid how about it” and before you know it you’ve got your bronze greaves on and your ships are cutting through the wine-dark sea before you remember you haven’t flossed.
ACHILLES: It’s possible. But I don’t see why just because I-maybe, hypothetically– wouldn’t follow through with flossing means that RCTs aren’t any good at telling us if something is helpful. Isn’t that the whole point of evidence-based medicine?
TORTOISE: RCTs are very valuable, but you have to be clear about the kinds of questions they can answer. If you’ve got a new pill you’re going to give somebody for ulcers, maybe not everybody will remember to take it every day, but being given the pill is the intervention you want to test out. People in pain are going to take some kind of pill to make it go away, and you want to see if this is the one that will be most helpful. Lots of questions aren’t like that, flossing included.
ACHILLES: Why not? People want their teeth to be clean, don’t they?
TORTOISE: Exactly- people already want their teeth to be clean. People already can afford floss if they want to get it. People already have been told their whole life, more-or-less, that they should floss. To a large degree, if they’re the kind of person who can follow through with flossing, they’re already doing it. So if you go and put up signs around your medical school telling people they can get paid for a study of flossing if they don’t already floss and then you randomly assign them to be told to floss or not, you’re not testing the effect of flossing, you’re testing the effect of being told one more time to floss if you’ve already proved that you don’t like to do it. Maybe it’s not even that good for you personally, but that doesn’t mean it’s not good for most people who are already flossing.
ACHILLES: Somehow I have a feeling that you think this applies to more than just flossing.
TORTOISE: Of course. Why haven’t you ever gotten married?
ACHILLES: Hi Mom, I didn’t know Hera had turned you into a tortoise.
TORTOISE: No really, why haven’t you gotten married?
ACHILLES: I’m…not the marrying type. And I don’t think Patroclus would understand.
TORTOISE: So some people are the marrying type, and some people aren’t. So who is the marrying type?
ACHILLES: Odysseus, Nestor, Menelaus maybe. Well, Menelaus is the marrying type, but I’m not sure his wife Helen is.
ACHILLES: Nah, he was better off staying single, I think.
TORTOISE: Fine, fine. So it’s good for Odysseus to get married, and it would be bad for you to get married, and it was bad for Agamemnon to get married. And if I found a bunch of Achilleses to take part in a randomized trial–
ACHILLES: There is only one Achilles, Torty.
TORTOISE: Find a bunch of Achilles-like-people, not the marrying type of people, who haven’t already gotten married, and randomly assign half of them to be told how great it is to get married, and how you have to talk about your feelings and listen to your partner and everything, do you expect it to make a big difference?
ACHILLES: That’s just, whatchacallit, “external validity.” Everybody knows that randomized trials only have internal validity, they only can tell about the effects on a very similar population. But so what? Baby steps, baby steps towards a better understanding of the world. Worry about external validity later. Collect all the different populations that you have RCTs on, and synthesize their findings.
TORTOISE: It’s possible that might work, and again for “giving people pills for their ulcers” type problems, it’s probably a good idea. But lots of things aren’t like giving people pills for ulcers. The basic problem with thinking you can solve problems with RCTs is that it takes everything from the perspective of the experimenter, the “policy-maker.”
ACHILLES: What’s wrong with that? Don’t we want policy-makers who believe in science and follow it where it leads?
TORTOISE: We want policy-makers who treat people like people and don’t pretend that being told to do something, whether to get married or to floss, is the same thing as doing it, or that people don’t, most of the time, know their own interests pretty well. Getting a professional flossing five days a week did reduce plaque and cavities, incidentally. And now I think it’s time for you to be on your way.
TORTOISE: You’ve been tapping your foot faster and faster for the last five minutes. You may not be the marrying type, but you’re definitely the running type.
ACHILLES: And I think it’s time for you to get back to flossing.
TORTOISE: So I convinced you?
ACHILLES: No, you just have a huge piece of strawberry on your beak.