Why I Don’t Trust The Science on Sleep Training

It’s just not good enough to override the evolutionary biology

First, a quick bit of housekeeping. I am interested in learning more about who my paid subscribers are. If you are a paid reader of Motherhood Until Yesterday, I’d love it if you’d take this quick (anonymous) survey and I’ll publish aggregated results in the chat. I’ll also be publishing a longer piece this weekend on frustration with feminism, which will include some of this data. Stay tuned.

…

I had already made up my mind, even before giving birth, that I would sleep train my first child, my son. Why? Because I had heard so many horrible things about parenting and sleep deprivation (and I really like sleeping) and because Emily Oster’s Cribsheet was my bible. At the time, I worked in tech, and like all good little tech employees, I was trained to be “data driven”: I believed that if there was a good peer-reviewed randomized controlled trial in support of something then it must be true. Oster’s review of the data on sleep training was very convincing. According to her book, we have good data to suggest that not only does sleep training work (i.e. babies who are sleep trained tend to sleep more), but it reduces maternal depression and there are no negative outcomes for children. A no-brainer, right?

At this point I just want to remind everyone that although I have been criticizing Oster a lot lately (because we diverge on questions like the importance of breastfeeding and safety of sleep training), I actually have a good deal of respect for her. She’s very good at parsing the data, and I think she’s helped moms make sense of a lot of the VERY conflicting advice we receive. And although I don’t agree with her interpretation of the sleep training data, she also shares a lot of data about the relative safety of bed-sharing. So, credit where credit is due.

But here’s my story, and here’s how—knowing what I now know about human evolution and co-sleeping norms in non-Western cultures—I have come to feel that the sleep training science is highly suspect:

When my son was six months old, I decided it was time to get him to sleep through the night. I was scheduled to return to work shortly and I was stressed about being able to keep up with the demands of my job. The truth is that my son wasn’t waking up that often at this stage (maybe once a night around 1 AM for a long feed and then again early in the morning around 5:30 AM, at which point I usually got up with him for the day). I was definitely tired, but I wasn’t trashed. I was functional. I could have gone back to work without sleep training him, and things would have been fine, but if–as Oster claimed–there were literally no downsides to sleep training, and I could eliminate that 1 AM wake-up, then why would I not do it?

I hired a sleep consultant, who I found through a local birth and lactation-support nonprofit, because I knew I wouldn’t be able to sleep train him without some emotional support. I told her my son’s age and weight, and she told me that given how strong and healthy he was, it would be totally fine for us to go full-on “extinction.” In other words, we were going to just let him cry, for as long as it took, until he exhausted himself and fell asleep. She promised me this was the fastest and most effective way of “training” him, and she would be there for “emotional support” throughout the night via Whatsapp (a service for which she charged an astronomical fee).

This is what we did.

On the first night, my son cried for hours, alone in his own room in his crib. I remember lying in the other room, listening to him, digging my fingernails into the flesh of my palms, burying my face in the pillow to try and drown out the sound, fighting every nerve and fiber of my being, every shred of maternal instinct, that was telling me: just go to him. Hold him. He needs you.

My husband found it difficult too, but I think he was less impacted by it (after all, his brain had not undergone matrescence). And besides, we were paying an “expert” who said it was fine! And Emily Oster said it was fine! We had to stay strong, not let ourselves be “tricked” by our emotional hardwiring.

It only took about two or three nights of this before he stopped waking up. I say “only” because the truth is that from an “outcome” POV it was pretty efficient, but those nights were some of the most painful nights of my motherhood experience, and they will haunt me forever. I wish I could take them back. Even if Emily Oster is right, and there really are no long-term, adverse effects, you couldn’t pay me to do it again. When my daughter came along two years later, there was no question of us doing this to her, of doing this to me.

I am telling you this story because I want to be upfront about the fact that I have a strong emotional bias against sleep training. I find it very hard to believe that there are no adverse effects whatsoever, especially for this kind of aggressive “extinction” approach. But the thing about science is: you either trust it or you don’t. You don’t get to cherry pick which randomized controlled trials you put faith in, unless you have a valid objection to the methodology. And I am a woman of science. I’m a science writer for crying out loud! And I use peer-reviewed scientific research to bolster all of my other arguments: from the benefits of extended breastfeeding to the importance of children having multiple attachment figures to the importance of longer interbirth intervals to the safety of bedsharing. When people disagree with me on various opinions I put forward, I find it hard to engage with them if they don’t bring some credible data or science to back up their dissent. So how can I say that I don’t trust the science behind sleep training without undermining my entire platform and point of view? Without undermining the scientific method?

But I DO think it’s possible to believe in science, to have faith in randomized controlled trials and in peer-reviewed papers, and still be skeptical of the research on sleep training.

First of all, I think we ought to be giving a lot more weight to evolution as a credible source of scientifically valid information. For example, it should come as no surprise to us that extended breastfeeding is really good for mothers and babies, given that this is what the mammalian female body evolved to do over millions of years. Similarly, it should come as no surprise to us, given the compelling evidence from contemporary hunter-gatherer societies for communal childcare, that children thrive when they are cared for by multiple, loving adults and not just the mother.

It’s when science doesn’t line up with evolution that I tend to get suspicious. For instance, public health officials and the food industry were, for decades, telling us that cholesterol is bad for us (never eat eggs!), even though hunter-gatherers ate tons of cholesterol and had extremely low rates of cardiovascular disease. Only later did science catch up on that one (although in many cases, doctors who went to med school 50 years ago are still doling out outdated advice).

Sleep training is another one of those areas where there seems to be a major mismatch between how humans evolved to sleep and respond to their babies and what the contemporary randomized controlled trials have to say on the matter. And therefore, we have reason to be suspicious.

As Sarah Hrdy explains in Mothers and Others, every single primate baby sleeps in close physical proximity to its mother (and to her breasts). There are no known exceptions. Even in those few primate species where the fathers do most of the daytime care, the babies are back with their mothers at night. She says, “mother-infant co-sleeping may be as close to a primate universal in childcare as can be found.”

Humans are primates, and there is no known hunter-gatherer society in which mothers sleep apart from their babies. They always co-sleep, in close physical proximity, so that babies can access the mothers’ breasts at night. This kind of close-contact co-sleeping is so common that mothers in hunter-gatherer societies are genuinely confused by researchers asking them why they do it. There is simply no other way! In one fascinating study of mother-infant sleep habits among the Hadza, women were asked, “why do you co-sleep?” to which they all replied, “because that’s how you sleep with a baby,” or simply, “because she is my baby.” Duh.

Doesn’t all of that evidence have to count for something? Most evolutionary biologists think it should count for something. For instance, Nikhil Chaudhary, a researcher at the Cambridge who studies the Agta and Aka hunter-gatherers, explains in this editorial piece on children’s mental health that “children in [Western] societies live in much safer environments compared to their hunter-gatherer counterparts…[but] babies may be psychologically adapted to expect and solicit caregiving that increased the chance of survival in our evolutionary past.” In other words, being left alone or ignored might not be as dangerous in the contemporary West as it was in our ancestral past, but babies don’t know that, and as a result, “absence of cues indicating caregiver presence can be psychologically distressing for children in [Western] societies, despite their objective safety.”

Still, even Chaudhary says that the debate on sleep training can only be resolved via “further direct empirical investigation.” We cannot completely discount the importance of empirical research in the modern context, even when it goes against the tenets of evolutionary biology.

However, there is an elegant way of taking the evolutionary evidence into account, and factoring it into our assessment of the randomized controlled trials—a way that Emily Oster (as an economist trained in Bayesian statistics), should respect. It’s called a “prior.”

Simply put, a prior is your belief or assumption about the value of something that you take into account before collecting new evidence. Priors can come from previous data or domain knowledge (in our example: the fact that all primates sleep with their infants is a good example of domain knowledge) or even common sense observations or feelings (infants and mothers get really stressed out when infants are left to cry for long periods).

The stronger the prior, the more evidence we need to overturn it. There is a mathematical formula for this that I don’t totally understand and that I won’t get into here, but my husband, who is very good at these things, tells me it is quite elegant.

For example, if you have a coin that you are about to flip, then your prior (based on your domain knowledge about most coins being weighted equally on each side) is that the chance of heads is about 50%. Now you flip the coin 10 times and you get heads 9 times. Does this mean that coins are not weighted 50/50 after all? Is the coin a special coin that favors heads? You don’t know. You compare that data to your prior (your assumption that this coin is a normal coin, and that with a normal coin the odds of heads are 50/50, based on domain knowledge and previous research) and you would realize that you need more data (more flips) in order to figure out whether the coin is biased.

So back to sleep training. How strong, really, is the evidence that sleep training is safe for babies, and is it strong enough to overturn our prior, which is informed by our understanding of primate evolution and biology?

I don’t think so. In this case, I believe the evolutionary science behind co-sleeping (and maternal responsiveness to crying) is so compelling that we would need many, replicated, high-quality studies in order for me to be convinced that it’s totally safe and has no negative effects on children or mothers.

But DO we have many, replicated, high-quality studies on sleep training? In Cribsheet, Oster spends most of her time talking about one randomized controlled trial carried out on 326 Australian children that I am going to spend a good deal of time reviewing here, not because it’s the only one we have, but because it is the most cited, and because I think close examination of the methodology behind this paper helps illustrate how tenuous this field of research is.

First of all, it’s worth saying that, scientifically speaking, there is nothing wrong methodologically with this study. It’s a well-designed randomized controlled trial and the conclusions are statistically sound. It’s just that the findings, on closer examination, are not nearly as convincing as the paper’s abstract makes them out to be (or as Oster interprets them to be). I am also increasingly of the opinion that studying things where outcomes are as complex as children’s mental health are maybe not well served by randomized controlled trials. Double-blind RCTs are GREAT at testing things like vitamin or drug efficacy on certain precise, measurable outcomes: Things like, does vitamin D reduce menstrual cramps (as I wrote about here)? But I am not sure we are all that good at indexing children’s behaviors and mental states, such that we should be relying on RCTs as the ultimate arbitrators of parenting decision-making.

Let’s look more closely at the study:

The study design was relatively simple and straight-forward. The researchers used a database of mothers and babies living in the greater Melbourne (Australia) area to identify a pool of parents who claimed to be struggling with sleep. They sorted these parents into an intervention and control group, and starting at seven to eight months of age, the parents in the intervention group received training and information on how to sleep train their babies. Parents were given information on two sleep training techniques: “controlled crying,” in which the parent gradually extends the amount of time they allow a baby to cry and “camping out” whereby the parent stays with the baby by the crib until they fall asleep on their own. It was up to the parents which technique they chose to implement, but almost all of the parents in the intervention chose to try one or the other. Mothers maintained sleep diaries for the two weeks following the intervention and then completed questionnaires at 4 months, 7 months, 10 months and 12 months post-intervention.

First of all, sleep diaries and questionnaires (kinds of data collectively known as “self-reporting”) tend to carry less weight in the scientific community, because they are highly subject to bias and reporting errors, but let’s assume these were well done. What did they find? After adjusting for potential confounds, the odds of reporting a sleep problem in the intervention group were 42% lower at 10 months and 50% lower at 12 months compared with controls. So the intervention appears to have worked: babies in the intervention group slept better. This does not surprise me. In my personal experience, sleep training my son “worked” in that it DID get him to sleep more. But the efficacy of sleep training has never been a major source of contention. The controversy is over whether it’s okay for babies.

Still, without getting into long-term effects, I think the “success” of the intervention is somehwat questionable. Yes, babies slept more, but the effect on mothers was mixed. Even though babies slept better, mothers in the intervention group didn’t report sleeping that much better: at 12 months after the start of the intervention, 52% of mothers in the intervention group still reported poor sleep, while 63% of mothers in the control group reported the same. The study also found that maternal mental health was not much better in the intervention (average SF-12 mental health scores for intervention and control mothers were 50 and 46, respectively) which the study made a big deal about in the abstract, but which really seems like a very minor difference (only 4 points on a 100-point scale). Finally, physical health was actually worse in the intervention group, something that did not come up at all in the abstract.

So basically, if we really stick to the findings without getting our head all the way up our ass here, what the researchers ACTUALLY found is that when mothers receive INFORMATION on a relatively gentle sleep training intervention, which involves “controlled crying” or “camping out,” and which was not implemented until babies were SEVEN OR EIGHT MONTHS OLD, then there IS positive effect on the quality of infant sleep, but their MOMS (the supposed primary beneficiaries of the intervention) don’t sleep all that much better, are almost just as depressed, and somehow end up with slightly worse physical health. Not exactly a huge win IMHO.

Then there is the famous follow-up study, done by the same research team, of these same children at age six. They mailed parents a questionnaire, conducted a 40-60 minute behavioral assessment in the children’s homes, and then collected a saliva sample to measure cortisol levels. The goal was to assess whether or not the sleep training intervention had any measurable effects on children’s mental health, sleep, psychosocial functioning, stress regulation, and child-parent relationship five years later. Basically, they found no differences in any of these measures between the control and intervention groups. Conclusion: sleep training is SAFE! Go ahead and do it any way you like.

Please note that when I say the six-year follow-up study found NO DIFFERENCES, between the control and intervention groups, this was true for SLEEP as well. In other words, whatever small improvements the original intervention had on children’s sleep quality had completely disappeared by age six. There were as many children with reported sleep problems in the intervention group as in the control. Oops.

Now (I am so sorry guys but I am about to get VERY snarky here), I would like to call your attention to the utter ridiculousness of trying to assess, in the words of the researchers, a child’s “mental health, sleep, psychosocial functioning, stress regulation, and relationship with his parents” in a 40-minute home visit. If a researcher came to my home to assess these things in a 40-minute window, then depending on whether or not we are having a good day or a bad one, they might come to radically different conclusions. For example, if they had visited my home at approximately 8 PM last night, they probably would have concluded that we are all clinically DERANGED. Tonight, however, we are more or less functional. That’s how parenting goes. Of course, skeptics will say, they had the questionnaire to back this up! But again, self-reporting is usually used with caution by researchers, especially on things as complex as children’s mental health. We are relying on some parent self-reporting alongside a 40-minute home visit in order to generate a complete picture of the mental well-being and behavior of a child.

Okay okay, you might be thinking, but what about the CORTISOL? That doesn’t lie! But again: one cortisol sample? One? I mean, my cortisol is all over the place on any given day, depending on various environmental triggers. Do you really think that there is ANY chance that a crappy sleep training information class given to parents SIX YEARS AGO is going to have MORE of an effect on a child’s cortisol, as measured by ONE SAMPLE taken today, than say, whether or not he spent too much time watching SpongeBob SquarePants in the last hour? (If you want to go deeper on the relationship between sensitive caregiving and cortisol levels in children, then I recommend this paper, which explains that the interaction of caregiving patterns and cortisol patterns in children is COMPLEX. Less sensitive caregiving does have an effect on cortisol, but it has an effect on cortisol PATTERNS over the course of a day more than on absolute levels, and this pattern depends on things like the child’s temperament, in addition to the kind of care they receive. Furthermore, neglectful parenting can both blunt AND enhance cortisol levels, meaning that these effects will be lost in the noise of looking at averages. In other words, comparing ONE cortisol sample per child across intervention and control groups is unlikely to yield any kind of meaningful results).

But to me, the real kicker is what we are comparing this sleep training intervention TO, which is to say, the control group. The entire population in this study is a self-selected group of families who are having trouble getting their infant to sleep, so both the control and the intervention group are drawn from this pool. Only 20% of Australian families intentionally bed-share, and I’d be willing to bet that very few of those ended up in this study. So what we are comparing–when we look at the social and behavioral outcomes of the control versus the intervention group–is a group of parents who are probably putting their babies to sleep alone, in a crib, perhaps in a separate room, and who are struggling to get them to sleep through the night that way (and probably letting them cry a lot because that’s just how it goes), versus parents who are doing that same thing but with a little more education on how to “train” the baby to sleep better. In other words, if we wanted to assess whether the biological norm, as described by Sarah Hrdy, of sleeping in intimate contact with your baby (versus NOT doing that) has any effect on children’s mental health or behavior, then this study tells us nothing. None of the sleep training studies can tell us anything about this, because bed-sharing is so pathologized in Western society that we hardly have any research on it.

So, do I feel that the evidence from this study is so compelling that it overrides my prior, informed by millions of years of mammalian evolution, that mothers and babies were meant to sleep together?

Nope.

But what about the other research? Surely there must be other research out there that reaches this same conclusion, or Oster wouldn’t feel so confident claiming it’s safe, right?

There ARE other studies out there that conclude sleep training is safe, but for the most part, just like the study above, the “intervention” is just some education for parents, who may then choose to implement it or not. Typically these interventions are quite gentle: in many cases, it’s just about improved bedtime routines and sleep hygiene, or about having a more regular sleep schedule, so it really begs the question of what do we even mean by sleep training. There will probably never BE a randomized controlled trial of cry-it-out sleep training, implemented at 100% in the intervention group, because such a study would not be considered ethical. And really, doesn’t that kind of say it all?

The problem is that the people who popularize the science take the conclusions of the papers, which say things like “sleep training is safe,” and then, without reviewing the details, extrapolate to say that it’s fine to sleep train your baby pretty much any way you like. That is not what the research says. We have no idea if what I did to my son is okay, and I will forever be asking questions.

The other thing worth considering is why so many mothers feel totally sleep deprived and unable to cope in the first place. Breastfeeding, co-sleeping hunter-gatherer mothers apparently don’t feel sleep-deprived in the morning, as I discussed here (at least not any more than anyone else in the group). Maybe part of the reason that we have such a need for these kinds of sleep training interventions is because we don’t teach mothers how to “breastsleep” (sleep and breastfeed at the same time) for fear of increasing the rate of SIDS (more on that to come). Maybe if that were the starting point, we wouldn’t need sleep training at all (or at least, much less). I’ll admit that I’ve spoken to mothers who tried breastsleeping and said it was a disaster (and needed to explore other options, like separate sleep surfaces combined with gentle “sleep training”/sleep hygiene techniques). That’s fine too. Not everything that works in the hunter-gatherer context translates perfectly to our modern lives. Sometimes we may not even know why.

I’ll add that, as always, I have no judgment for individual mothers. Mothers are just doing their best. Your health and well-being count too. If you are exhausted, can’t keep up, know that bedsharing isn’t right for you or don’t want to take the risk (again, more on that later), then encouraging your baby to sleep on a separate surface, using gentle, age-appropriate sleep hygiene techniques, might be the right option. I just don’t think it should be the default approach for all moms and babies. And please, don’t do like I did and override your strongest maternal instincts because of one (misinterpreted) randomized controlled trial.

I think we haven’t heard the last word on this, but given the research we have at this stage, wouldn’t you rather err on the side of giving your child too much love and comfort (provided it doesn’t cost you your sanity)?