A top-priority program aimed at enhancing soldiers’ psychological resilience and mental health has proven to be effective, the U.S. Army says in a forthcoming report. However, several leading mental health professionals say the findings do not prove the program works.
Last month, health correspondent Betty Ann Bowser explored the nuances of the Army’s Comprehensive Soldier Fitness program. This new and anticipated report adds one more layer to the debate.
The program, launched in 2009 to teach soldiers how to better handle battlefield trauma as well as stress in their daily lives, seeks to improve well-being and reduce anxiety, depression and post-traumatic stress disorder, all of which have become prevalent among troops who have deployed to Iraq and Afghanistan over the last decade.
At the heart of the program is a 10-day class that non-commissioned officers take at the University of Pennsylvania’s Positive Psychology Center, where they learn about their personal strengths, how to be more aware of their thoughts and emotions, and how to enhance relationships with loved ones so that during tough times they will have better support. Service personnel are also taught techniques for positive thinking and skills to reduce a negative focus when things go wrong.
Officers who finish the class are called “Master Resilience Trainers,” and they, in turn, teach the skills to troops.
The soon-to-be-released report, titled “The Comprehensive Soldier Fitness Program Evaluation, Report #3: Longitudinal Analysis of the Impact of Master Resilience Training on Self-Reported Resilience and Psychological Health Data,” concludes, “There is now sound scientific evidence that Comprehensive Soldier Fitness improves the resilience and psychological health of soldiers.”
The new study (an advanced draft was provided to the NewsHour) follows two others — “Report #1: Negative Outcomes (Suicide, Drug Use, & Violent Crimes)” and “Report Number 2: Positive Performance Outcomes in Officers (Promotions, Selections, & Professions)” — and finds that troops in four combat brigade teams instructed by Master Resilience Trainers “experienced significantly higher rates of growth” than soldiers without such training.
The report also concludes that soldiers in units with trainers did 1.6 percent less “catastrophizing” — worrying that the worst case will happen — than troops without the trainers. They also developed 1.3 percent better coping skills, were 1.3 percent more emotionally fit and were 1.1 percent more adaptable than soldiers in units without the trainers, whom the Army also studied.
But several leading psychiatrists and psychologists disagree. “The findings do not seem to be very impressive,” George Bonanno, a professor of clinical psychology at Columbia University, wrote in an email to the NewsHour. The “report does not support the legitimacy” of the program, he contended. “It’s not clear they actually showed anything.”
Even if the Army study is accurate, “it’s such a small effect one would have to question whether it was worth it,” Bonanno said.
Dr. Bessel van der Kolk, a professor of psychiatry at Boston University, voiced similar concerns.
“This report reads more like propaganda than a serious scientific study,” he said in an email after reviewing the Army study results. “The big question, though, has not yet been addressed: Does this intervention make combat soldiers more resilient and prevent PTSD and somatization [a condition in which a person has many physical symptoms but no physical cause that can be detected]?
“Does it make it easier,” van der Kolk continued, “to tolerate the central traumatizing issues of combat: killing, witnessing or engaging in atrocities, seeing one’s friends being blown up, and being reminded of horrendous scenes after returning home, and being able to sleep comfortably after combat?”
One of the main creators of the Comprehensive Soldier Fitness program, Martin Seligman, said the criticisms were “off base.” The program is largely aimed at “creating a more resilient fighting force — building the strengths of the top 90 percent” of the Army.
The $140 million initiative should not only be measured on how well it meets the difficult goal of preventing PTSD, according to Seligman. The effects of the program “are large” when viewed in context, he said. The training is not only for soldiers who might be more at risk for developing mental health problems but for entire units of troops — even those who might not need it — without any prior screening. That “universal” approach means that relatively small percentages of the entire trained group will see improvement, but this would translate to a significant portion of at-risk troops who have benefited, Seligman said.
Referring to an alternative approach in which only those with the greatest need would be trained, Seligman said: “You don’t know who is at risk and should be targeted, or you don’t want to stigmatize those at risk. Both, I think, are relevant to why the Army is doing universal prevention.”
The manner in which the study was conducted is also being criticized.
According to Bonanno, the study’s design was weak. “If they wanted to conclude that there was something special” about providing units with resilience trainers, then for scientific proof, Army officials should have compared three different situations: units with resilience trainers; units with no trainers, and units receiving training in “how to relax, or how to be better leaders, or just about any alternative.”
Bonanno, who studies the impact of adverse events on individuals, said it is possible that the Army’s program might even have a negative impact.
Absent any specialized psychological training, soldiers are already “remarkably resilient,” he wrote. This program does not “consider the possibility” that it “might somehow undermine” a natural resilience that soldiers have.
“The study isn’t perfect,” acknowledged Capt. Paul Lester, a research psychologist with the CSF program and author of the Army’s report, wrote in an email.
At the same time, he said, the huge volume of troops assessed in the study should lend the analysis enormous credibility. Lester asked how many of the critics “have led a 15-month longitudinal study with 22,000 people on three continents and two war zones.”
Said Seligman: “I don’t know how the Army could have ethically or practically instituted 12 or more hours of placebo to tens of thousands of soldiers in the middle of two wars. Since these soldiers are already getting hundreds of hours of other training, those constitute a pretty fair control [group].”
Seligman also said there is “not the slightest evidence” that the Army’s program could have negative consequences. There have been “7,000 Military Resilience Trainers trained with no report of harm,” and additionally “thousands of trained soldiers deployed with subjectively reported benefits, and no evidence of harm.”
Lester emphasized the investigation’s strengths: It was run on Army units rather than conducted in a laboratory; instead of sampling, they looked at large units that are representative of the Army’s combat mission; and soldiers had anonymity and were not made aware of which group they were in.
In addition, there was a “Chinese wall between the Army, stakeholders, and the data: we contracted for an independent data analysis team,” Lester said. “It is important to note that my team alone interpreted the results without pressure from anyone outside of the team.”
Frank Ochberg, founder of Columbia University’s Dart Center for Journalism and Trauma, said that the analysis would benefit from a third party taking the lead, rather than having Army officials and contractors with a stake in the program’s success evaluating their own effectiveness.
Reports on the effects of CSF training “would have higher credibility if they were conducted by an outside agency with no ties to the program management,” Ochberg said.
The report is also being criticized for relying on soldiers’ self-reporting. Some observers said that a study asking participants to evaluate his or her own psychological state is widely regarded as not fully scientific.
“As the subtitle of the report points out, this is a measure of ‘self-reports’ by the soldiers,” said Bryant Welch, a clinical psychologist and former official with the American Psychological Association. “It is not ’empirical’ evidence of resilience as they claim.”
“All the study shows,” Welch wrote, “is that if soldiers are given a set of attitudinal questions to answer and then are given several months of instruction by a superior officer explaining why the attitude the Army would like them to have is better for them, then, when they are retested, more of them will endorse the answers the Army would like them to have.”
He added: “In other words, the study suggests that the Comprehensive Soldier Fitness has the ability to indoctrinate the soldiers. It says nothing about resiliency and does not even attempt to measure resiliency with any empirical measures.”
The Army disputes this notion.
Asked if soldiers might be providing answers that they think the Army wants to hear, Lester said the data show an even distribution of scores by rank, military occupational specialty and location where troops were surveyed. When Lester and his team looked at the data, they concluded that only 16 percent of the answers were due to the fact that the soldiers did not trust the survey.
There is one aspect of the report that all sides agree on: The results are not based on hard behavioral data. Seligman “wholeheartedly agreed” with van der Kolk’s point that there were no objective measures of domestic violence, suicide attempts, job performance, illness or amount of medication taken, or rates of PTSD.
Lester wrote that his team aimed to look at such objective outcomes next year. He said there was not enough time to do that in this current study.
Photos by Kael Alford/Liaison, Win McNamee/Getty Images, Chris Hondros/Getty Images, John Moore/Getty Images.