Can the Crowd Solve Medical Mysteries?
Frustrated by unsolved medical mysteries, some patients are turning to a crowdsourcing site for answers.
The change in Robert Loop’s twelve-year-old son happened overnight. His wide smile and happy-go-lucky nature vanished into the December night, replaced by anxiety, temper tantrums, and profound distress. Christian’s eyes kept rolling—perhaps not an uncommon behavior for many adolescents—but no matter how hard he tried, he just couldn’t stop. The straight-A student rapidly stopped being able to complete even simple homework assignments. Soon, he stopped attending school altogether and instead spent his days at home, afraid to leave the house.
“All we wanted to know was what did our son have, and how can we treat it,” Loop says.
The Loops immediately took Christian to a child psychologist, who recommended he see a neurologist. That doctor diagnosed him with Tourette’s syndrome, in which sufferers make movements and sounds they can’t control, and obsessive-compulsive disorder, a disorder that traps sufferers in a pattern of obsessive thoughts and compulsive behaviors to reduce anxiety. Behaviorally, these symptoms fit what Christian was experiencing, but most cases of Tourette’s and OCD have a gradual onset. Christian’s did not. Within just a few hours, his personality had been overtaken by a frightful disorder that no one could understand.
In the six weeks since Christian’s symptoms had suddenly appeared, he experienced no relief.
The family traveled from their home near San Francisco to a top pediatric neurologist at Cedars-Sinai in Los Angeles. In the six weeks since Christian’s symptoms had suddenly appeared, he experienced no relief. Countless times each day, his long, sinewy limbs were wracked with anxiety, frustration, and despair. He could no longer get a glass of water without help. His only escape was playing squash, which he did for up to six hours each day. Robert had pinned his hopes on this neurologist having the answer for Christian’s illness. Instead, the neurologist confirmed the previous diagnoses and sent them home by saying, “Welcome to your new life.”
The Loops found themselves in purgatory. They had a diagnosis they didn’t believe, treatments that weren’t working, and a son that had been completely subsumed by his illness. If they could find out what was really wrong with their son, they believed, they could get him the help he so desperately needed. But no one could properly diagnose their son.
Jared Heyman knows the Loops’ frustration all too well. Several years ago, his younger sister was debilitated by an illness that no one could diagnose. Two years, nearly unspeakable agony, and half a million dollars in tests later, his sister finally got a diagnosis and treatment. “I knew there had to be a better way,” he says.
So Heyman set out to create that better way and started CrowdMed in 2013. The site allows patients to submit their cases to the site to be solved by a cadre of medical detectives from around the world. Heyman’s background in economics and the decade he spent as the head of a startup had given him an appreciation for the wisdom of crowds, which states that a large group of people tends to be smarter and more accurate than any single expert. What tricky diagnoses needed, Heyman realized, was a crowd of people doing their best to solve medical mysteries. Which is exactly what CrowdMed provides. He tested the concept by submitting his sister’s symptoms. Within three days, the site’s medical detectives had correctly identified her condition.
In the year since it began, CrowdMed has soared in popularity, with users in 21 countries around the world. Investors have also flocked to the project, including actor Patrick Dempsey (aka “Dr. McDreamy” on the television show Grey’s Anatomy), who recently pledged $1 million. The site’s promise, however, is tempered by concern from bioethicists and physicians alike, who worry about everything from privacy to medical errors.
Heyman remains undeterred. “I’ve seen how ill-equipped our medical system is when it comes to solving tough cases. If large groups of people are so much wiser than individual experts, then why are we still relying on these individuals to make medical decisions?”
Call Me Miss Marple
Patients from around the world can submit their case to CrowdMed for $50, which is refunded when the patient completes the process. They can also offer a monetary reward—usually around $200—to the medical detectives to arrive at the correct diagnosis to increase the attention paid to their case. Patients are assigned pseudonyms, although their locations (state and/or country) are accurate. They provide details on current medications, results of previous tests, diagnoses they may have received, and a history of symptoms. This information is then turned over to the medical detectives, many of whom have a medical background, although one isn’t required.
When I first signed up to be a medical detective on CrowdMed, I felt a curious mixture of apprehension and excitement. Maybe I could decode someone’s strange symptoms . For someone who spends most of her time sitting in front of a computer, tapping out drafts, and repeatedly reloading Gmail for new messages, the thought of actually saving someone’s life was rather intoxicating. Although I had never been to medical school, I do have a public health degree that has familiarized me with an assortment of human maladies. I also have a fair amount of first-hand knowledge courtesy of a chronic illness that had plagued me throughout my 20s and left me more than familiar with blood tests and medical lingo.
Maybe I could decode someone’s strange symptoms.
As I started to review the cases, however, I was overwhelmed. Many of the symptoms were, frankly, rather bizarre. Feeling like your body was itching on the inside? Strange rashes? Persistent air hunger? I couldn’t make heads or tails of most of the symptoms, let alone the underlying conditions. More than a few times, I turned to Google and Wikipedia to find ideas. Instead of helping, this mostly made things worse. My searches often spit out an array of bizarre Latinate conditions that sounded quite unpleasant. So much for my dreams of being a virtual Dr. House.
Still, I came up with a few ideas that seemed worth pursuing, so I added them to the list of potential diagnoses for each patient. Reviewing the other diagnoses for patients from my fellow Medical Detectives, as CrowdMed calls us, I nodded in agreement with a few and raised an eyebrow at others. Mostly, though, I had a hard time shaking the feeling that all of us were turning the quest for answers into the equivalent of pin the tail on the donkey: blindfolded and disoriented, the best I could hope for was to get the tail in the general vicinity of the donkey’s rump. Luck seemed just as important as skill.
When I confess this to Heyman, he says that this is actually part of the CrowdMed process. “Creating the diagnosis list is a lot like brainstorming. We want you to throw anything out there, even something that might otherwise be dismissed as crazy.”
“Creating the diagnosis list is a lot like brainstorming,” Heyman says.
Medical detectives can ask patients to clarify their symptoms through an online chat process. Over time, the list of potential diagnoses lengthens, ranging from just a handful to well over ten. Then, the diagnoses enter the betting process, in which medical detectives can bet virtual points on what they believe is the correct diagnosis. Unlikely diagnoses have longer odds but win more points if correct, which Heyman says encourages detectives to think outside the box. “It’s like playing the stock market,” Heyman says, where risky bets can be rewarding when they’re right.
Although unusual diagnoses might have the potential for a higher payoff, Heyman says that the betting system still encourages detectives to choose the diagnosis that is most likely to be correct. They may not win as many points, but selecting a popular diagnosis, if correct, still promises at least some points and improves a player’s standing in the CrowdMed community.
After betting is completed, CrowdMed uses a proprietary algorithm to evaluate the diagnoses and bets. Although Heyman won’t disclose specifics, he says that the system factors in how many points have been bet on the different diagnoses and how often each particular detective has been correct. This algorithm then sends the patient a list of the top three most likely diagnoses. Cases take several weeks to a month to solve—which might seem like a long time, but given that the average CrowdMed patient has been experiencing symptoms for six years, it tends to be much quicker than other rounds of diagnoses they received in the past.
“Medical specialists tend to see symptoms through their own particular lens, which doesn’t necessarily affect one of our medical detectives,” Heyman says. This means that they can get to the solution more quickly, he adds.
Patients on CrowdMed rate the site favorably, with more than half saying that they felt the site brought them closer to a solution for their condition. However, that subjective assessment doesn’t provide any information as to whether CrowdMed’s diagnoses are accurate. But, as Heyman points out, physicians don’t always know how accurate their diagnoses are, either.
Promises and Pitfalls
Physician Darshak Sanghavi, a pediatric cardiologist at the University of Massachusetts and a current fellow at the Brookings Institute in Washington, DC, finds the CrowdMed concept both promising and intriguing. “Crowdsourcing in medicine isn’t a completely foreign concept. It’s like a second opinion writ large,” he says. In the past, crowd-based wisdom, in the form of case consults, professional discussions, and medical journals was only available to doctors, Sanghavi said, but now patients can also access this wisdom.
There’s also a right time and a wrong time to crowdsource your medical problems. “If you think you might be having a heart attack, don’t ask CrowdMed what might be going on. This shouldn’t be used for medical problems that have the potential to be emergencies,” Sanghavi says.
CrowdMed wouldn’t be actually providing medical advice or services, Sanghavi points out. Instead, the website provides information for patients to take to their own physicians, an extremely common practice in medicine. Nearly all of his patients’ parents have used the Internet to find medical information about their child’s condition. Although sometimes the information is good, other times it isn’t. A recent study in the Journal of the American Osteopathic Association found that nine of the ten Wikipedia entries on the most common chronic diseases (such as diabetes, heart disease, and cancer) carry significant inaccuracies. Given that inexperienced medical detectives like me often turn to Wikipedia as an information source, this doesn’t always bode well for our guesses. If the diagnosis is wrong, says Marleen Eijkholt, a bioethicist at Albany Medical Center, physicians have to spend time un-educating the patient instead of working toward a correct diagnosis. Still, Heyman says, the system is set up to theoretically root out incorrect ideas.
A recent study found that nine of the ten Wikipedia entries on the most common chronic diseases carry significant inaccuracies.
Arthur Caplan isn’t convinced. The New York University bioethicist believes that the potential for serious medical mistakes is too high. “Crowds are great places to turn when you need to know where to find the best fries in town. Lots of people can become experts on fries. But everyone can’t be a medical expert,” Caplan says.
A misdiagnosis can cost time, by increasing how long it takes for patients to receive the right diagnosis, and money, by leading to unnecessary and expensive testing to rule out unlikely conditions. Unlike in the traditional medical system, patients have no recourse if something does occur. “The liability waiver you have to sign undercuts the legitimacy of what they’re doing. You don’t need the potential to hold someone liable when you’re buying pantyhose or ordering fries, but not in medical situations,” Caplan says.
Caplan is also concerned about the potential for the site to be taken over by “people with kooky ideas who think kale shakes are the solution to everything.”
Heyman isn’t surprised at the medical community’s resistance to CrowdMed. In fact, he fully expected it. “Medicine is a credential-based system: you need to graduate medical school, complete residencies and fellowships. CrowdMed is a performance-based system. If you are good at solving cases, we give your diagnoses more weight, even if you haven’t gone to medical school,” Heyman says.
Eijkholt has other concerns. Although the site uses pseudonyms, many of the conditions are somewhat rare, and Eijkholt doesn’t believe it would be all that difficult to identify a patient based on his or her symptoms if one knew what they were looking for. The site also has the potential to undermine the doctor-patient relationship.
“If I were a patient, I would take all of this information that was provided to me by the medical detectives and I would take it to the doctor, thinking that he is not an expert, that he hasn’t done his work, that he can’t be nearly as smart as the crowd,” Eijkholt says.
Despite her reservations, Eijkholt also admits that, were she struggling with a condition that nobody could diagnose, she might very well turn to CrowdMed. “As much as I worry about the possibility of false hope and false expectations,” Eijkholt says, “I also can’t say that if I was dealing with something that no one could diagnose, that I wouldn’t give CrowdMed a try.”
Answers at Last
The breakthrough for young Christian came not from a physician, but by research done by his mother. In her readings, she came across a rare condition called Pediatric Autoimmune Neuropsychiatric Disorders Associated with Streptococcal infections (PANDAS). The disorder is characterized by abrupt onset vocal and physical tics, as well as behavioral changes, such as emotional outbursts and obsessive-compulsive behaviors that follows an infection with the bacterium Streptococcus . Two weeks before Christian’s overnight behavioral change, he had been treated for a strep infection with the antibiotic amoxicillin.
PANDAS is extremely rare, and it’s even more uncommon for physicians to consider it when diagnosing a young person.
The antibodies the body creates to fight off the bacteria also attack an area of the brain known as the basal ganglia, which helps to control motor movements, habits, and motivation. Damage to the basal ganglia creates the symptoms of the disorder. Treatment generally consists of high-dose antibiotics and/or intravenous antibodies given every few months. If treatment is begun within 30 days of the onset of symptoms, Loop says, the disease can often be cured. If symptoms persist longer, however, treatment and management of PANDAS must continue long-term. PANDAS is extremely rare, and it’s even more uncommon for physicians to consider it when diagnosing a young person. When his parents suggested the diagnosis to Christian’s physicians, the doctors brushed them off.
Christian’s parents persisted, and six months after his personality vanished into the frigid night air, they finally got a diagnosis. They had to take him across the country to the National Institutes of Health and enroll him in a study to test intravenous antibodies to treat PANDAS. The first round of treatment was blinded, meaning that Christian, his family, and his physicians didn’t know whether he had received the antibodies or the placebo. His behavior didn’t change. Later, he received the intravenous antibodies in the second part of the study. Within a week, his behavior began to improve.
Several months later, Loop ran into one of Heyman’s friends, who mentioned CrowdMed. Loop got in touch with Heyman to propose Christian’s case as a test. Heyman was ecstatic and had the Loops write up Christian’s symptoms and experiences up to his initial diagnosis—leaving out their suspicions of PANDAS—and post it to CrowdMed. Overnight, which was nearly as quickly as Christian’s symptoms had appeared, more than 25% of CrowdMed’s medical detectives had suggested PANDAS as a diagnosis, putting it in the two most likely diagnoses for Christian.
Loop sees his son’s case as a powerful example of how CrowdMed could help other individuals. “We lost a tremendous amount of time looking for a diagnosis. If we got him treated sooner, he might not still be struggling with symptoms today. I wish the site would have been around when Christian first got sick,” Loop says.
Photo credits: rosmary/Flickr (CC BY) , CDC
Carrie ArnoldPosts By This Contributor