This article was written by guest authors Paul C. McAfee, M.D., MBA, Chief of Spinal Surgery University of Maryland St Joseph Medical Center; Jordan McAfee, a recent Syracuse University graduate, who is volunteering at NYU Hospital for Joint Disease and is in his final year of Post-Bac Medical studies at Fordham University Lincoln Center; and John P. (JP) McAfee, a cum laude graduate with honors from the University of Pennsylvania and a magna cum laude law school graduate from the University of Baltimore School of Law, was editor for the University of Baltimore Law Review, a Steven L. Snyder Litigation Fellow, and a member of the Heuisler Honor Society. Mr. McAfee clerked for the Honorable James R. Eyler in the Maryland Court of Special Appeals.
__________________
Last week we described how social media (Facebook, Twitter, Instagram, Google Plus or Stumbleupon, for example) had become a new, powerful force for uncovering pharmaceutical or medical device adverse event information; including how some users look for how to Download Instagram videos.
The FDA, which had been slow off the mark in past instances, has been making significant progress and clearly embraces social media as an information source. Peer review journals, however, have been strangely and troublingly silent.
This week, in Part II, we take a deep dive into the pros and cons of such established adverse event databases as MAUDE and social media as a source of medical device information.
Bias and Conflicts in Social Media
Operating in their own self-interests, plaintiff’s attorney groups have been able to harness the power of social medial to circulate and publicize the negative results of post market approved devices like the DePuy ASR from the UK and Australian registries and to recruit litigants (Figure 2).
While the raw information in social media is useful, it is also subject to the same limitations that Sedrakyan et al. found when they examined the professional voluntary reporting systems such as the MedWatch program, the MAUDE (Manufacturer and User Facility Device Experience), and the MedSun (Medical Product Safety Network).
“These reporting systems have important weaknesses, such as incomplete, inaccurate, or nonvalidated data, reporting biases related to event severity, concerns that reporting may result in adverse publicity or litigation, and general underreporting of events. Most importantly, denominator data are missing, which makes evaluation of the incidence or prevalence of a safety-related event impossible.”
The FDA, which has made a conscientious effort to incorporate social media and first warning signs into their database, also acknowledges the limitations of these formal, professional reporting processes and agrees that it is suboptimal, stating:
“Although MDRs [medical device reports] are a valuable source of information, this passive surveillance system has limitations, including the potential submission of incomplete, inaccurate, untimely, unverified, or biased data. In addition, the incidence or prevalence of an event cannot be determined from this reporting system alone due to potential under-reporting of events and lack of information about frequency of device use. Because of this, MDRs comprise only one of the FDA’s several important post market surveillance data sources.”
A Robot Shows MAUDE’s Limitations
The best documented illustration of the limitations of MAUDE concerns the da Vinci robotic procedures.
Intuitive Surgical, Inc., the developer and manufacturer of the da Vinci system, reported that its system has been used in as many as 150, 000 procedures annually (Figure 3) and has increasingly been selected by U.S. hospitals as an important surgical tool over the past decade.
Martin A Makary and colleagues from Johns Hopkins conducted a study of da Vinci’s post market performance. In their study, the investigators cross-matched the reports of legal judgments on LEXIS-NEXIS with court cases which found device failure with da Vinci robotic procedures. One area where the investigators found a comparatively high number of successful plaintiff rulings was regarding “damage to viscera” which had an insulation problem resulting in inadvertent bowel cauterization and injury.
Makary and his fellow researchers found eight da Vinci cases which qualified for self-reporting under the FDA’s guidelines but which did not in fact appear in the MAUDE database. Here is a direct quote from Makary’s study:
“Our search found eight cases (3% of reported cases) where incidents were not appropriately filed with the FDA. In five of these cases, no FDA report was ever filed. In one case, the FDA report was filed 1 year after the patient’s death, 2 weeks after a Wall Street Journal article ran citing the case (Carreyrou, 2010). The report includes an event date that matches the month and day of the patient’s death, but gives as the year 2010 rather than 2009. It is difficult to know if the incorrect event date was a mistake or a deliberate change following a nearly year-long delay in reporting the death. In another case, despite an injury being reported to an Intuitive representative, the Intuitive supervisor failed to file the FDA report. Our findings demonstrate that FDA reporting was not prompt or standardized.”
Gaming the MAUDE System
Ninety percent of MAUDE’s listings are self-reported, self-policing statements filed by manufacturers—the remaining 10% are filed by hospitals or healthcare providers. Patients who feel they are the subject of a medical device failure may file information on the FDA’s MedWatch site.
When we (the authors) analyzed the MAUDE database we found a positive correlation between the size of the company, as defined by annual sales in 2013, and the number of reports on MAUDE. The larger the company, logic holds, the more likely the number of filings for medical device failures should be higher—and we found that to be generally true.
There was one surprising exception to this rule, however, and it was Medtronic, Inc. According to MAUDE, Medtronic (annual sales in excess of $16 billion) had fewer self-reported medical device failures than K2M, Inc. (annual sales $163 million).
So to investigate this further the authors searched the MAUDE database for a specific Medtronic device recall—the September 11, 2008 recall of the “Medtronic Sofamor Danek CD Horizon Spinal System AGILE Dynamic Spinal Stabilization Device.”
In that specific instance, the FDA listed the recalling manufacturer as “Medtronic Sofamor Danek USA, Inc.” Searching the MAUDE database from June 1, 2005 to June 1, 2014 using the brand name “AGILE” uncovered some interesting results. Specifically, of the 51 AGILE complaints listed on MAUDE, 1 was from “Danek” (8/27/10); 4 were from “Medtronic” (9/8/08, 10/2/08, 3/19/09, and 9/30/10) but the overwhelming number (46) came under the name “Warsaw Orthopedics, Inc.”
We searched the 2013 Medtronic Annual Report and could not find a reference to Warsaw Orthopedics listed as one of the 10 business units of Medtronic nor as an intangible asset, or listed under legal proceedings.
The MAUDE database is flawed due to underreporting. But, as the Medtronic example indicates, it is also vulnerable to fragmented reporting information with some companies listing subsidiaries (who may or may not have overhead, or salesforces, or inventory) as manufacturers of record for implant related adverse events. One interpretation is that companies are gaming the system in order to protect the parent company’s name.
The FDA Embraces Social Media
On June 17, 2014 Thomas Abrams, director of the FDA’s Office of Prescription Drug Promotion in the Agency’s Center for Drug evaluation and Research (CDER) introduced three new guidance documents regarding social media:
“Our first guidance provides recommendations for the presentation of risk and benefit information for prescription drugs or medical devices using Internet/social media sources with character space limitations, such as Twitter and the paid search results links on Google and Yahoo. These recommendations address the presentation of both benefit information and risk information in this setting—http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM401087.pdf
“Our second guidance provides recommendations to companies that choose to correct third-party information related to their own prescription drugs and medical devices. This draft guidance provides FDA’s recommendations on the correction of misinformation from independent third parties on the Internet and through social media sites—http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM401079.pdf
FDA sees social media as an important resource for industry and is committed to developing additional guidance for drug and device manufacturers that outline the agency’s current thinking. We do all of this work with the best interest of patients in mind—http://www.fda.gov/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDER/ucm397791.htm
What to Do Now
Physicians can help detect early widespread medical device failures by checking websites like MedWatcher when they are presented with a device failure. If, upon checking, the physician finds reports similar to their own experience, they can help expedite the process of publishing peer-reviewed literature covering a medical device failure.
Patients are taking to social media to report their complications because the alternatives are not optimal. MedWatch is one option, but even after filling out the form patients may well feel as if their voices are unheard.
Furthermore, valuable patient data becomes scattered throughout the Internet because when a patient does a web search for issues related to their medical problem, they find Facebook groups, online petitions and web forums. In the midst of such a fragmented environment, the task of searching and filtering this patient data becomes overwhelming.
Is there a role for the FDA to create and monitor web forums for patients who wish to post and discuss their pharmaceutical or device problems? Could the FDA, for example, develop search algorithms that monitor Facebook, Twitter, web forums and other sites?
Large hospital, social media, and FDA databases can be searched for patient symptom terms such as “dizziness, ” “blurred vision, ” “shakiness” and “hypoglycemia.” These terms could be hypothetically cross-referenced with “emergency hospitalization” and one might find a five- or six-fold increase in reports in September 2007, for example. This might also correlate with a particular manufacturer’s serum glucose monitor malfunctioning and relaying artifactually high readings to patients.
The surveillance of big data analytics would be the first societal warning sign for a root cause analysis and focus the providers on the specific serum glucose monitor—in turn, the patients utilizing this brand of medical equipment would be emergently contacted, hopefully averting more hypoglycemic crises.
We advocate a balanced approach—a first phase data analytics approach for speed and rapid recognition of medical device problems followed by a second phase conscientious peer-reviewed Evidence-based medicine follow up.
With speed comes imprecision. There is excessive noise in most mega databases. Proper data mining requires an astute programmer who can dissect out terms that are close to symptoms but are not indicative of medical device failure.
Return to Evidence-Based Medicine?
Evidence-based medicine is a more refined and accurate approach than brute force data mining.
In four social media papers on vaginal mesh presented at the American Urologic Society Meeting in 2013 the authors demonstrated a highly progressive approach towards incorporating social media.
Alas et al. analyzed the content of Facebook, Twitter, and YouTube for key words “urogynecology, ” “pelvic organ prolapse, ” “stress incontinence, ” “urge incontinence” and “incontinence” and related them to vaginal mesh adverse events.
Searching over a 13-month period the authors were able to show a stable amount of useful information and an increase in the number of health professionals providing content on the social media sites. However, of the 817 search results, only 406 (50%) were deemed useful. Only 28% of all the social media comments were written by health professionals, but of the informative results, 56% were written by health professionals.
The authors of these studies also found that a majority of patients who’d monitored social medial sites misinterpreted the FDA Advisory regarding transvaginal mesh issued on July 12, 2011. The patients thought that the FDA had recalled vaginal mesh, which they had not.
In conclusion, the need for more physician input on social media sites is clear. In addition, we (the authors) advocate for application of data analytics to uncover early warning signs from social media sites of potentially serious medical device failures.
ILLUSTRATIONS
Figure 3. The annual number of Robot-Assisted minimally invasive surgical (MIS) procedures both in the US (dark bar) and internationally (grey bar) was dramatically increasing from 2004 through 2011 partly due to marketing by Intuitive Surgical’s da Vinci system.
