When the new Tufts study on the costs of R&D for development of a new drug is released Tuesday at 10AM, here are 10 things to look for:
1. What is the basis of the selection of drugs in the sample, and is it representative of all new drugs? For example, does it exclude small firms, in-licensed drugs, and under-sample orphan drugs? Note that since 2005, 57 percent of new cancer drugs were approved as Orphan Drugs, and 78 percent received at least one orphan designation. Note also that the UK based consultant, the Office of Health Economics, says that in-licensed drugs have better success rates.
2. What are the size of the trials? Do a few drugs with large costs skew the averages? How many drugs in the sample are “below average” in terms of trial size? One can look at the ratio of average to median to get one measure of the degree to which the data is skewed, in terms of departures from the mean.
If the data is not showing a significant skew, this is evidence the data is not representative of drug approvals in general, and if the data is skewed, it means the mean is not a good prediction for individual drugs, often because the average is influenced too much by outliers.
For some context, see: Elizabeth Rajasingh, KEI Research Note 2014:3, Size of Clinical Trials, data from the FDA 2010 NME and BLA approvals, preliminary results, November 17, 2014. In this research note, Rajasingh notes that in 2010, more than half of all the patients enrolled in trials were associated with just 2 of the 21 drugs, making the averages of almost no meaning to the rest of the drugs.
3. What are the per patient costs of the trials? Where does the data come from for this, and what if any of the costs are associated with overhead, rather than direct R&D outlays?
4. Does the study make assumptions about pre-clinical expenses? In the 2003 study, DiMasi estimated that risk adjusted pre-clinical R&D costs were always 42.9 percent of risk adjusted clinical expenses, and even more when capital costs were added. This is particularly inappropriate when looking at drugs where the NIH funded all or part of the pre-clinical research.
5. Does the study make the various elements of the costs transparent, and can one easily distinguish between outlays on a particular drug and estimates of the risk adjusted costs for that same drug? For example, if out-of-pocket costs on drug A were $100 million, and the risk adjusts were $200 million, is it obvious that the out-of-pocket cost were $100 million, and the risk adjusted numbers were $200? In DiMasi’s 2003 paper, he used the term out-of-pocket for both cases, which confused a lot of readers, and made the investments seem more risky. Are the costs of capital also clearly identified as a separate cost element?
6. How does DiMasi define a “new” drug? Does he calculate the R&D costs through the lead indication, or does the new drug keep adding up expenses as it is tested for new indications or formulations, while earning money from the first approved indication?
7. Note that since 2005, 57 percent of new cancer drugs were approved as Orphan Drugs, and 78 percent received at least one orphan designation. This status qualifies the drug for a 50 percent tax credit for its expenses on qualifying clinical trials, a huge subsidy when available. Did the study attempt to reduce net costs to reflect this subsidy?
8. Is it possible to verify the study estimates of time to market? Or, for that matter, make a list of what can be verified, and what cannot.
9. How does the data in the new DiMasi study relate to the costs of drug development for cancer drugs? Not only are new cancer drugs very likely to benefit from the Orphan Drug tax credit subsidy, but they also have much smaller trials, on average, than do other drugs. For example, in her study of 2010 new drug approvals, Rajasingh found that oncology drugs had trials that were only 29 percent of the average for all 2010 drug approvals. And in additional research on all new oncology drugs registered between 2005 and the present, the median size of the trials cited in medical reviews were just 1,125 — just 21 percent of the size of the average number of patients in the 2003 DiMasi study, and 21.5 percent of the average for all drugs in Rajasingh’s 2010 data.
Also, note that the NIH budget for the National Cancer Institute was $4.923 billion in FY 2014, a very large number compared to the number of new oncology drugs registered in a given year. How does this large NIH subsidy for research change the DiMasi estimates of the pre-clinical costs for R&D on new cancer drug?
10. When reporting on the estimate, do journalist make it clear that the estimate was prepared by researchers who consult for big drug companies, and the estimate will be used by those companies to lobby for higher prices?
For additional background context, see:
- Research note on oncology drugs, including trial size and orphan drug status. https://www.keionline.org/node/2125
- KEI Research Note: Elizabeth Rajasingh, KEI Research Note 2014:3, Size of Clinical Trials, data from the FDA 2010 NME and BLA approvals, preliminary results, November 17, 2014, https://www.keionline.org/node/2124
*missing only Phase 1 trials from one of twenty one drugs.