Donghan (Don) Luo, PhD, Associate Director, Biostatistics, Janssen Research and Development, USA*
Interviewed by Eric Y. Wong, PhD, MBA, ISMPP CMPP™, Janssen Scientific Affairs, LLC, and with contributions from Robert Matheis, PhD, MA, ISMPP CMPP™, Celgene Corporation.
* The opinions expressed within are the author’s own and do not necessarily reflect the views of the author’s employer.
“What I Would Like You to Know” is an article series that shares perspectives and insights from functional area colleagues that collaborate with medical publication professionals on the planning and development of scientific publications. This new article series will appear periodically in The MAP.
This article spotlights a biostatistician’s perspective, presented in a question-and-answer format.
What are one or two things you want your medical publication colleagues to know from your experience working on clinical research publications?
There is a common misconception about probability and the use of the p-value in inferential statistics that can lead to misinterpretation of results in medical publications. Indeed, the p-value is one of the most misunderstood statistics yet is often regarded as the “gold standard” to evaluate the validity of a clinical finding. In addition to misinterpretation, p-values are sometimes misused, particularly when comparing multiple outcomes in one study. My colleagues often do not realize that one cannot look at p-values in isolation; they are part of a family of comparisons. Thus, if you produce five comparisons (p-values) in a manuscript, the probability of error assuming a 5% (0.05) threshold is actually (5 x 0.05) or 25%! All too often, papers are written without thoughtfully accounting for the actual level of statistical significance achieved. The solution is to report only p-values from the primary hypotheses and other pre-specified analyses. Avoid exploratory analyses whenever possible and, if they are to be reported, make sure the publication properly indicates the exploratory nature of the comparison.
Another aspect that my colleagues in medical publications should know is the importance of properly discussing sample size in a publication. Sample size is always a big question at the start of a clinical study and is closely associated with study design and objective. In short, a study must have enough participants to properly test the hypotheses (ie, a statistical power analysis), and a particularly large sample is required when there are multiple arms and/or comparisons. In general, inferential statistical testing should be avoided on small samples. On the other hand, excessively large study samples can lead to false positive results, due to chance alone. Thus, it is advised that publications provide ample discussion of power analysis and sample size so that journal reviewers and readers can make informed decisions about the findings.
What is one improvement you would suggest to the development of clinical research publications?
It would be helpful to generate and report data in a complete and standardized manner, so that readers or researchers can extract or interpret information consistently. For example:
- Demographics summaries should include the number of subjects with means (standard deviations) or medians (ranges), as appropriate.
- Proportion of efficacy or safety events should not just report a percentage but should also include the number of subjects and events.
What is your biggest challenge when contributing to the development of clinical research publications?
Making sure the results are not overstating the implications or conclusions from clinical trials. Each study has its own objectives to answer specific research questions. The publication should reflect the scope of the clinical study and should faithfully report the findings from the study, either positive or negative, and it should clearly describe the limitations.
My key recommendation and advice to colleagues working in medical publications is to avoid drafting publications that overemphasize inferential statistics. Instead, use statistics judiciously and always with full context and explanation. As with all things related to publishing, transparency and full disclosure are critical.