The quality of randomized controlled trials in pediatric orthopaedics: Are we improving?

Dodwell E, Dua S, Dulai SK, Astone K, Mulpuri K. The quality of randomized controlled trials in pediatric orthopaedics: Are we improving? J Pediatr Orthop. July 2015. DOI:10.1097/BPO.0000000000000324.

Abstract

Purpose:

The quality of randomized controlled trials (RCTs) in orthopaedics is a topic of considerable importance, as RCTs play a major role in guiding clinical practice. The quality of RCTs published between 1995 and 2005 has previously been documented. The purpose of the current study was to assess and describe the quality of pediatric orthopaedic RCTs published from 2005 to 2012, by identifying study characteristics associated with higher quality and outlining areas for improvement.

Methods:

A standardized literature search was used to identify pediatric orthopaedic RCTs published in 7 well-recognized journals between September 2005 and July 2012 inclusive. The Detsky Quality Assessment Scale and the CONSORT checklist for Non-Pharmacologic Trials were used to assess the quality of the RCTs. Scores for the Detsky and CONSORT were calculated by 2 independent blinded orthopaedic surgeon reviewers with epidemiologic training.

Results:

Forty RCTs were included in this analysis. The mean percentage score on the Detsky quality scale was 67%. Sixteen (40%) of the articles satisfied the threshold for a satisfactory level of methodological quality (Detsky >75%). Twenty-five (63%) of these studies were negative studies, concluding no difference between treatment arms. In 52% of the negative studies, an a priori sample size analysis was absent, and 28% were self-described as underpowered. In multiple variable regression analysis, only working with a statistician was significantly associated with higher Detsky percentage scores (P=0.01).

Conclusions:

There is a trend for improving quality in pediatric orthopaedic RCTs. Compared with past reports, the mean Detsky score improved from 53% to 67%, and the proportion meeting an acceptable level of quality improved from 19% to 40%. One of the most concerning findings of this study was the lack of attention to sample size and power analysis, and the potential for underpowered studies. Ongoing efforts are necessary to improve the conduct and reporting of clinical trials in pediatric orthopaedics.

Significance:

Pediatric orthopaedic surgeons, JPO, and POSNA are working toward improving levels of quality in pediatric orthopaedic research. This paper highlights progress that has been made, and addresses some high-yield areas for future improvement.