Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial open data platform and infrastructure that supports research on pragmatic trials. It shares clean trial data and ratings using PRECIS-2, which allows for multiple and varied meta-epidemiological research studies to evaluate the effect of treatment on trials with different levels of pragmatism and other design features.
Background
Pragmatic trials are increasingly recognized as providing real-world evidence for clinical decision-making. The term "pragmatic" however, is a word that is often used in contradiction and its definition and evaluation require clarification. The purpose of pragmatic trials is to guide clinical practices and policy choices, rather than prove a physiological or clinical hypothesis. A pragmatic trial should also strive to be as close to actual clinical practice as is possible, including its selection of participants, setting and design as well as the implementation of the intervention, determination and analysis of the outcomes, and primary analyses. This is a major distinction between explanatory trials, as described by Schwartz and Lellouch1 which are designed to confirm the hypothesis in a more thorough manner.
Truly pragmatic trials should not be blind participants or clinicians. This can result in a bias in the estimates of the effects of treatment. The trials that are pragmatic should also try to recruit patients from a wide range of health care settings to ensure that their findings can be compared to the real world.
Furthermore, pragmatic trials should focus on outcomes that are important for patients, such as quality of life or functional recovery. This is especially important in trials that require invasive procedures or have potentially serious adverse effects. The CRASH trial29 compared a 2-page report with an electronic monitoring system for patients in hospitals with chronic heart failure. The trial with a catheter, on the other hand utilized symptomatic catheter-related urinary tract infection as its primary outcome.
In addition to these aspects, pragmatic trials should minimize the trial's procedures and requirements for data collection to reduce costs. Furthermore pragmatic trials should try to make their findings as applicable to clinical practice as they can by ensuring that their primary analysis follows the intention-to treat approach (as described in CONSORT extensions for pragmatic trials).
Many RCTs that don't meet the requirements for pragmatism however, they have characteristics that are in opposition to pragmatism, have been published in journals of different types and incorrectly labeled as pragmatic. This can lead to misleading claims of pragmatism, and the term's use should be made more uniform. The creation of a PRECIS-2 tool that offers a standardized objective evaluation of pragmatic aspects is the first step.
Methods
In a pragmatic study the aim is to inform policy or clinical decisions by showing how an intervention can be integrated into routine treatment in real-world settings. This is distinct from explanation trials that test hypotheses regarding the cause-effect connection in idealized conditions. In this way, pragmatic trials may have a lower internal validity than explanatory studies and be more susceptible to biases in their design as well as analysis and conduct. Despite their limitations, pragmatic studies can provide valuable information for decision-making within the healthcare context.
The PRECIS-2 tool evaluates an RCT on 9 domains, with scores ranging from 1 to 5 (very pragmatist). In this study, the areas of recruitment, organization, flexibility in delivery, flexible adherence and follow-up received high scores. However, the principal outcome and the method for missing data was scored below the pragmatic limit. This suggests that a trial could be designed with good pragmatic features, without damaging the quality.
However, it's difficult to determine how practical a particular trial is, since pragmaticity is not a definite characteristic; certain aspects of a trial can be more pragmatic than others. A trial's pragmatism can be affected by changes to the protocol or logistics during the trial. In addition 36% of the 89 pragmatic trials discovered by Koppenaal and co. were placebo-controlled, or conducted prior to approval and a majority of them were single-center. Thus, they are not very close to usual practice and can only be called pragmatic in the event that their sponsors are supportive of the lack of blinding in such trials.
A common aspect of pragmatic research is that researchers try to make their findings more meaningful by studying subgroups of the trial sample. However, this often leads to unbalanced comparisons with a lower statistical power, thereby increasing the risk of either not detecting or incorrectly detecting differences in the primary outcome. This was the case in the meta-analysis of pragmatic trials due to the fact that secondary outcomes were not adjusted for differences in covariates at the baseline.
Furthermore practical trials can be a challenge in the collection and interpretation of safety data. This is because adverse events are generally reported by the participants themselves and prone to reporting delays, inaccuracies or coding deviations. It is therefore crucial to enhance the quality of outcomes ascertainment in these trials, ideally by using national registries instead of relying on participants to report adverse events in a trial's own database.
Results
While the definition of pragmatism may not require that all trials be 100 percent pragmatic, there are benefits to incorporating pragmatic components into clinical trials. These include:
Increased sensitivity to real-world issues which reduces study size and cost and allowing the study results to be more quickly implemented into clinical practice (by including routine patients). However, pragmatic trials may have disadvantages. For instance, the appropriate kind of heterogeneity can allow a study to generalize its results to many different patients and settings; however the wrong kind of heterogeneity could reduce assay sensitiveness and consequently lessen the ability of a trial to detect small treatment effects.
Several studies have attempted to classify pragmatic trials using a variety of definitions and scoring methods. 프라그마틱 데모 and Lellouch1 created a framework to discern between explanation-based studies that prove the physiological hypothesis or clinical hypothesis, and pragmatic studies that help inform the choice for appropriate therapies in the real-world clinical practice. The framework was comprised of nine domains evaluated on a scale of 1-5 with 1 being more explanatory while 5 was more pragmatic. The domains included recruitment setting, setting, intervention delivery with flexibility, follow-up and primary analysis.
The initial PRECIS tool3 featured similar domains and a scale of 1 to 5. Koppenaal et al10 created an adaptation of this assessment called the Pragmascope that was easier to use in systematic reviews. They found that pragmatic systematic reviews had a higher average scores across all domains but lower scores in the primary analysis domain.
The difference in the primary analysis domain can be explained by the way most pragmatic trials analyze data. Certain explanatory trials however, do not. The overall score for pragmatic systematic reviews was lower when the areas of organization, flexible delivery, and follow-up were merged.
It is important to note that a pragmatic trial doesn't necessarily mean a low quality trial, and in fact there is a growing number of clinical trials (as defined by MEDLINE search, however this is neither specific nor sensitive) which use the word 'pragmatic' in their abstract or title. The use of these terms in abstracts and titles may suggest a greater awareness of the importance of pragmatism but it isn't clear if this is manifested in the content of the articles.
Conclusions
In recent times, pragmatic trials are becoming more popular in research as the value of real world evidence is increasingly recognized. They are randomized trials that evaluate real-world treatment options with new treatments that are being developed. They involve patient populations that are more similar to those who receive treatment in regular medical care. This method could help overcome limitations of observational studies which include the biases associated with reliance on volunteers and limited accessibility and coding flexibility in national registries.
Other advantages of pragmatic trials include the ability to use existing data sources, as well as a higher probability of detecting significant changes than traditional trials. However, these tests could still have limitations which undermine their reliability and generalizability. For example, participation rates in some trials might be lower than expected due to the healthy-volunteer effect as well as incentives to pay or compete for participants from other research studies (e.g., industry trials). Many pragmatic trials are also restricted by the need to recruit participants quickly. Additionally certain pragmatic trials don't have controls to ensure that the observed differences are not due to biases in the conduct of trials.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs that self-labeled themselves as pragmatic and that were published up to 2022. They evaluated pragmatism using the PRECIS-2 tool, which includes the domains eligibility criteria as well as recruitment, flexibility in adherence to interventions and follow-up. They discovered that 14 of these trials scored pragmatic or highly practical (i.e. scores of 5 or more) in any one or more of these domains and that the majority of them were single-center.
Trials with a high pragmatism rating tend to have higher eligibility criteria than traditional RCTs that have specific criteria that are unlikely to be used in the clinical environment, and they comprise patients from a wide variety of hospitals. The authors argue that these characteristics could make pragmatic trials more meaningful and useful for daily practice, but they do not guarantee that a trial conducted in a pragmatic manner is completely free of bias. Moreover, the pragmatism of the trial is not a predetermined characteristic A pragmatic trial that does not possess all the characteristics of an explanatory trial can produce reliable and relevant results.