Demographic diversity of participants in clinical trials conducted in Singapore

ABSTRACT

Introduction: The under-representativeness of participants in clinical trials limits the generalisability of results. This review evaluates the representativeness within pharmaceutical randomised controlled trials (RCTs) in Singapore.

Method: Four bibliographic databases were searched for papers on pharmaceutical RCTs which included Singapore adults (≥18 years old), published between 2017 and 2022. The demographic characteristics of study participants were compared against the population in the 2020 Singapore census. Recruitment strategies and authors’ comments on the generalisability of their findings were reviewed.

Results: Thirty-three publications were included (19 Singapore-only studies and 14 multiregional trials which included Singapore). Where data were available, we found that females and Indians were under-represented compared to the census (41.3% versus [vs] 51.1%, P<0.05; 7.3% vs 9.0%, P<0.05). Ethnic diversity varied between individual studies, and less than half (46.2%) of Singapore-only studies achieved census levels. However, more than one-third of the trials provided no data (31.6%) or partial data (5.3%) on ethnicity. Half of the multiregional publications stated the number of participants recruited from Singapore, but only 1 reported any detail beyond Asian participants. Recruitment strategies were mentioned in fewer than half (42.4%), and almost a quarter (24.2%) commented on sample representativeness or the external validity of the evidence generated.

Conclusion: There is room for improvement regarding the recruitment of RCT participants in Singapore, with particular attention to female gender and Indian ethnicity. Demographic data should also be presented in full. RCTs should be designed and reported such that clinicians can ascertain the generalisability to the Singapore population and the potential benefits from the studied interventions in clinical practice.

CLINICAL IMPACT

What is New

• This is the first review examining the representativeness of participants in pharmaceutical randomised controlled trials (RCTs) conducted in Singapore.
• Our analysis revealed significant under-representation of key demographics, including females and Indians, compared to the national census, highlighting many areas for improvement in RCT data reporting.

Clinical Implications

• The study underscores the urgent need for investigators to recruit a demographically diverse patient population that accurately reflects Singapore’s population.
• We urge authors to ensure comprehensive data reporting to enhance the generalisability and relevance of trial findings for a wider audience.

Randomised controlled trials (RCTs) and systematic reviews are the most reliable ways of relaying information on the most effective treatments for our patients in clinical practice. The careful design of trials to minimise the possibility of bias ensures that findings are internally valid, but this may threaten their clinical utility, as the results are not relevant to all the patients encountered in clinical practice—a concept known as external validity, applicability or generalisability of evidence. Over 50 years ago, Archie Cochrane commented that “Between measurements based on RCTs and benefit … in the community, there is a gulf which has been much underestimated.”¹ Today, external validity remains a frequent concern for clinicians when considering the results of RCT or systematic reviews, and the relevance of guidelines.

The result from a trial or the average result from pooled trials in a systematic review  may not necessarily be applicable to all because the trial recruitment criteria distort representatives; for example, many RCTs exclude the elderly and occasionally females;² similarly they may not be attentive to ethnicity which can also influence the efficacy and safety of pharmaceutical interventions.³ An analysis of 213 Pfizer-funded, interventional clinical trials, between 2011 and 2020 and involving 103,103 participants in the US, found that the involvement of minority ethnic groups was not uniform across trials, and they were often not represented at all.⁴ More than half of the trials achieved population census levels for African–American, white and Hispanic/Latino participants, while Asian, Native Hawaiian or other Pacific Islander and American Indian or Alaska Native participants were commonly under-represented, achieving only census levels in 16%, 14.2% and 8.5% of trials, respectively.

Exclusion criteria may also prevent participation in trials of patients with comorbidities or multimorbidity, or those with more severe disease or illness of longer duration. A review of cardiology, mental health and oncology trials compared the baseline characteristics (i.e. demographic, socioeconomic and clinical) of trial participants with patients encountered in clinical practice.⁵ It demonstrated that in 37 of 52 studies (71.2%), the population studied was not representative of real-world patients and thus limited the external validity of the RCT. The authors recommended that more appropriate evidence could be generated by tailoring trial design—including the recruitment of more representative patient samples—and carefully taking into account trade-offs made between internal and external validity.

There is a growing literature highlighting how the under-representativeness of research participants in clinical trials may misinform clinical practice, and their inclusion in systematic reviews and meta-analyses may subsequently misinform health policies.^2,6 Singapore is a multiethnic nation, with 4 ethnic categories (i.e. Chinese, Malays, Indians and Others). In this study, we sought to review the representativeness of participants in pharmaceutical RCTs in Singapore. To the best of our knowledge, no such study had been conducted.

METHOD

Literature search

Searches were run in PubMed, Embase, Cochrane Library and Web of Science to identify pharmaceutical RCTs involving the Singapore population over 5 years from 1 January 2017 to 12 August 2022. The search strategy was developed in consultation with a medical librarian to ensure that the search strategies (Appendix S1) were optimal. All searches were run on 12 August 2022. Deduplication was initially performed using EndNote and then 2 authors (JB and AC) independently screened titles and abstracts on Covidence, manually removing duplicates and ineligible publications according to the inclusion and exclusion criteria listed in Table 1 (EndNote 21, Clarivate PLC, London, UK; Covidence software, Veritas Health Innovation, Melbourne, Australia). Potentially eligible full-text publications were retrieved, screened once more, with senior reviewers (LES or HES) resolving any disagreements on study selection.

Table 1. Inclusion and exclusion criteria.

Data extraction

A data extraction table was developed and tested on a sample of studies before being finalised. The data extracted from each publication included patient population and country of study, baseline demographic characteristics (i.e. ethnicity, sex and age), disease/therapy area (determined by study aims, authors’ disciplines and journal) (Appendix S2). Recruitment strategies, inclusion and exclusion criteria, recruitment challenges and authors’ comments on the generalisability of their studies were also extracted verbatim for inductive content analysis.⁷

Data analysis and interpretation

Descriptive statistics were used to summarise the demographic characteristics (i.e. sex, age and ethnicity) of trial participants. The study population demographics were compared with the 2020 Singapore Census of Population,⁸ employing similar methodology used to assess demographic diversity in the US,⁴ but adapted for Singapore and focusing on Chinese, Malay, Indian and Other ethnicities. When variables were reported only for the treatment or the control population, the overall sample population was calculated to enable comparison with the census data. Comparison between the census population and the study population were made using the 2 proportion Z-tests.⁹ When a specific trial is cited in the text, the first author’s name follows in a bracket and in italics.

RESULTS

From the 932 potentially relevant articles retrieved from the databases searched, 33 publications had a Singapore footprint and were eligible for inclusion (Fig. 1). There were 19 Singapore-only-based trials and 14 multiregional studies, of which the US (n=7) was the most frequent  collaborator (Fig. 2). Studies involving 5 regions were the most common, with a range from 2 (US and Singapore) (Szmulewittz et al.) to 17 (Latin America [2 countries], Europe [12 countries] and Others [3 countries: Singapore, Israel and Singapore]) (Reference 19 in Appendix S3). The data presented were often incomplete. In 24 publications (72.7%), demographic characteristics were described for treatment and control arms, but not reported in similar detail for the overall study population.

Fig. 1. PRISMA flow diagram for the selection of manuscripts to explore the demographic diversity of participants in pharmaceutical trials in Singapore.

PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Fig. 2. Location of other centres which were part of the multiregional clinical trials that Singapore participated in.

Fields of study

Infectious disease trials formed the greatest proportion of publications (9 trials, 7405 participants), with 5 relating to the management/treatment of COVID-19. Oncology trials (5 trials, 1242 participants) were the other major focus. The other fields of study were gastroenterology/hepatology (n=4 trials), cardiology (n=3), endocrinology (n=3), anaesthesia (n=2), dermatology (n=2), critical care (n=1), neurology (n=1), ophthalmology (n=1), psychiatry (n=1) and urology (n=1).

Multiregional publications

Seven of the 14 multiregional publications did not state the number of patients recruited from Singapore. When the number of participants recruited from Singapore was listed, the percentage they contributed to the study ranged from <1.0% to 59.8%.

Age

The reporting of participants’ age varied in format. Data were most commonly presented as mean/standard deviation, but others used mean/range, median/interquartile range and median/range. Unfortunately, due to the lack of country-specific data and the reporting of age in different ways, we were unable to conduct comparative analysis with Singaporean census data. Fourteen trials explicitly excluded older individuals, with upper age limits ranging from 45 to 85 years. There were 4 trials that excluded patients with common comorbidities (e.g. diabetes, hypertension) which were unlikely to be directly related to the intervention. Such exclusion criteria are more likely to apply to older people, as the prevalence of chronic disease and comorbidity increases with age. In Singapore, the mean number of chronic diseases among 45–64 year-olds is 1.68, and among those aged 65–84 years is 3.92.¹⁰ Thus, even if the inclusion criteria seem older people-friendly, there is a possibility that they will not be eligible because of a co-existing health problem.

Ethnicity

Thirteen of the 19 Singapore-only publications reported ethnicity in detail (Chinese/Malay/Indian/Others). In 1 publication, there was partial information, with only the number of Chinese and Malay participants reported, but nothing about the remaining participants. In 5 publications, there was no detail whatsoever. Only 1 of the multiregional studies (Reference 2 in Appendix S3) reported ethnic distribution in detail (Chinese/Malay/Indian/Others), when others did comment on ethnicity, they tended to use the broad descriptor of “Asian.” When exploring the representativeness of ethnicity, 1 Singapore-based study (Reference 21 in Appendix S3) was excluded as it did not involve citizens or permanent residents, recruiting only temporary migrant workers from India and Bangladesh.

In the 13 studies where clinical trial population representativeness was compared with the real-world Singaporean population, we found that the proportion of the recruited population who were of Chinese and “Other” ethnicities was similar to the census, but there was an under-representation of Indian participants (7.3% vs 9.0%, P=0.0217) and an over-representation of Malay participants (15.7% vs 13.5%, P=0.0124). Less than half (6 of 13; 46.2%) of the trials achieved census levels of ethnicity (Fig. 3).

Fig. 3. Proportion of participants in RCTs by race.

Sex

The analysis of sex focused on Singapore-only trials, as there was insufficient Singapore-specific gender demographic data included in any multiregional studies. The analysis was relevant to 16 of the 19 studies, as 3 were single-sex, 2 exclusively male (focusing on erectile dysfunction [Reference 12 in Appendix S3] and COVID prophylaxis in migrant workers [Reference 21 in Appendix S3]), and 1 (Reference 24 in Appendix S3) was a female-only study that recruited women undergoing a Caesarean section. In Singapore-only trials (n=16) recruiting males and females, females were under-represented in comparison to the census (41.3% vs 51.1%, P<0.00001) (Fig. 4). Females were under-represented significantly (P<0.05) in almost two-thirds of studies (62.5%); these were in the fields of infectious diseases, cardiology, endocrinology and psychiatry.

Fig. 4. Proportion of participants in Singapore-only RCTs by sex compared to Singapore census.

RCT: randomised controlled trial

Recruitment strategies, recruitment challenges and generalisability of results

Fourteen of the 33 publications reviewed (42.4%) included some details of the recruitment strategies used. This was scanty but included information such as the number of recruitment centres, the status of patients (inpatient/outpatient) or mode of recruitment (e.g. advertisement, personal approach from study team). Fewer publications (24.2%) made comments on the applicability or generalisability of the study’s findings, facilitating readers’ assessment of external validity and real-world applicability.

DISCUSSION

This review explored the representativeness of participants in pharmaceutical RCTs in Singapore over 5 years. This external validity analysis included 19 Singapore-only and 14 multiregional studies addressing studies in 12 clinical specialties.

Demographic diversity of participants in RCTs conducted in Singapore

When the data were collated, we found significant under-representation of women in 56.3% of trials in comparison to the 2020 Singapore census. Studies where women were under-represented were in the fields of infectious diseases, cardiology, endocrinology and psychiatry. Similar under-representation of women in these clinical areas has been reported by studies that analysed sample populations recruited in other countries, including the US.^11-13 Further research is needed to ascertain specific barriers to recruiting female participants, and if there are particular challenges that are specialty-specific.

Our analysis of ethnicity found significant under-representation of Indians and over-representation of Malays compared to the Singapore census; the number of Chinese and “Others” was similar to the census. Surprisingly, the absence of reporting of ethnic diversity was found in almost a third (31.6%) of Singapore-only publications, when Singapore commonly boasts its ethnic diversity in research. Detailed reporting of ethnic distribution in a multiethnic country, such as Singapore, is essential to enable the clinician or systematic reviewer to assess the external validity of the study’s findings. One must wonder whether among those trials failing to publish data on ethnicity, there was a preponderance of over- and under-representation and an unwillingness to share. Many other countries have been challenged with improving the external validity of their trials. For example, a retrospective study showed that no data on race/ethnicity were reported by two-thirds of the clinical trials that informed the 2007 cardiovascular disease prevention guidelines of the American Heart Association.¹⁴ This study was able to examine trends over 3 decades and found that the percentage including ethnic data has increased significantly, from 12.5% in the 1970s to 46.2% in the 2000s. The best improvement in reporting race/ethnicity as a baseline characteristic has been seen in trials funded by the National Institutes of Health. This illustrates how funding agencies can influence transparency and enable the consideration of external validity.

Multiregional clinical trials (MRCTs)

The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use promotes worldwide development and approval of safe, effective and high-quality medicine in the most resource-efficient manner. It recommends that the regional allocation in MRCTs “should have a scientific basis (rather than arbitrary targets), should support the evaluation of consistency and should provide the information needed to support regulatory decisions,” emphasising that no single region or regions should dominate enrolment, and therefore dominating the trial outcome.¹⁵ Unfortunately, in our analysis, only half of multiregional publications reported the number of participants recruited from Singapore. Without precise reporting of sample size in different regions, it is impossible to assess the external validity and applicability to the country of interest.

The MRCTs reviewed did not only lack data on the number of participants from each country but also often had inadequate or incomplete data on ethnicity. When ethnicity was given, the broad descriptor “Asian” was used, and only 1 study reported the number of Chinese, Malay, Indian and Others recruited. The use of the term “Asian” suggests homogeneity and fails to recognise the heterogeneity of population within Asia, whose physical and biological differences influence their response to medication even for commonly used drugs.¹⁶ For example, in Singapore, the mean weight-normalised maintenance warfarin dose in Indians is almost twice that for Singaporean Chinese and Malays, due in part to differences in the gene-encoding Vitamin K epoxide reductase complex 1.¹⁷ The Asian Diversity Project and Pan-Asian Single nucleotide polymorphism study have demonstrated the extent of heterogeneity between subgroups of Asians, concluding that the use of single-label “Asian” undermines genetic diversity within Asia, but at the same time could overstate its influence on differences in drug disposition and pharmacodynamics.¹⁸ The awareness of inter-ethnic differences in pharmacokinetics and pharmacodynamics is a growing research field. The reporting of MRCTs needs to support the completeness and standardisation of participants’ ethnicity.

Recruitment strategies, selection criteria and generalisability

In 2013, the Standard Protocol Items: Recommendations for Interventional Trials statement in 2013 recommended protocol descriptions of (1) where recruitment takes place, (2) who is the individual recruiting participants, (3) when participants are recruited and (4) how potential participants are informed of the trial.¹⁹ Transparency in each stage of recruitment is key to the assessment of bias and generalisability by reviewers.^20,21 Only 14 included publications (42.4%) described recruitment strategies beyond detail on the location of recruitment. Further research is needed to identify potential barriers to publishing recruitment strategies. Two possible barriers include the perceived lack of importance by authors and word limit imposed by journals.

Study strengths and weaknesses

This is likely the first study on the demographic diversity of participants in clinical trials conducted in Singapore. We used a robust methodological approach, but our review was limited to adults and to pharmaceutical trials. Thus, the findings may be different for other healthcare interventions (e.g. psychological interventions, service delivery evaluations) or for trials involving children. The number of trials identified was relatively small, but boosting the number of included trials would have required increasing the search to earlier years, which was not desirable because we wanted our review to reflect contemporary, not historical, clinical trial practice.

We recognise that the COVID-19 pandemic disrupted in-person recruitment to trials, in-person data collection and follow-up.^22,23 Thus, when repeating this review, it would be valuable to look for trends over time and to examine whether the pandemic threatened the representativeness of trials.

Similar to studies conducted in other countries, we compared the demographic characteristics of patients recruited in these trials with population census data, but it would be preferable to make this comparison with the epidemiology (i.e. age, sex, race) of the specific disease under investigation. Such comparison would allow more meaningful detection of the disparity between participants recruited in RCTs and patients encountered in clinical practice.

CONCLUSION

By understanding current study populations, future clinical trials can be designed to improve the generalisability of findings, and evidence-based care can be better provided to our patients. There is some room for improvement regarding the recruitment of RCT participants in Singapore, with particular attention to female gender and Indian ethnicity. Attention must also be given to presenting demographic data in full. Even though Singapore prides itself on being multicultural, more than a third of publications did not describe the ethnic diversity of the patients studied. The reasons for incomplete data presentation are many, perhaps oversight or intentional (e.g. to detract from potential poor external validity) or constrained by medical journal word counts. Moving forward, we need to ensure that studies are designed and reported in such a way that allows the clinician to decide on the generalisability to the Singapore population and the benefit that will be derived from treatment in clinical practice.

Supplementary Materials

Appendix S1. Search strategies.
Appendix S2. Data extraction form.
Appendix S3. References of studies included.

Declaration

The authors declare there are no affiliations with or involvement in any organisation or entity with any financial interest in the subject matter or materials discussed in this manuscript.

Correspondence: Assistant Professor Lee Eng Sing, Director, Primary Care and Family Medicine Research Programme, Lee Kong Chian School of Medicine, 11 Mandalay Road, Singapore 308207. Email: [email protected]

REFERENCES

1. Cochrane AL. Effectiveness and efficiency: random reflections on Health Services. London: Nuffield Provincial Hospitals Trust; 1972.
2. Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?” Lancet 2005;365:82-93.
3. Clark LT, Watkins L, Piña IL, et al. Increasing Diversity in Clinical Trials: Overcoming Critical Barriers. Curr Probl in Cardiol 2019;44:148-72.
4. Rottas M, Thadeio P, Simons R, et al. Demographic diversity of participants in Pfizer sponsored clinical trials in the United States. Contemp Clin Trials 2021;106:106421.
5. Kennedy-Martin T, Curtis S, Faries D, et al. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials 2015;16:495.
6. Tan YY, Papez V, Chang WH, et al. Comparing clinical trial population representativeness to real-world populations: an external validity analysis encompassing 43 895 trials and 5 685 738 individuals across 989 unique drugs and 286 conditions in England. Lancet Healthy Longev 2022;3:e674-e689.
7. Vears DF, Gillam L. Inductive content analysis: A guide for beginning qualitative researchers. Focus on Health Professional Education: A Multi-Professional Journal 2022;23:111-27.
8. Department of Statistics Singapore. Singapore Census of Population 2020 Statistical Release 1 – Key Findings, 2020. https://www.singstat.gov.sg/-/media/files/publications/cop2020/sr1/findings.ashx. Accessed 5 September 2022.
9. Social Science Statistics. Z Score Calculator for 2 Population Proportions. https://www.socscistatistics.com/tests/ztest. Accessed 26 March 2024.
10. Low LL, Kwan YH, Ko MSM, et al. Epidemiologic Characteristics of Multimorbidity and Sociodemographic Factors Associated With Multimorbidity in a Rapidly Aging Asian Country. JAMA Netw Open 2019;2:e1915245. Erratum in: JAMA Netw Open 2021;4:e211476.
11. Khan SU, Khan MZ, Raghu Subramanian C, et al. Participation of Women and Older Participants in Randomized Clinical Trials of Lipid-Lowering Therapies: A Systematic Review. JAMA Netw Open 2020;3:e205202-e.
12. Daitch V, Turjeman A, Poran I, et al. Underrepresentation of women in randomized controlled trials: a systematic review and meta-analysis. Trials 2022;23:1038.
13. Sosinsky AZ, Rich-Edwards JW, Wiley A, et al. Enrollment of female participants in United States drug and device phase 1–3 clinical trials between 2016 and 2019. Contemp Clin Trials 2022;115:106718.
14. Berger JS, Melloni C, Wang TY, et al. Reporting and representation of race/ethnicity in published randomized trials. Am Heart J 2009;158:742-7.
15. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. ICH Harmonized Guideline: General Principles for Planning and Design of Multi-regional Clinical Trials E17, 16 November 2017. https://database.ich.org/sites/default/files/E17EWG_Step4_2017_1116.pdf. Accessed 21 February 2023.
16. Ling WH, Lee SC. Inter-ethnic differences–how important is it in cancer treatment? Ann Acad Med Singap 2011;40:356-61.
17. Lee SC, Ng SS, Oldenburg J, et al. Interethnic variability of warfarin maintenance requirement is explained by VKORC1 genotype in an Asian population. Clin Pharmacol Ther 2006;79:197-205.
18. Singh R, Teo YY. “Asian” Phenotype Underestimates the Genetic Diversity of Asia yet Overstates its Impact on Variability in Drug Disposition and Pharmacodynamics. Clin Pharmacol Ther 2019;105:802-5.
19. Chan AW, Tetzlaff JM, Gøtzsche PC, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 2013;346:e7586.
20. Hudson BF, Oostendorp LJ, Candy B, et al. The under reporting of recruitment strategies in research with children with life-threatening illnesses: A systematic review. Palliat Med 2017;31:419-36.
21. O’Sullivan Greene E, Shiely F. Recording and reporting of recruitment strategies in trial protocols, registries, and publications was nonexistent. J Clin Epidemiol 2022;152:248-56.
22. Audisio K, Lia H, Robinson NB, et al. Impact of the COVID-19 Pandemic on Non-COVID-19 Clinical Trials. J Cardiovasc Dev Dis 2022;9:19.
23. Bagiella E, Bhatt DL, Gaudino M. The Consequences of the COVID-19 Pandemic on Non-COVID-19 Clinical Trials. J Am Coll Cardiol 2020;76:342-5.