Talk to the bot: A scoping review on using AI-powered transcription tools in qualitative research

Dear Editor,

Untangling complex social phenomena for health-related problems relies on qualitative research methods like one-to-one in-depth interviews and focus group discussions.¹ However, transcription of these interviews and discussions is costly, labour-intensive² and prone to errors.¹ Furthermore, manual transcription is translational. The transcriber interprets the interview during transcription, inevitably incorporating their own biases during the act of translation,³ affecting subsequent analyses.

Artificial intelligence (AI)-powered transcription tools are set to replace manual transcription. However, there are doubts about the reliability and validity of the transcription, especially in healthcare research. Thus, to understand the usability of AI transcription tools in real-world research settings, researchers need to assess whether these tools can meet their specific needs. This study conducted a scoping review to summarise the studies that have evaluated AI transcription tools for qualitative interviews, and created a guide to help healthcare researchers choose a suitable tool.

A literature search conducted on 2 May 2025 on PubMed, JSTOR and IEEE, generated 6, 83 and 790 hits, respectively. The following search terms were used: (“text analytics” or “text analysis” or “speech analysis” or “speech to text” or “automated speech recognition” or “speech processing” or “visual analytics” or “video analysis”) AND (“research interview” or “qualitative interview” or “qualitative research” or “qualitative data”) in all fields. Ten additional articles identified via a Google Scholar search were also included. A total of 886 articles were identified after removing duplicates. After excluding articles for reasons such as the study not involving transcription, qualitative research interviews, or being descriptive technical papers, 9 comparative analysis studies were identified.

These 9 articles used 11 different automatic speech recognition tools. Seven studies were published from 2024 to 2025, 1 in 2017 and 1 in 2018. Three studies transcribed interviews in multiple languages,^1,4,5 3 involved interviews conducted in English,^2,3,6 and the rest were conducted in German,⁷ Indonesian⁸ and Swedish.⁹ Majority involved one-on-one interviews, with 1 study involving both one-on-one and group interviews.²

AI transcription tools were relatively accurate in transcribing interviews, with 8 out of 9 studies supporting its use. The accuracy varied with language, with English interviews unsurprisingly performing best.^1,5 Audio conditions (e.g. minimal background noise, no crosstalk)^1,2 and speech (e.g. not heavily accented, slow tempo)³ also improved accuracy. Some researchers found that these errors did not have a significant impact on meaning, and were easily rectified.^2,6

Conducting qualitative research in the healthcare setting presents an additional challenge. Medical jargon, colloquial language and non-verbal or tonal cues limit the accuracy of these tools.^1,2,7,9 More dangerously, biases inherent to the training methods of AI tools reinforce stereotypes. Bircan and Ceylan observed errors (e.g. substitution, insertion errors) that resulted in biases, which reinforced Orientalist and Islamophobic stereotypes.⁴ For instance, “staying in Arab countries is difficult with no visa” was erroneously transcribed to “staying in Arab countries is difficult with all these conflicts.”

AI transcription tools can lead to considerable cost and time savings. Eftekhari³ saved GBP3772 from allocated transcribing costs by using Microsoft Teams (Microsoft, Redmond, WA, US), a free tool. KALDI (http://github.com/kaldi-asr/kaldi) transcribed at 1.1 times the recording duration,¹ Microsoft Teams almost instantaneously³ and Google’s Speech-to-Text about 7 times faster than humans.⁷

However, these time savings were, in turn, spent correcting the transcripts manually. Tolle at al. reported correction time ranging from 1.7-fold for Portuguese to 16-fold for Tamil,¹ while Eftekhari required 1.5–3.5 hours correcting interviews averaging 65 minutes.³

AI transcription tools were generally easy to use, with positive feedback. Participants, despite most having no or limited experience with these tools, could independently install the software and use the tools.^1,8 Tolle et al.’s participants evaluated the tool highly, scoring it 4.42 (standard deviation [SD] 0.90) on perceived usefulness and 4.50 (SD 0.80) on likeliness of future use via 5-point Likert scales.¹

Healthcare research contains especially sensitive data. Researchers can maintain privacy by ensuring that data are kept locally.¹ Cloud- and website-based tools, such as Microsoft Teams and Google’s Speech-to-Text, require data upload to external servers, introducing security risks.^2,5 Researchers should also evaluate vendors’ security protocols and compliance with privacy regulations such as the Health Insurance Portability and Accountability Act, and choose vendors that have the necessary approvals and passed relevant security reviews.⁵

Researchers need to prioritise the features of different AI transcription tools to align with the research team’s needs. For instance, in dealing with sensitive healthcare data, an AI transcription tool with local data storage offers better security, but requires additional hardware and programming expertise. Paid tools generally offer a more seamless experience and better integration with downstream data analysis software, but can be costly, cloud-based and therefore susceptible to security breaches. Cloud-based tools should not be considered when working with highly sensitive data, as they require uploading of recordings and may monetise data, compromising participant anonymity.^10,11 Clinical researchers can consider free or low-cost options, such as aTrain,¹² NoScribe (https://github.com/kaixxx/noScribe?tab=readme-ov-file) and MacWhisper (https://goodsnooze.gumroad.com/l/macwhisper).

Transcription of interviews that feature technical or colloquial language, strong accents or controversial topics tend to be prone to inaccuracies, leading to higher word error rates (WER). Correcting a transcript with WER >20% can be time-consuming, requiring up to 4-fold the recording time.⁹ Urban and Mehrotra suggested that the research team should consider additional training and customisation of the tool if WER >30%, though not all teams have such resources.¹³

Choosing an AI transcription tool can be a complicated process, especially for the uninitiated. Post-marketing assessments of commercial AI tools are inherently biased, and researchers should aim to assess tools independently. This study proposes a framework of major considerations in selecting an AI transcription tool in Table 1.

Table 1. Main considerations in selecting an AI transcription tool for interviews in qualitative research.

Researchers should consider post-transcription strategies to “scrub” the AI-generated transcript. Some studies promote using AI transcription tools as a “good enough” first step, with manual cleaning for accuracy.^1,2 Others suggest using another AI tool, such as ChatGPT, to clean the transcript, though with caution.⁹ Researchers may opt for manual cleaning for recordings that are challenging for AI tools, such as those involving colloquial or technical language, which is usually the case for interviews conducted in Singapore as they often involve Singlish.

In conclusion, AI transcription tools offer significant time and cost savings in qualitative research, but researchers should address challenges such as transcription quality, biases, security and trade-offs when selecting a tool. The authors hope that researchers can use Table 1 as a guide to selecting the most appropriate tool, but acknowledge that the considerations are not exhaustive, especially given the evolving nature of AI. Stricter guidelines to identify acceptable AI tools in research from regulatory bodies might aid in addressing these challenges.

REFERENCES

1. Tolle H, Castro MDM, Wachinger J, et al. From voice to ink (Vink): development and assessment of an automated, free-of-charge transcription tool. BMC Res Notes 2024;17:95.
2. Bokhove C, Downey C. Automated generation of ‘good enough’ transcripts as a first step to transcription of audio-recorded data. Methodol Innov 2018;11:2059799118790743.
3. Eftekhari H. Transcribing in the digital age: qualitative research practice utilizing intelligent speech recognition technology. Eur J Cardiovasc Nurs 2024;23:553-60.
4. Bircan T, Ceylan D. Machine Discriminating: Automated Speech Recognition Biases in Refugee Interviews. J Immigr Refug Stud 2024;1-16.
5. Meanwell E, Walsh A, Berg P, et al. Automated Transcription Service: Enabling Transcription at Scale for Social Science Research. In: Practice and Experience in Advanced Research Computing 2024: Human Powered Computing. Providence RI USA: ACM; 2024
6. Pfeifer VA, Chilton TD, Grilli MD, et al. How ready is speech-to-text for psychological language research? Evaluating the validity of AI-generated English transcripts for analyzing free-spoken responses in younger and older adults. Behav Res Methods 2024;56:7621-31.
7. Höhne JK, Lenzner T, Claassen J. Automatic speech-to-text transcription: evidence from a smartphone survey with voice answers. Int J Soc Res Methodol 2025;1-8.
8. Adianto R, Satriawan CH, Lestari DP. Transcriber: An Android application that automates the transcription of interviews in Indonesian. In: 2017 International Conference on Advanced Informatics, Concepts, Theory, and Applications (ICAICTA). Denpasar: IEEE; 2017.
9. Helin M. Evaluating the Accuracy of Automated Transcription using Open AI’s Whisper Model for Swedish IT-Related Interviews : A Comparative Analysis with Manual Transcription, 2024. https://urn.kb.se/resolve?urn=urn:nbn:se:du-49741. Accessed 22 January 2025.
10. Da Silva J. Producing ‘good enough’ automated transcripts securely: Extending Bokhove and Downey (2018) to address security concerns. Methodol Innov 2021;14:2059799120987766.
11. Ong JCL, Chang SYH, William W, et al. Ethical and regulatory challenges of large language models in medicine. Lancet Digit Health 2024;6:e428-32.
12. Haberl A, Fleiß J, Kowald D, et al. Take the aTrain. Introducing an Interface for the Accessible Transcription of Interviews. J Behav Exp Finance 2024;41:100891.
13. Urban E, Mehrotra N. Test accuracy of a custom speech model, 2024. https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-custom-speech-evaluate-data?pivots=speech-studio. Accessed 22 January 2025.