Mixed-Language Meeting Transcription: How to Handle Code-Switching

A multilingual meeting rarely switches languages in neat blocks. A product manager may say, “We need to fix the onboarding 流程 before launch,” a sales team may alternate between English and Spanish customer terms, and an international engineering call may move between local-language discussion and English acronyms without warning. Humans use context to follow these switches. Automatic speech recognition has to infer them from the audio.

This behavior is called code-switching: alternating between languages within a conversation or utterance. It is a recognized speech-recognition problem, not simply the sum of two monolingual models. The 2025 CS-Dialogue dataset contains 104 hours of spontaneous Mandarin-English conversations from 200 speakers specifically because continuous mixed-language dialogue remains difficult to model. Earlier resources such as TALCS collected roughly 587 hours from real bilingual teaching scenarios, while the ASRU 2019 challenge released dedicated Mandarin-English code-switching data and evaluation tracks.

Why is code-switching harder than monolingual transcription?

A speech recognizer must decide not only which sound corresponds to which token, but also which language is active at a particular moment. In an utterance such as “Let’s move the demo 到星期五,” the language boundary occurs inside a sentence. The decoder has to change vocabulary and writing conventions without losing the surrounding context.

Research systems therefore add mechanisms that would be unnecessary in a simple monolingual pipeline. An Interspeech 2020 study used separate encoders for Mandarin and English plus a gating network for language identification, while Interspeech 2023 work explicitly modeled language-specific acoustic boundaries. These studies do not measure any consumer app, but they demonstrate why “supports English and Mandarin” is not equivalent to “handles English-Mandarin code-switching well.”

Pronunciation adds another variable. Embedded words are often pronounced according to the rhythm and accent of the surrounding language. A 2021 study on Mandarin-English recognition focused specifically on pronunciation variation in embedded English words. The practical implication is straightforward: a meeting tool has to recognize how your team actually says a borrowed term, not how a dictionary recording says it.

Which words fail most often in mixed-language meetings?

Short acronyms deserve special attention. API, CRM, KPI, QA and product initials provide little acoustic context, and a recognizer may convert them into ordinary words or phonetic approximations. A transcript can remain grammatically smooth while the acronym that matters to the project is wrong.

Proper nouns are another high-cost category. Product names, customer names, internal project codes and place names may be rare or absent from general training data. If the transcript feeds search, CRM notes or project documentation, one misspelled entity can make later retrieval harder even when the surrounding sentence is correct.

Numbers, dates and negation are more consequential than punctuation. “15 percent” versus “50 percent,” “approved” versus “not approved,” or the wrong quarter can change the meaning of a decision. These errors should be weighted more heavily than whether the system inserted a comma in the right place.

How should mixed-language transcription accuracy be measured?

There is no single universal metric that captures every multilingual workflow. English ASR is often evaluated with word error rate, while character-based metrics are common for languages such as Chinese. Code-switching research often uses a mixed error rate so different token systems can be evaluated together. The ASRU 2019 Mandarin-English challenge is one example of a benchmark designed around mixed speech rather than a monolingual score.

For a business evaluation, a simpler method is often more useful: create a critical-term list before opening the transcript. Include 20 to 50 names, acronyms, product terms, numbers and phrases that must be preserved. Score each tool on those terms using the same recording. This prevents a fluent-looking transcript from hiding the errors that actually cost time.

Separate error classes as well. Track speech-content errors, language-switch errors, speaker-attribution errors and formatting errors independently. A system that recognizes every word but assigns a commitment to the wrong speaker can still produce an unusable meeting record.

How do you run a fair code-switching transcription test?

Use real audio rather than a bilingual reading script. Natural meetings contain interruptions, incomplete sentences, borrowed terms, accents and variable microphone distance. Those conditions are the reason you need the tool in the first place.

1. Choose representative audioTake 10–20 minutes from a real meeting, interview, lecture or customer call with natural language switching.
2. Build a critical-term listWrite down names, acronyms, product terms, dates, numbers and switched-language phrases before comparing outputs.
3. Use the identical source fileEvery tool must receive the same audio so room acoustics and microphone quality cannot distort the comparison.
4. Score content before presentationCheck words and speakers first. Evaluate punctuation, paragraphing and visual polish only after the underlying transcript is correct.
5. Evaluate summaries lastConfirm important transcript details before judging whether a generated summary captured decisions and action items.

Your sample should contain more than one type of switch. Include a local-language sentence with an English noun, a complete phrase in the second language, and several acronyms or proper nouns. A test that contains only one easy pattern can overstate real-world performance.

Should you choose one language or automatic detection?

An explicit multilingual or mixed-language mode is the best starting point when a product offers one. However, “automatic language detection” is ambiguous across products. It can mean detecting one dominant language for the entire file rather than recognizing multiple languages inside a sentence.

If a tool forces you to choose one language, select the dominant language and then inspect the embedded language carefully. This can be workable when a meeting is overwhelmingly in one language with occasional borrowed terms, but it should not be assumed to work for balanced bilingual conversations.

Atter AI supports more than 90 languages, which makes it a candidate for multilingual workflows. Language count alone is not evidence of code-switching quality, however. The correct purchasing test is still your own mixed-language recording.

How do audio quality and overlapping speakers change the result?

Code-switching is only one source of difficulty. Reverberation, distant microphones, laptop fans, conferencing compression and overlapping speech can degrade the acoustic signal before the model even has to resolve a language boundary. When two people speak at once in different languages, both recognition and speaker diarization become harder.

This is why comparisons must use the same source file. Testing one product with close-mic audio and another with a room recording says little about their models. Keep the audio fixed, then inspect whether errors cluster around language switches, noisy segments or speaker overlaps.

Speaker attribution should be scored separately from text accuracy. A sentence can be transcribed perfectly but assigned to the wrong participant. For decisions, approvals and action items, that is a substantive error rather than a cosmetic one.

Why can one wrong switched-language term damage the summary?

Meeting summaries are downstream outputs. If the transcript changes a product name, drops a negation or mishears a number, the summarizer receives incorrect evidence. A polished summary cannot recover information that disappeared upstream.

For important meetings, review high-risk fields before accepting the summary: names, numbers, dates, decisions, owners, deadlines and negation. You do not need to manually proofread every filler word. The goal is to intercept errors that can change an action or business record.

This distinction also prevents a common evaluation mistake. Users sometimes choose a transcription tool because its summary reads better, even though its underlying transcript is less faithful. Transcript fidelity and summary usefulness are related but separate criteria.

Does a published accuracy number predict code-switching performance?

Not by itself. Atter AI’s verified 98.7% accuracy figure applies to clean audio. It should not be interpreted as a guarantee for every mixed-language, noisy, accented or multi-speaker recording. Any accuracy claim is meaningful only with its test conditions.

Code-switching researchers build dedicated corpora for the same reason. CS-Dialogue emphasizes spontaneous full-length conversations rather than isolated bilingual sentences, and the SEAME, TALCS and ASRU resources all exist because monolingual benchmarks do not fully represent mixed speech.

For a real workflow, define accuracy around error cost. A legal or financial meeting may weight numbers and negation heavily. A software team may care most about acronyms and repository names. A customer interview may prioritize speaker identity and product terminology. The “best” recognizer can therefore differ by use case.

What can improve mixed-language meeting transcripts?

Start with the recording. Place microphones closer to speakers, reduce room echo where possible and avoid having several people share a distant laptop microphone. Better signal quality gives every recognition system more information to work with.

Maintain a team glossary of recurring names, acronyms and product terms. If a product supports custom vocabulary, use it; if not, the glossary still accelerates review because editors know exactly what to search for. Do not force colleagues to pronounce borrowed words unnaturally just to satisfy the software. A useful tool should be evaluated against normal speech.

Finally, separate transcription review from summary review. First ask whether the record is faithful. Then ask whether the summary is useful. Combining those questions makes it too easy for polished prose to conceal recognition errors.

When should you switch transcription tools?

Occasional punctuation mistakes are rarely a reason to migrate. Repeated failures on core terminology, names, numbers or speaker attribution are different, especially when the transcript feeds systems of record.

The useful comparison is not which vendor has the largest language list. It is which system makes the fewest expensive errors on your audio and lets your team correct the remaining ones efficiently.

A practical mixed-language transcript checklist

After transcription, search for every acronym in your glossary. Then verify product names, participant names, dates, percentages, currency amounts and version numbers. These are compact pieces of text with disproportionate business impact.

Next, spot-check five to ten seconds on both sides of each language switch. If words disappear or change script around the boundary, language switching is probably a primary failure mode. Then inspect overlapping speech and speaker labels. Formatting should come last.

If accent variation is the larger problem, read our guide to accent and speech-recognition accuracy. The two issues can coexist, but tracking them separately makes tool comparisons much more informative.

Frequently asked questions

Can AI transcribe mixed-language meetings?

Yes, but bilingual language support alone is not enough. Test intra-sentence switching, acronyms, names and natural pronunciation with a representative recording.

Why is code-switching hard for speech recognition?

The model has to resolve speech content and language boundaries at the same time. Mixed-language data are also less abundant than major monolingual datasets, which is why code-switching remains a dedicated ASR research area.

How should I test multilingual meeting transcription?

Use the same 10–20-minute real recording in every tool. Prepare critical terms first and score content, language switches and speaker attribution separately.

Should I select one language or automatic detection?

Prefer a documented multilingual mode. If only one language can be selected, use the dominant language and explicitly inspect the embedded language for errors.

Can transcription errors corrupt AI meeting summaries?

Yes. Summaries depend on the transcript, so wrong names, numbers, negation and decisions can flow directly into the generated summary.

Does 98.7% accuracy apply to code-switched meetings?

No. Atter AI’s 98.7% figure is for verified clean-audio conditions and should not be generalized to every multilingual meeting.