Multi-speaker environments: how TakeNote keeps up

The problem with real meetings

In an ideal world, every client meeting would be a clean two-way conversation: adviser speaks, client responds, no interruptions, no background noise, both parties with a neutral accent and a consistent speaking pace. In practice, meetings involve couples who finish each other's sentences, advisers who speak quickly when walking through technical detail, clients who express concern by talking over the explanation, and the ambient noise of a home office, a coffee shop, or a shared workspace.

Generic transcription tools struggle in these conditions. They conflate speakers, drop words during overlaps, and collapse nuance into flat, unattributed blocks of text. For a compliance record, that is not a minor inconvenience — it is a fundamental failure. If the transcript cannot accurately attribute who said what, it cannot evidence that the adviser fulfilled their suitability obligations.

Speaker diarisation: the technical challenge

Speaker diarisation — the process of segmenting audio and assigning each segment to a speaker — is one of the hardest problems in speech recognition. It requires the model to simultaneously track acoustic signatures (voice characteristics) and temporal patterns (turn-taking dynamics) while maintaining that tracking across interruptions, overlapping speech, and varying audio quality.

Most models achieve acceptable diarisation under ideal conditions. Performance degrades sharply when:

• Two speakers have similar acoustic profiles (common in couples, or same-gender meetings)
• One speaker frequently interrupts or speaks over the other
• Audio quality varies due to speakerphone, Bluetooth, or room acoustics
• A third party — a partner, a colleague — joins the meeting partway through

TakeNote is built to handle all of these scenarios.

How TakeNote approaches diarisation

MAI-Transcribe-1's diarisation engine uses a multi-pass approach: an initial acoustic segmentation pass identifies speaker boundaries, a second pass resolves ambiguous segments using broader conversational context, and a final pass applies a role-detection heuristic trained on financial advisory dialogue to label speakers as “Adviser” or “Client.”

On top of the base model, TakeNote applies a post-processing layer that:

• Merges consecutive segments from the same speaker where a brief overlap caused an artificial split
• Flags low-confidence attribution segments for adviser review rather than silently assigning them
• Preserves simultaneous speech in the transcript rather than dropping it, with a visual indicator showing overlap

Practical guidance for better results

TakeNote performs well across a wide range of recording conditions, but a few simple steps improve output further:

• Use a dedicated microphone where possible. Built-in laptop microphones pick up considerably more room noise than a USB desk microphone or a lapel mic. The improvement in transcript quality is noticeable.
• Record in-person meetings with the device centred on the table rather than in front of one participant. Even acoustic distance between speakers significantly helps diarisation.
• For phone calls, use TakeNote's direct dial-in feature rather than recording a speakerphone. The clean audio channel produces materially better results than ambient room capture.
• Review the flagged segments. TakeNote highlights any passage where speaker attribution confidence is below threshold. These reviews take seconds and are far faster than editing a poor transcript from scratch.

What advisers tell us

The most common feedback from advisers using TakeNote in their first month is that they expected to spend significant time correcting transcripts — and were surprised to find they barely needed to. The most common correction is a proper noun: a fund name, a client's address, a company the client mentions. Speaker attribution errors are rare in standard two-person meetings and flagged (rather than silently wrong) in more complex scenarios.

That is the standard TakeNote is built to. Not perfection in ideal conditions, but reliability in real ones.