Velnquix / Canada
Detecting language changes inside a message
A message can switch languages within a sentence. Identifying those changes helps you inspect how a text-processing pipeline handles the actual input.
Define what counts as a switch
Separate named entities, borrowed words and meaningful changes in language. Label examples consistently and record cases on which reviewers disagree.
Evaluate at the right level
A document-level language label can hide short French passages in an English message, or the reverse. Inspect token or span-level behaviour where the task requires it.
Use a practical fallback
When language identification is uncertain, preserve the original text and route the message for review. Test the downstream effect rather than treating detection accuracy as the final outcome.