ai-coding-minesIndexGitHub

PowerShell 5.1 reads UTF-8 files as CP949

Windows and PowerShell

Symptom

Read a UTF-8 file with Get-Content, transform it, write it back: every non-ASCII character self-destructs. No exception.

Cause

Get-Content in PS 5.1 defaults to the system ANSI code page (949 on Korean Windows, 1252 on US English, 932 on Japanese). The UTF-8 bytes are decoded as that code page the moment they become a string, so the damage is already done; writing back just bakes it into the file.

Why this bites non-ASCII users

On an English-locale machine with ASCII-only content, code page 1252 and UTF-8 agree on every byte you'd normally see, so this bug is invisible. Add one Hangul syllable (three UTF-8 bytes) and 949 decodes them as one-and-a-half garbage characters. Non-ASCII content is what makes the default encoding matter.

Fix

Don't trust the defaults of Out-File / Set-Content either. Any file another tool will read gets an explicit encoding.

Verification

Read the file back after writing and compare it to the original string. Eyeballing console output is not enough: the console code page adds one more translation layer and blurs the verdict.