StringMash.com

Excel, CSV files and UTF-8

Why a CSV file opens full of é and ’, and the ways to open, save and repair it.

Why it happens

A CSV file is plain text with no label saying which encoding it uses, so the program opening it has to guess. Most CSV files written today are UTF-8. When Excel opens one by double-clicking, it can read it as the system's legacy encoding instead, which on most Western systems is Windows-1252, and every accented letter and curly quote turns into two or three junk characters: café becomes café.

The exception is a file that starts with a byte order mark. Microsoft's own guidance is that a CSV file encoded with UTF-8 opens normally if it was saved with a BOM. Without one, open it through Get Data instead.

Opening a UTF-8 CSV file correctly

Start from a blank workbook rather than double-clicking the file. On the Data tab, choose Get Data, then From File, then From Text/CSV, and pick the file. The preview has a setting for the file's encoding: make sure it's set to UTF-8 before you load the data. The exact labels vary between Excel versions, so look for the encoding or file origin setting at the top of the preview.

Older versions have the Text Import Wizard instead, which Microsoft says is available from Tell me and in the ribbon as Get Data From Text. Its first step also asks for the file's origin, the same encoding choice.

Saving so the file opens correctly next time

When a CSV file has to open by double-clicking, give it a byte order mark. Recent versions of Excel offer a CSV file type with UTF-8 in its name in the Save As list, which writes one; if yours has it, use that. If it doesn't, save from another program that can add a BOM, or add it when you generate the file.

If you're the one producing the file, from a script or an export, write the three bytes EF BB BF at the start. Most CSV readers ignore them, and Excel uses them to recognise UTF-8. The byte order mark page has more on what those bytes do elsewhere.

Repairing a file that's already garbled

If the file was saved after Excel misread it, the garbled characters are now really in it. Copy the affected column or the whole sheet, paste it into the mojibake fixer, and paste the result back. Correct cells pass through unchanged, so it's safe to paste everything. Then save with a byte order mark so it doesn't happen again.

Questions

Why does my CSV show é in Excel?

The file is UTF-8, and Excel read it as Windows-1252. Open it through Data, Get Data, From Text/CSV with UTF-8 chosen as the encoding.

Why does the first column header start with ?

That's a byte order mark read with the wrong encoding. Opening the file as UTF-8 hides it, or the mojibake fixer can remove it.

Sources

Added . What's new