What character encoding is
Computers store text as numbers. A character encoding is the agreement about which number means which character. If text is saved with one encoding and read with another, you get garbled output — the famous ’ or � symbols, sometimes called mojibake.
ASCII: the original 128
ASCII maps 128 characters — English letters, digits, and basic punctuation — to the numbers 0–127. It's simple and universal, but it has no way to represent accented letters, emoji, or non-Latin scripts.
UTF-8: room for everything
UTF-8 is the modern standard. It can represent every character in every language, plus emoji, using 1 to 4 bytes per character. Crucially, it's backward-compatible: the first 128 characters are identical to ASCII, so plain English text is the same in both.
Avoiding garbled text
- Save files as UTF-8 (it's the default in most modern editors).
- Declare it in HTML with
<meta charset="utf-8">. - Set the right
Content-Typecharset on API responses. - When you see garbage, the fix is almost always matching the read encoding to the save encoding.
How UTF-8 stores a character
UTF-8 is a variable-length encoding. Characters in the ASCII range take one byte. Most accented Latin letters, Greek, Cyrillic, Hebrew and Arabic take two bytes. Most Chinese, Japanese and Korean characters take three. Emoji and some rare scripts take four. For example, A is the single byte 41, é is C3 A9, and € is E2 82 AC. The first bits of each byte tell a decoder how many bytes belong to the character, which is why a decoder can resynchronise after a corrupted byte instead of garbling everything that follows.
Reading mojibake backwards
Garbled text usually has a recognisable shape, and the shape tells you what went wrong. If you see é where é should be, UTF-8 bytes (C3 A9) were read as Windows-1252 or Latin-1, where C3 is à and A9 is ©. The curly apostrophe turning into ’ is the same mistake with a three-byte character. If you instead see the replacement character �, the opposite happened: bytes that are not valid UTF-8 (often Latin-1 text) were read as UTF-8, and the decoder gave up on them.
Common places encoding breaks
- CSV files opened in Excel: older versions of Excel assume a legacy encoding unless the file starts with a byte-order mark. Exporting as “CSV UTF-8” avoids it.
- Databases: in MySQL, the old
utf8character set only stores up to three bytes, so emoji fail. Useutf8mb4. - Email and HTTP headers: non-ASCII text in headers needs its own encoding (for example RFC 2047 in email subjects).
- URLs: non-ASCII characters in a URL are sent as UTF-8 bytes written in percent-encoding, so
ébecomes%C3%A9.
Checking what you actually have
When text looks wrong, look at the bytes rather than the glyphs. Percent-encoding a string with our URL encoder shows its UTF-8 bytes directly, and the HTML entity encoder shows which characters would need escaping in a web page. If the bytes are right but the display is wrong, the problem is the declared encoding, not the data — fix the header or the meta charset, not the text.
Frequently Asked Questions
Is UTF-8 better than ASCII?
For modern use, yes — UTF-8 supports every language and emoji while remaining compatible with ASCII for basic English text.
Why do I see ’ instead of an apostrophe?
That's UTF-8 text being read as a different encoding (often Latin-1). Re-reading it as UTF-8 fixes it.
Does UTF-8 use more space?
Only for non-ASCII characters. Plain English text is the same size as ASCII because those characters use a single byte.