Storing Characters: ASCII and Unicode
How text becomes binary, why ASCII was not enough, and the exact trade off between ASCII and Unicode that examiners ask about every year.
Character sets
A computer cannot store the letter A. It can only store binary. So every character is given a number, and that number is stored in binary.
A character set is the agreed list of characters a computer can use, together with the binary code that represents each one.
The agreement matters enormously. If the sending computer used one mapping and the receiving computer used a different one, every message would arrive as gibberish. A shared character set is what makes text portable between machines.
A character set is a defined list of characters recognised by a computer system, where each character is represented by a unique binary code.
ASCII
ASCII stands for American Standard Code for Information Interchange.
- Uses 7 bits per character
- 27 = 128 different characters
- Covers the English alphabet in upper and lower case, digits 0 to 9, punctuation, and control characters such as carriage return
- Extended ASCII uses 8 bits, giving 256 characters, which adds some accented characters and symbols
Useful ASCII values
| Character | Denary code |
|---|---|
| A | 65 |
| Z | 90 |
| a | 97 |
| z | 122 |
| 0 | 48 |
| 9 | 57 |
| space | 32 |
You do not have to memorise the table, but you should know two patterns:
- Characters are stored in a logical order, so B is one more than A. If A is 65 then F is 70.
- The digit character '5' is not the number 5. The character '5' has the ASCII code 53. This is why
"5" + "3"gives"53"in a program while5 + 3gives8.
A question gives you A = 65 and asks for the code for E. Count carefully: A 65, B 66, C 67, D 68, E 69. Off by one errors here are extremely common.
Step 1 of 4 Every character has a number. In ASCII, capital H is 72 and lower case i is 105.
- Each character has a code number, and the computer stores that number in binary.
- The codes run in order, so you can work out C from A. The character 7 is not the number 7.
- More bits per character means more characters: 128, then 256, then over a million.
Unicode
ASCII has 128 characters. There are thousands of writing systems in the world. Chinese alone uses tens of thousands of characters, and none of them fit.
Unicode was created to solve this.
- Uses 16 bits or more per character, depending on the encoding
- Can represent over a million characters
- Covers every major writing system: Latin, Cyrillic, Arabic, Greek, Hebrew, Chinese, Japanese, Korean, Devanagari, and also emoji and mathematical symbols
- The first 128 Unicode codes are identical to ASCII, which means old ASCII text is still valid Unicode
The trade off
| ASCII | Unicode | |
|---|---|---|
| Bits per character | 7, or 8 for extended | 16 or more |
| Number of characters | 128, or 256 extended | Over a million |
| File size | Smaller | Larger for the same text |
| Language support | English and basic symbols only | Almost every written language |
Unicode uses more bits per character than ASCII, so text files are larger and take up more storage and more bandwidth to transmit.
Which to use
- A system handling only English text where storage is very tight, such as a small embedded device, may still use ASCII.
- Anything on the modern web uses Unicode, because websites must handle names, addresses and content in any language.
Key terms you must be able to define
- Character set
- A defined list of characters recognised by a computer, each with a unique binary code.
- ASCII
- American Standard Code for Information Interchange. A 7 bit character set representing 128 characters.
- Extended ASCII
- An 8 bit version of ASCII representing 256 characters.
- Unicode
- A character set using 16 or more bits per character, able to represent over a million characters from all major writing systems.
- Character code
- The unique number assigned to a particular character within a character set.
Flashcards
Say the definition out loud before you flip the card. Retrieving it from memory is what builds recall, reading it again is not.
Reach the top grade on this topic
There are only two hard marks on this topic, and they are both about why.
Why a character set is needed at all. Not "so computers can store text", which is circular. The real reason: computers can only store binary, so each character must be assigned a numeric code, and the code must be agreed between systems so that text transfers correctly.
Why Unicode costs more. Not "Unicode is bigger". The reason: Unicode uses at least 16 bits per character compared with 7 or 8 for ASCII, so the same message takes roughly double the storage and double the transmission time.
The application marks come from choosing correctly for a scenario. A global social network needs Unicode because users write in many languages. A tiny sensor logging English status codes into 2 KB of memory may sensibly use ASCII.
- Be able to calculate the size of a text file given the character set
- Be able to work out any character code from a given starting value without off by one errors
- Be able to give one advantage and one disadvantage of Unicode over ASCII
Mistakes that cost marks here
- Saying Unicode 'replaced' ASCII entirely. The first 128 Unicode codes are the same as ASCII, so ASCII text is a subset of Unicode.
- Miscounting character codes. If A is 65, then E is 69, not 70.
- Treating the character '7' as the number 7. Its ASCII code is 55.
- Saying ASCII has 256 characters. Standard ASCII is 7 bit and has 128. Extended ASCII is 8 bit with 256.
- Giving 'it supports more languages' as the only advantage, without mentioning the file size cost as the trade off.
In ASCII, the code for uppercase A is 65 and lowercase a is 97, a difference of exactly 32. That is not a coincidence: 32 is a single bit, so changing letter case is one bit flip. Early programmers designed it that way deliberately.
Knowledge check
Exam-style questions
Five written questions in the style of the real paper. Write a full answer, then let the marker check it against the mark scheme.
- LA list of characters that a computer system recognises
- LEach character is represented by a unique binary code or number
A character set is the defined list of characters that a computer system is able to recognise and use, such as letters, digits, punctuation and control characters. Each character in the set is assigned its own unique binary code, so that the computer can store and transmit text as binary values.
- LAdvantage: Unicode represents far more characters
- LAdvantage explained: it supports the writing systems of most world languages
- LDisadvantage: Unicode uses more bits per character
- LDisadvantage explained: files are larger, using more storage and bandwidth
An advantage of Unicode is that it can represent over a million different characters rather than the 128 available in standard ASCII, which means it supports the writing systems of virtually every language including Chinese, Arabic and Cyrillic, as well as symbols and emoji. This makes it essential for any system used internationally. The disadvantage is that Unicode uses at least 16 bits per character compared with 7 or 8 bits in ASCII, so the same piece of text takes up roughly twice as much storage space and takes longer to transmit over a network.
- LCorrect answer 81
- LExplains that characters are stored in sequential order, so counting forward from M gives Q
The ASCII code for 'Q' is 81. Characters in ASCII are stored in sequential order, so counting forward from M at 77 gives N as 78, O as 79, P as 80 and Q as 81.
- LASCII size calculated as 4000 bytes
- LUnicode size calculated as 8000 bytes
- LDifference stated as 4000 bytes
In 8 bit extended ASCII each character takes one byte, so 4000 characters take 4000 bytes. In 16 bit Unicode each character takes two bytes, so the same 4000 characters take 8000 bytes. The difference in file size is therefore 8000 minus 4000, which is 4000 bytes, meaning the Unicode version is twice the size.
- LCustomers will write in many different languages
- LASCII only covers English letters, digits and basic symbols
- LUnicode can represent the characters of virtually all writing systems
- LDrawback: larger files, so more storage and bandwidth is used
The company should use Unicode because its customers are spread across the world and will enter names, addresses and messages in many different writing systems. Standard ASCII can only represent 128 characters, which covers English letters, digits and basic punctuation, so any customer writing in Arabic, Chinese, Greek or Russian would find their text could not be stored or displayed correctly. Unicode can represent over a million characters and covers essentially every major script, so all customers are supported equally. The drawback is that Unicode uses at least 16 bits per character rather than 7 or 8, which roughly doubles the size of stored text and increases the bandwidth needed to send pages, adding to hosting costs and slightly slowing page loads.