Bush hid the facts

Bush hid the facts is a common name for a bug present in some Microsoft Windows applications, which causes a file of text encoded in ASCII or its superset (such as in a Windows code page) to be interpreted as if it were UTF-16LE, resulting in mojibake. When "Bush hid the facts" (without newline or quotes) is put in a new (pre-Vista) Notepad document and saved, closed, and reopened, the nonsensical Chinese characters "畂桳栠摩琠敨映捡獴" (pinyin: Liù bèn rěn mó tiǎn tǒu yìng jiǎn měng) appear instead.

While "Bush hid the facts" is the sentence most commonly presented on the Internet to induce the error, the bug can be triggered by many sentences with characters and spaces in a particular order so that the bytes match the UTF-16LE encoding of valid (if nonsensical) Chinese Unicode characters. Other popular strings are "this app can break", "acre vai pra globo" (Portuguese for "Acre goes to Rede Globo"), and "aaaa aaa aaa aaaaa".^[1] The bug is triggered even by the text "a ".

The bug occurs when the string is passed to the Win32 charset detection function IsTextUnicode with no other characters. IsTextUnicode sees what it thinks is valid UTF-16LE Chinese and returns true, and the application then incorrectly interprets the text as UTF-16LE.^[2]

Many text editors and tools exhibit this behavior because they use IsTextUnicode as well.

Discovery

The bug appeared for the first time in Windows NT 3.5, released 1994, but was not discovered until early 2004.^[3] Older versions of Notepad such as those that came with Windows 95, 98, ME, and NT 3.1 do not include Unicode support, so the bug does not occur there.

The bug exists in all successive versions of Windows until and including Windows XP, but was not present in Windows Vista and afterwards.

Workarounds

Editing the text to not be a pattern that triggers this bug will avoid it. For instance, adding a new line in the first 20 characters will work.

If the file is saved as "UTF-8" rather than "ANSI" (ANSI means in reality Windows-1252 on systems using western European languages) the text displays correctly, because Notepad prepends a byte order mark as a non-standard UTF-8 flag, which is a different pattern that does not trigger this bug. UTF-8 without the byte order mark would still trigger the bug, as this sequence is represented identically in UTF-8 as in ASCII.

The bug is also avoided by saving as "Unicode", which in Microsoft Windows usually means UTF-16LE with a byte order mark. (The Chinese characters seen when the bug is active comes from interpreting as UTF-16LE without a byte order mark)

To retrieve the original text using Notepad, bring up the "Open a file" dialog box, select the file, select "ANSI" or "UTF-8" in the "Encoding" list box, and click Open. (Under Windows 2000, Notepad lacks the "Encoding" list box. Notepad2 makes the same error (by trusting IsTextUnicode), and also lacks an option to override encoding when opening a file. However, WordPad opens the text file correctly by default.)

References

External links

• The Notepad file encoding problem, redux – Raymond Chen
• IsTextUnicode – MSDN Library