Powershell – ASCII encoding is changing special characters to question marks

Question:

I’m using a Powershell script as follows to convert a string to XML then export to a file (done this way to keep indenting):

The destination has to be ASCII formatted due to a vendor restriction. The issue I’m seeing is ASCII just changes special characters into questions marks (example: Ö becomes ?).

If I use UTF8 encoding the output looks totally fine. I’ve even tried saving to UTF8 then converting to ASCII, does the same thing (exports a question mark):

If I try and replace the characters in the string before the conversion to XML (using ASCII code Ö) it simply converts the ampersand and leaves the rest, making it useless.

Is there any way to have Powershell correctly save those characters into the file?

EDIT: I would like to see the special character in the outputted file, but if that is not ASCII-compliant, I’d like to see the ASCII code for it (in this example, Ö)

I also don’t want to see just an O, I need the actual character.

Answer:

All characters in an XML document are Unicode. However, a representation of an XML document has a document encoding. Characters that are not members of that character set are written as character entity references, often numerically and in hexadecimal notation. The number is the Unicode codepoint.

It seems your partner’s requirement is to use ASCII as the document encoding.

XmlDocument is a bit hard to work with but an XmlWriter with settings for the document encoding will work:

This puts out an ASCII-encoded text file with an XML declation declaring the document encoding is ASCII and uses hexadecimal numeric character entity references for XML content characters that can’t be represented in ASCII:

As you can see here in the C1 Controls and Latin-1 Supplement block, U+00D6 (&#D6;), is Ö LATIN CAPITAL LETTER O WITH DIAERESIS

Source:

Powershell – ASCII encoding is changing special characters to question marks by licensed under CC BY-SA | With most appropriate answer!

Leave a Reply