I wrote a small app to redact sensitive data from log files using regular expressions. The code iterates through the logs one line at a time and processes them. The problem I'm having is related to seemingly mixed encoding in some lines.
Example: I see the word Microsoft followed by the copyright symbol. the copyright symbol is hex AE. I also see C++ in the same line. Each plus sign is hex 2B.
If I open the file as UTF7 I can write to the output file and the copyright symbol is intact in the output and the plus signs are not. If I open the file as UTF8 the plus signs are intact and the copyright symbol is not in the output file. I'm mystified about this behavior. Can anyone clue me in to why this is happening? If I open the file in an editor like UltraEdit all the characters look fine.
...
Go to the complete details ...