Update character encoding

In update 150502 of RIMMF3, we added an utility named 'Data encoding updater' to the Tools menu:

This utility will convert diacritics in older RIMMF data (data created before Christmas, 2014; at this time, support for unicode code points was added RIMMF3) to use unicode code points.

This conversion is required if you want to use this data with 'unicode RIMMF'.

If you do not have any older RIMMF data that you want to bring forward, or if it does not use diacritics, then you can ignore this note.

The 'Data encoding updater' operates on the currently selected data folder.

First it copies all of the records in the folder to a folder of the same name (with the suffix '-backup') to the RIMMF 'temp' directory (typically, 'My Documents\RIMMF2\temp\').

Then loads it each file, checks it for diacritics, and if present, tries to convert them to unicode code points. The conversion should be 100% successful for the common Latin language diacritics. After that, however, there may be some data loss (these characters may be converted to the 'unknown unicode' character \uFFFD, for example).

Once the conversion is done (and it runs very quickly), the data folder is ready for unicode RIMMF; it does not need to be run again.

If you have multiple data folders, then you will need to go through the steps here for each one

1. Go to Tools and select 'Change data folder'
2. The EI will be indexed
3. Go back to Tools and select 'Data encoding updater'
4. The EI will be re-index

Its important to note that this utility cannot be run in 'unicode RIMMF'.