Internal Text Editor corrupts UTF-16 TOC files

[Dodo1]

We were forced to change all of our TOC files from UTF-8 to UTF-16 as Flare constantly entered UTF-16 into the header of the file instead of the correct UTF-8, even when it didn't change the format of the file itself. That came as an "improvement" with some update of Flare ("TOC files are now in UTF-16 format, be happy" or something alike, no possibility of staying with the same encoding as we use Source control and changing the encoding on its own destroyed the updated files).

Now Flare once again corrupts our files by changing its encoding while saving in the internal text editor.
How to get to the effect:
- create a project with UTF-16 toc file. Our TOC file is a little bit complicated, so it might come from that.
- Open Flare and load the project
- open the TOC in Flare editor and in the Internal Text Editor
- save the TOC (even with no changes)
- try to open the file in the flare toc editor (doubleclicking on the toc in the list). Message about the corruption on line 1, character 1 shows.
- after opening the file in Notepad++, the file format is undoubtedly UTF-8 without BOM, although the first line still holds the UTF-16 label.

We use Flare 4.2, still didn't have the will to go to version 5 - there is always some two-weeks work to convert the files so they are compatible with the new "enhanced" version and the automatic build works OK (something around 50 flare projects, some of them massive). The question is - does this bug also appear in Flare v.5 or was it repaired? Can anyone test that for me? I can't even find the change list between v4.2 and v5 anywhere on the MadCap site to check for starters.

[KevinDAmery]

I'm curious as to why you need to TOC to be open in both the Flare TOC editor and the text editor at the same time. Maybe there's a way to do what you need in the TOC that won't involve the text editor at all. (I think I've only opened a TOC file in a text editor twice: once to copy topic entries from one TOC to another, and the second time to help someone on the forum with a problem. So I wouldn't consider using a text editor on the TOC to be a normal workflow step.)

[Dodo1]

The actual reason I usually have to write in both views at once is that in the Flare editor is more simple when working with links etc, but sometimes it creates "weird" tags or leaves the <span></span> residues in the code, and this is not a desired behavior as we have to post-process the created xml file and search for particular parts of the code in it. Residues like that leave some parts of the code unrecognized. This is mainly because the word output cannot read background images in paragraph styles correctly etc. The process of creating a topic is usually composed of the simultaneous work in both the Flare editor and text editor and then cleaning the code solely in the text editor.

As for the TOC, the possibility to add topics in the flare editor is great, but linking to external files is not as intuitive and it is much more simple to copy/paste the part in the text editor and change the appropriate part of the path there. And again, I have encountered residues producing errors during compilation when I did that in the flare editor. The same goes for working with the alias files - older ID's not currently used in the resource.h file will not be shown in the flare editor even if the myalias file still has them included and that produces errors. These are critical when autobuilding the outputs.

On the side note, you don't have to have the TOC file opened in the Flare editor at all to produce this bug, it is sufficient to save the file in the internal text editor. The file also opened in the flare editor only warns you of the error right away instead only after closing the text editor view and opening it in the flare view (can't understand the reason why it is impossible to open the same file in the text editor first and in the Flare editor second, the only possible way of opening it in both is Flare editor first, internal text editor second... another bug or a feature?)

[Sharon_G]

We are seeing what appears to be this problem in reverse. When new TOC files are created they are created as UTF8. When the file is opened, edited, and saved in the TOC Editor, it is saved as UTF16. It is the TOC Editor that is changing the encoding, not the Text Editor.
To prove this I did the following:

• Created a new project from the Flare "Blank" Template.
  Added a TOC (Factory Templates > MyTOC > Test1TOC)
  Opened the Test1TOC with the Internal Text Editor to check that it was created in UTF8.
  Changed and saved the file (no change to the encoding)
  Closed the Text Editor.
  Opened the file in the TOC Editor.
  Changed the file, saved, and closed the file.
  Opened the file in the Text Editor and the encoding had changed to UTF16.

I then used tools available to a colleague in Engineering to open a TOC file that had not been opened in the Text Editor and it showed that the encoding was UTF8 before it was opened in the TOC Editor and UTF16 after it was changed and saved using the TOC Editor.
I'm wondering if this issue is related to the other in this thread? Also wondering if there is any way to prevent the TOC Editor from saving the files as UTF 16 since this is not going down well with our source control system?
I'm thinking that if not I'll have to submit it as a bug.
Thanks as always for your words of wisdom,
Sharon

[KevinDAmery]

I'm not aware of any tools in Flare to control the encoding of files, so submitting a bug would be a good course of action.

[trent the thief]

The UTF-8/16 issue has been around for quite some time http://forums.madcapsoftware.com/viewto ... 765#p50936.

I've identified another issue that occurs when creating TOC files.

There is something happening that cause Flare to make UTF-16 files with the characters "0d 00 0a 0d", which equates to CR and CR/LF. I can't tell where the CR/LF is coming from or why. I can't open those in Flare or in a text editor, but the other writer can open them and generate help without no issues. The other writer can read my files without difficulty.

I've removed the 0a character to fix the issue. That makes the string 0d 00 0d 00 as the remaining character shift to the left leaving them properly aligned to be read as UTF-16.

I'm waiting on the other writer to return from vacation to determine if the TOC was made from an imported Word file or built from scratch.