Page 1 of 1

Topics can get incorrectly included in HTML5 output

Posted: Thu Apr 15, 2021 11:33 am
by paul_collins
I'd like to make people aware of what I believe to be an extremely serious bug in Flare that results in content (topics) that should be conditioned out being included in HTML5 output.

The "excluded" topics are findable only through search (as well as the topic htm files physically being present in the output if you know the path/filename).

In our case we generate several different web outputs, aimed at difference audiences, from the same Flare project (as there is some shared content), and it is important that there is no bleed of unrequired content from one set of output to the other. This happened to us and we only realised when we noticed that some "wrong" topics were appearing in search results.

Here's how you can make it go wrong:
1. In the HTML5 target - Advanced - set "Content to include" as Content linked directly or indirectly from the target.
2. Create a new topic e.g. newtopic.htm. Do not add it to a ToC.
3. Create a new snippet e.g. linksnippet.flsnp that includes among its content a link (hyperlink or cross reference) to the new topic (newtopic.htm), and then add that snippet to an existing topic that is already included in your target output e.g. existingtopic.htm so as to (correctly, for now) pull in that new topic.
4. Create an "excludeme" condition and apply it to the link in linksnippet. (The condition can be applied directly on the link, or to the paragraph containing the link.)
5. In existingtopic.htm apply a snippet condition to linksnippet to make it exclude the conditioned content (the link to newtopic) in the snippet.
6. Build the target.
7. View the output and from within the existingtopic.htm topic you'll see that the paragraph/hyperlink (that you applied the "excludeme" condition to) is correctly excluded i.e. there is no link to newtopic.htm from existingtopic.htm.
8. However, the newtopic.htm file will have been included in the output (and searching for any content in that topic will find it and list it in the search results).

It's worth noting that if the "excludeme" condition is applied at the target level (rather than within the topic hosting the snippet) then the newtopic.htm file is fully excluded from the output, as expected.

I raised this as a bug with Madcap and they say it is intentional behavior: "The reasoning behind keeping the topic in the output (even though the text containing the link to the topic being removed was successfully conditioned out) is that the snippet and its linked content can be present in the project elsewhere where the paragraph tag and its linked topic would be included intentionally. This functionality is there to prevent breaking other single sourced content."

To me this response is nonsensical. Surely the point when the snippet and the linked content is intentionally used is the time to include that linked topic in the output i.e. don't assume
it might be used elsewhere; it should only include it when it is actually used. My concerns about security and unintentional leaking of sensitive content were disregarded. I assume when people check their output they don't always use search to confirm that content they thought they had excluded is actually being excluded.

If you agree with me and this bothers you then please raise it as a bug with Madcap and hopefully they will change the behaviour of Flare if enough people complain.

Re: Topics can get incorrectly included in HTML5 output

Posted: Thu Apr 15, 2021 3:37 pm
by SteveS
This has always been a feature of Flare, most people become aware of it and explicitly exclude content they don't want to be included in a build.

If you are unhappy with the behaviour I suggest you lodge a feature request. Maybe a request for "exclude all except explicit include" option or similar.

Re: Topics can get incorrectly included in HTML5 output

Posted: Thu Apr 15, 2021 5:06 pm
by paul_collins
I'm still failing to see any justification for the current behavior.

Why should I have to explicitly exclude any of the other content in my project that isn't linked directly or indirectly to anything in the build?

Re: Topics can get incorrectly included in HTML5 output

Posted: Fri Apr 16, 2021 3:09 am
by ChoccieMuffin
I can see your reason for concern, so I tend to avoid relying on that option, and instead apply conditions to files (topics, images, snippets) in Content Explorer and then condition out things I don't want at a target level. Topics that are needed in all outputs don't have any conditions applied.

Re: Topics can get incorrectly included in HTML5 output

Posted: Fri Apr 16, 2021 6:24 am
by NorthEast
If it's important to you that you have full control and visibility over what topics are excluded/included in a build, then I'd suggest not to use the Content linked directly or indirectly... feature.

Even when the Content linked directly or indirectly... feature is "working", you will probably find topics being included in your output that you didn't expect. I know in my projects there's no way I could keep track of what's linked to what, and be able to predict what topics might end up being included.
There's also no easy to spot changes when topics are included or excluded, since you'd have to do file comparisons for every single output you build.

So I'd only use the Content linked directly or indirectly... feature if it's not very important what is/isn't included in a build.

But if it is important, then use conditions to control what topics are included.
When you use conditions you can clearly see (in the source) what topics will be included, and will get predictable results from one build to the next.

Re: Topics can get incorrectly included in HTML5 output

Posted: Tue Apr 20, 2021 12:33 pm
by paul_collins
It seems that people are happy to work around it and use different strategies. However I do agree with Dave and ChoccieMuffin that the belt and braces technique of using conditions on the topics/files to make the output more predictable is a good approach, particularly in large projects or where there are several authors involved.

Re: Topics can get incorrectly included in HTML5 output

Posted: Mon May 03, 2021 12:24 pm
by pattivaitaitis
I guess I have learned to work around it, as well, and fortunately, my content doesn't have information that would be terribly harmful if released by accident.

That said, I do agree that if I mark something to be excluded, it should be excluded. Period. If not having it breaks some link or causes other problems, then that should be reported as an error, just like any other broken link.

Re: Topics can get incorrectly included in HTML5 output

Posted: Mon May 03, 2021 7:40 pm
by doloremipsum
From Flare's point of view, I suspect that this is just down to the order in which it builds the output. I imagine that the builder decides which topics to include BEFORE it applies the conditions within the topics (so it doesn't have to apply conditions in every topic in the project before it decides whether or not that topic is required).

I know this doesn't help anyone, it's just interesting to think about.

Re: Topics can get incorrectly included in HTML5 output

Posted: Tue May 04, 2021 1:10 am
by paul_collins
Yes doloremipsum, that's exactly what I thought. I tried several times with Madcap to argue that their logic doesn't make sense, but they still insist that it's deliberate behavior.

Re: Topics can get incorrectly included in HTML5 output

Posted: Wed May 05, 2021 3:09 am
by NorthEast
pattivaitaitis wrote:I guess I have learned to work around it, as well, and fortunately, my content doesn't have information that would be terribly harmful if released by accident.

That said, I do agree that if I mark something to be excluded, it should be excluded. Period. If not having it breaks some link or causes other problems, then that should be reported as an error, just like any other broken link.
The conditions here were actually working as expected.

If you just set the condition on the link to the topic (not the topic itself), then only the link is excluded, and not the topic.

If you set the condition on the topic, then the topic would be excluded, and you could see broken links to the topic in the build process.

The real problem is that using the Content linked directly or indirectly... feature gives you zero visibility on what's included/excluded in your output.

Re: Topics can get incorrectly included in HTML5 output

Posted: Wed May 05, 2021 3:40 am
by paul_collins
If you set the condition on the paragraph containing the link (rather than applied directly on the link) the linked topic is still included i.e. Flare will correctly exclude the entire paragraph containing the link to "newtopic" but still includes newtopic in the output. That has to be wrong.

Re: Topics can get incorrectly included in HTML5 output

Posted: Wed May 05, 2021 7:03 am
by NorthEast
paul_collins wrote:If you set the condition on the paragraph containing the link (rather than applied directly on the link) the linked topic is still included i.e. Flare will correctly exclude the entire paragraph containing the link to "newtopic" but still includes newtopic in the output. That has to be wrong.
Yep, the conditions will work fine - but it's the Content linked directly or indirectly... feature that's not working the way you expect it to. Whatever MadCap say, it's a common problem, judging from the numerous posts I've seen in the forums.

My advice is that if it's important to control what is/isn't included in your build, then don't use the Content linked directly or indirectly... feature; instead use conditions.

Re: Topics can get incorrectly included in HTML5 output

Posted: Wed May 05, 2021 7:29 am
by paul_collins
Yes, I agree entirely with the advice to use conditions on the topics for absolute control and clarity. Even if people don't see it as a clear bug, I think it's worthwhile to alert people to the behavior as they may want to check their output to make sure they're not publishing more topics than they intended, and possibly change their strategy in light of how Flare works.