Foregoing the existing skin engine and building my own

[Eric Lachance]

Hi there,

Though the current skinning engine is fairly customizable using CSS, we'd like to integrate the design a lot more with our main website, which doesn't use frames at all. Now, I know how to use Master Pages in order to give the same layout to every single one of my topics - that's easy. What's not easy is to recreate some of the functionalities the webhelp output gives us, and that we cannot ignore or remove.

The first thing is the ContextID/HelpID. Currently, using Default_CSH.html#helpID (or #page.html) works as we want it to, other than not expanding the TOC structure. However, if I am to not use the default skin at all, then it's not directly possible to recreate this feature. As a workaround I could use the default_csh anyway and have each of my topics break out of frames around it, but that would cause an unnecessary reload and delay for our users.

The second this is the Search feature, which is the single greatest thing that our documentation got by going to Flare's webhelp output (the previous documentation didn't have this at all). I know how to add a search box, that part is easy. But I don't know how I could implement displaying results in a page with the same layout, using my own CSS. I guess I could use something like branded Google Search, but I'd much prefer using the tools included with Flare (the search of which is excellent).

So, does anyone have any idea as to how I should proceed with this, or will I have to reverse-engineer the WebHelp output to achieve what I want to do?

Thanks!

[NorthEast]

I can't answer your question; but have you looked at the WebHelp Mobile output?
The WebHelp Mobile output doesn't use any frames, so it may be easier to 'hack' that output to suit your own needs.

[Eric Lachance]

Dave,

Thank you very much! That's exactly the kind of out-of-the-box thinking that saves my sanity every day

I'm sure that will work just fine!

Cheers,
Eric.

[kevinmcl]

Eric Lachance said:

I guess I could use something like branded Google Search, but I'd much prefer using the tools included with Flare (the search of which is excellent).

I'm having a hard time with that statement for my WebHelp situation.

My WebHelp goes on a CD and gets loaded on the customer's client PC (Win or Linux) or UNIX workstation (Solaris, HP-UX, AIX) hard disk. If they get it served, it's by their own local file servers, but usually, the WebHelp just lives on the CD/DVD drive or on the customer's hard drive... so no web serving involved. Absolutely no IIS, except for the few who are using our product in conjunction with their IIS implementation... and then they aren't serving our WebHelp to their customers.

I continue to get complaints about partial word searches and multi-word (phrase) searches, which either don't return results or return tons of useless results.
I try to address those, as they come up, by adding to the next release's index.... but that's not search. And people these days go to "Search" before any other kind of navigation aid.

Are most people including an SQL engine with their WebHelp? Are most of your customers OK with running that in order to get useful search with the WebHelp?

[Eric Lachance]

Kevin,

To be honest, Flare's WebHelp replaces... well, nothign. We had a custom HTML output generated by a Drupal plugin that was supposed to do books, it was crappy, slow, and didn't have search.

Already, to have something that can detect that "activate", "activated" and "activating" are the same word and will display either of those in the results is pretty amazing. I've only gotten positive feedback from our users in regards to the new search "we" implemented.

If you're putting the help files on a CD though, I'm not sure any database could really work, unless I'm wrong and there's some way to use, for example an SQLite database, without any extra drivers like SQLite ODBC, etc.

All in all, we're really happy with the search. YMMV.

[Eric Lachance]

I was able to create my own interface around the topic using a master page - that's was easy part. Each html topic is now looking like what I want it to. I'm still looking into implementing interface features like the search, which by default returns the results inside a div, dynamically - I'd like it to output in a new page, or perhaps at the very least a javascript-created window in a lightbox. That may require some fiddling around with the MadCapAll.js file, either directly or using jquery to "extend" it.

I'm still not sure about some of the details for topics, like getting their ratings, feedback, etc. I'd love to see variables such as "NumberofResultsinSearch" and "NumberofComments", but it seems everything is hardcoded and I'll probably have to figure out how to extract it from the javascript and files, somehow.

As for displaying the table of contents, I see that WebHelp builts it dynamically using an XML in the backend, which is alright when using frames - I just hope that if I want to load the TOC as a series of UL and LI tag structure for a drop-down menu, that I won't be shooting myself in the foot with performance.

I'll keep everyone apprised of my progress if there's any interest; and if someone would like to lend a hand with their own research, perhaps we can create a guide somewhere for everyone to benefit! Just let me know

[chunkee]

Eric,
How is your progress in this realm?

JC

[Eric Lachance]

Hi JC,

For the moment now much actual progress was made, partially because other priorities have been popping up (as they tend to do). I did advance in my research in regards to hijacking the search features, by locating what javascript files were involved in this. As you may know, the "MadCapAll.js" file that is present in your WebHelp output contains all the functions for everything in the output and that makes it hard to read.

However, there are "source" javascripts that are divided into different files in Flare's program files folder. These files are most likely used if you don't check the "condense javascript files" option in your WebHelp target settings. This means 2 things. A) it's easy to locate a group of functions working towards a single goal such as "search", "effects", "toc", "glossary", etc and B) if you edit the source files it will affect all your output if you want it to, which can be a good thing.

Location: C:\Program Files\MadCap Software\MadCap Flare V7\Flare.app\Resources\WebHelp\Content\SkinSupport
(assuming you have Version 7. If not just replace 7 by 6 or 6.1)

What will most likely happen in the end however is that we'll figure out how WebHelp actually uses the XML files for search (we've got a good part of that figured out already) and create a server-side PHP that does the search in multiple manuals and returns the results. We'll then hijack the search function in the WebHelp so it uses that PHP instead of doing a search directly in the XML.

Of course, this cannot be used in an "offline" environment, but that's not our goal at all so we don't care.

[Eric Lachance]

For the interested parties, here's some information on how the Search XMLs work (consider this research on the topic of hijacking the search feature)

The XMLs are divided into 2 parts. One Search.xml file (located in \Data\) contains a list of all the topics in your output, a list of "stem" words along with which "chunk" they belong in, as well as a list of all the "chunk" files generated by Flare. Here's the Search.XML file structure:

Code: Select all

<?xml version="1.0" encoding="utf-8" ?> 
<index PreMerged="false" SearchType="Stem" NGramSize="1">
 <urls>
  <Url Source="../sometopic.html" Title="Topic Title" />
  <Url Source="../othertopic.html" Title="Another Topic" />
  <!-- ... all your other topics. The "base" folder is /Content, I don't use it, thus ../ is used. -->
 </urls>
 <ents>
  <!-- These are actual random samples from my own ouptut>
  <stem n="copyright" chunk="Search_Chunk1.xml" /> 
  <stem n="inform" chunk="Search_Chunk1.xml" /> 
  <stem n="knowledg" chunk="Search_Chunk14.xml" /> 
  <stem n="insid" chunk="Search_Chunk25.xml" /> 
  <stem n="startup" chunk="Search_Chunk25.xml" /> 
  <stem n="process" chunk="Search_Chunk25.xml" /> 
 </ents>
 <chunkfiles>
  <Url Source="Search_Chunk1.xml" /> 
  <Url Source="Search_Chunk2.xml" /> 
  <!-- etc , for all chunks -->
 </chunkfiles>
</index>

I'm not exactly sure what determines what a stem is. I notice a lot of "cut" words but on the other hand the longest string is a partial URL (://www.objectiflune.com/ol/en-ca/download/downloadcent). It *seems* like random characters are cut off from the start and end, but I'm sure there's a logical reason for that.

Each Search_ChunkX.xml contains a list of stems along with actual word results for each stem and some pretty precise details:

Code: Select all

<?xml version="1.0" encoding="utf-8" ?> 
<index>
 <stem n="copyright">
  <phr n="Copyright">
   <ent r="1000" t="0" w="1" /> 
   <ent r="102" t="0" w="4" /> 
   <ent r="3" t="0" w="7" /> 
  </phr>
  <phr n="copyright">
   <ent r="4" t="148" w="779" /> 
  </phr>
 </stem>
 <stem n="inform">
  <phr n="Information">
   <ent r="1000" t="0" w="2" /> 
   <ent r="102" t="0" w="5" /> 
   <ent r="7" t="133" w="1523" /> 
   <ent r="3" t="145" w="93" /> 
   <ent r="3" t="145" w="138" /> 
   <ent r="1000" t="172" w="7" /> 
   <ent r="82" t="172" w="17" /> 
   <ent r="1000" t="234" w="2" /> 
   <ent r="82" t="234" w="8" /> 
   <ent r="5" t="235" w="41" /> 
   <ent r="5" t="240" w="52" /> 
   <ent r="5" t="240" w="58" /> 
   <ent r="66" t="350" w="136" /> 
   <ent r="66" t="384" w="80" /> 
   <ent r="66" t="384" w="98" /> 
   <ent r="6" t="385" w="1658" /> 
   <ent r="1000" t="387" w="6" /> 
   <ent r="82" t="387" w="16" /> 
   <ent r="1000" t="389" w="3" /> 
   <ent r="82" t="389" w="10" /> 
   <ent r="66" t="393" w="270" /> 
   <ent r="66" t="458" w="26" /> 
   <ent r="66" t="496" w="20" /> 
   <ent r="1000" t="497" w="1" /> 
   <ent r="82" t="497" w="8" /> 
   <ent r="66" t="501" w="15" /> 
  </phr>
 <!-- etc for all "phr" -->
 </stem>
 <!-- etc for all "stem" -->
</index>

What we've figure out:
- "stem" is a key that may refer to multiple words. In this example "inform" can refer to the words "Information", "information", "informations" and, I assume if I had these words, "informative" and "informational". Each "stem" form Search_ChunkX.xml has its own counterpart in Search.xml
- "phr" stands for "phrase" and is a full word as written plain text in your topic. n="xxx" is the actual word of course.
- "ent" stands for "entity" and is an iteration of the word in a specific topic. What gets interesting is the properties of this tag:
- "r" (I THINK) is the relevance of the word in regards to the stem. 1000 is the largest number.
- "t" is the index of the topic, as it appears in Search.xml. The index is 0-based, so "0" is the first topic on the list, etc.
- "w" is the 1-based index of the word within the topic.

So, if you have the followowing:

Code: Select all

<ent r="82" t="389" w="10" />

This means that this word is found with a relevance of 82 in the topic which is 390th (0-based, remember?) in the list form Search.XML, and it's the 10th word in the topic. I assume at this point that "r" is used to sort the results by relevance (where the position of a result is proportional to the total "r" value for all results within a topic, for all words you searched for). I've seen a lot of "1000" values for r, which makes me thing that equals an exact match including case.

More info as I get it.

[chunkee]

Thank you Eric for posting your findings and following up.