Flare's internal HTM files appearing in SharePoint search

[bobmoon]

Hi all -

We are loading our product content in a SharePoint site (Flare tri-pane web help, PDFs, HTML FAQs, etc), and our application utilizes the SharePoint search engine to let customers search across that content. Users can see results that include PDFs and web help topics that have their search term.

The problem is that some of the "internal" Flare output files (Browse_Sequence.htm, Glossary.htm, <project>_CSH.htm) are being included in some search results, but they don't work when opened directly. This is confusing to clients.

The issue is that the product name appears in the file name and meta data. When you search for the product name (a common thing), those files come up.

I know that I can add a meta tag to each file <META name="robots" content= "NOINDEX, NOFOLLOW">, and the search indexer will ignore that file. But that means editing the output HTML files for each production build. That's a pain when you have dozens of help projects.

Does anyone know of a more automated way to do this?

Thanks,
Bob

[bobmoon]

In case anyone stumbles across this, I found a solution within SharePoint.

SharePoint search has its own index crawler, and I added crawl rules that tell the indexer to ignore files and folders within the webhelp. With a few rules, I can exclude the problematic files that were showing up in search results. For example, I added rules for the crawler to ignore files using these rules:

*://*/Online%20Help/*_csh.htm
*://*/Online%20Help/*_left.htm
*://*/Online%20Help/Content/*Glossary*.htm
*://*/Online%20Help/Skin/*
*://*/Online%20Help/Content/SkinSupport/*


I looked at adding a robots.txt file to the web site, but I don't have access to load a file to the SharePoint server itself. This solution was available using the SharePoint admin UI.

Bob