Page 1 of 1

Any experiences of large Flare projects (1000s of topics)?

Posted: Thu May 23, 2013 9:08 am
by Msquared
When I first set up Flare, I planned to have one project for each set of distinct content, and use conditionals/variables to handle the various product and content options. I identified some files as global (for example, stylesheets, page layouts, skins, some graphics etc) and put them in a global project, and imported them into each new "document" project, as I ported each of my existing documents to Flare.

I'm now considering throwing my entire document set into one Flare project, and I wondered about people's experiences of doing that, before I burn all my boats and regret it.

I've never really got my head properly round the global content. I use Global Project Linking, but however I handle it, I seem to have multiple copies of these files, one in each project that uses them, so I spend some time comparing and updating across the various copies of my "global" content. That's been an ongoing issue, and gets bigger and more fiddly each time I port another Word document to Flare. That wouldn't be an issue if my entire document set was in one Flare project - I'd just have a Global subfolder for the common content, and a subfolder for the content for each document (and more subfolders below that where necessary).

But the thing that looks like it may tip the balance is automating the build. I'd like to link the documentation build into our software build, and I have two ways to do it:
  • Hook in a Flare build to our software build system (the Visual Studio Build tool, which allows you to specify a Flare step to build a target, or a batch target). Each step can only build targets from *one* Flare project, so if I port another document to Flare, in another project, I have to get the software guys to add a step to their build process to build the new document.

    Write my own script to invoke Madbuild once for each Flare Project, then tell the software guys to add that script as a step in the build. In this case, if I port another document to Flare, i just update my script (which I can test locally) and the extra document gets built without a change to the build system, and ithout the possibility for introducing errors.


The latter appears the better engineering solution, but means that my script has to handle things like error checking, reporting errors and adding the Flare build logs to the overall build logs. However, the former apparently integrates well enough with Flare to collect the Flare logs, and detect any Flare errors, without me doing anything extra. Unless . . .

. . . and that's where the idea for putting all my documents into one Flare project came from. I can create one batch target, to build all my documents, and tell the Visual Studio Build tool to get on with it. No fiddling around for me, and no need to get developer help each time I port another document to Flare - I can just update the Flare batch target. I will no longer have multiple copies of "global" content, as it will all be in one project. And also, it will be easier to share snippets and other common content between documents (I haven't done much of that yet, but I plan to, as I port more content to Flare - I am expecting that to involve copying lots of duplicated content around between projects too).

So, dare I do this, or will I regret it?

Re: Any experiences of large Flare projects (1000s of topics)?

Posted: Thu May 23, 2013 9:42 am
by BedfordWriter
For what it's worth, my project is up to 5000 topics and counting. I figure there will be another 1500 to 2000 before it's done. I'm building HTML5 and CHM output on a standard-issue Dell Latitude E6420 with 8G ram and Windows 7.

I haven't had any troubles until today. Just finished switching the links between function definitions to use Madcap:xrefs instead of plain "a" links. That's 3000 links between 800 function definitions. Got an out-of-memory error on my first build attempt after that. But, then it ran just fine after a re-boot. Hard to say at this point whether that was a coincidence or extra memory required by the xrefs.
I have noticed that out-of-memory errors are routine if I try to compile the entire project to a Word or PDF output, but on reflection I decided that I didn't want to do that anyway. For print output, I break the project up into logical chunks and build separate documents. Nobody wants an 80-Mb Word file!

Anyway - your mileage may vary. Get yourself a workhorse of a computer and you'll probably be fine with many thousands of topics.

Re: Any experiences of large Flare projects (1000s of topics)?

Posted: Thu May 23, 2013 9:59 am
by Msquared
Thanks for the info. I don't ever see myself putting all my topics into a single document. I would still be building multiple outputs, mainly from distinct sub-groups of content, and mainly using separate TOCs. It's more the concept of putting all my documents into a single project, since that seems the best way to get Flare to share some things nicely.

Re: Any experiences of large Flare projects (1000s of topics)?

Posted: Fri May 24, 2013 1:34 am
by NorthEast
I've worked with projects with around 2500 topics, and not experienced any major problems.

If you're going to combine the projects, I'd suggest spending a bit of time re-organising the content of each project before you do this:

* Move all topics out of the root folder, and put them inside a new folder named after for that project. Then when you combine the projects, you'll be able to easily find the topics from each project, and it will also avoid any file name conflicts. You might also do the same for Resources/images, or other folders that may have conflicting files.

* Apply a condition (named after the project) to all topics in the project. Then when you combine projects, you can easily exclude those topics from a target if you ever need to. This is also handy for doing partial builds of large outputs, when you just want a quick check and don't want to wait 15mins for a full build.


If you do things like this to compartamentalise the files from each project before you combine them, it should make it a bit easier to work with the large project afterwards.

Re: Any experiences of large Flare projects (1000s of topics)?

Posted: Fri May 24, 2013 9:38 am
by Msquared
Thanks Dave,

I'm going for it! I'd already decided to structure Content and possibly other things on a 'mini-project' basis where necessary.

However, I'll do the reorganisation of the existing content *after* I've got the new content in. I use Perforce, and moving/renaming content in Perforce is such a bind, and almost impossible to roll back successfully (that's a Perforce issue, not a Flare issue) that I don't want to mess with the existing projects until I'm really, really, really sure it's what I want.

Re: Any experiences of large Flare projects (1000s of topics)?

Posted: Wed Jun 11, 2014 5:43 am
by RobinS
Picking up an old thread...
I have been wrestling with the 'many products in one project' or 'one project per product with global project linking' conundrum.
The 'one project' solution seems the most practical, on the face of it, but we are likely to be internationalising some of our products (only one or two, initially) within the next few months. The advice for translation purposes (elsewhere on this forum) seems to be 'one project per language, per product'...which leads me to the GPL approach. Any further advice, anyone? Final decision is imminent...! (I'm dealing with six closely related products that share multiple files out of a total of 1500-2000.)
Thanks! - R.

Re: Any experiences of large Flare projects (1000s of topics)?

Posted: Wed Jun 11, 2014 6:05 am
by Msquared
I don't think you would want to put the English and all translations into a single mega-project (unless someone else knows differently!).

What I do at present is have one main English project (although I am toying with GPL and smaller projects) and one project for each translated language. So I give the translators one project, and they return one translated project per language.

That works fine, except if not all your content is translated, then you may like to put all the stuff that is translated into one project, and the rest in one or more other projects, so you can give the translators a project containing just the content you want them to translate. However, Flare 10 does provide a Export project option that allows you to export just the files required by a particular target, so if you're using Flare 10, you would be able to send them a subset of your single enormous project.

Re: Any experiences of large Flare projects (1000s of topics)?

Posted: Thu Jun 12, 2014 4:26 am
by RobinS
Marjorie - Thanks for your reply. I think I'm heading down the GPL path, with (a) a 'Global' project for all the baggage (.css, page layouts, copyright etc. - not very much in here, but reusable across every product range) (b) A 'Shared' project for all topics and images that are common to more than one product in this specific range (c) Product-specific projects for everything that is unique to one product in the range, plus imports from (a) and (b), from where I'll publish. At the moment it looks like just one of our products, in one range, will be up for translation (to at least two other languages) and I'm trying bear that in mind without letting it dictate everything!
I see from your other posts that you have looked at various options for your Flare structure - any comments on the above? Thanks, R.