MadCap Help system in Clear Case...?

[Peter Breeden]

Hi folks:

I need to put my MadCap Help system source files into Clear Case and I have a few questions before I begin.

1) What is the best way to do this given that I work daily with the source Help files?
2) Has anybody had any bad experiences working with their Help system when the source is controlled by Clear Case?
3) I would need to check out all of my source files each day from Clear Case and then check them in at the end of the day. Would this break any links or become tedious?
4) Will I be able to build my Help system without any restrictions from Clear Case when the files are checked out? Will the new source files be easy to check in when they are generated?

My concern is with the number of files in the Help system. There are so many files and I am wondering how this might work in the Clear Case environment.

Many thanks.

Cheers,
Peter

[mattbnh]

Hi Peter:

First some opinionated answers to your questions.
1) What is the best way to do this given that I work daily with the source Help files?
Answered below, but what really matters is whether it is more important to preserve work or track changes. For docs, I believe preserving is more important, while with code, tracking changes is more important.

2) Has anybody had any bad experiences working with their Help system when the source is controlled by Clear Case?
We have had build problems trying to build help and PDFs directly on checked out files in place.

3) I would need to check out all of my source files each day from Clear Case and then check them in at the end of the day. Would this break any links or become tedious?
It would not usually break things, but we feel it is inefficient with little practical payback.

4) Will I be able to build my Help system without any restrictions from Clear Case when the files are checked out? Will the new source files be easy to check in when they are generated?
Kind of and No, respectively, if you work on your projects directly in Clearcase.
We believe the work involved to daily check in and check out files is wasted. What is most important is to avoid lost work, while preserving actual versions of source that goes with a given version of product. Backup preserves workr environment.

We use Clearcase for source control, primarily to back up versioned source, but we do not use it as a daily mechanism for differentiating versions. What I mean by this is that writers do not work directly in Clearcase, and we do not overlay successive Versioned source - we store Versioned source in separate folders (version 1 in a separate folder than version 2).

Also our data store (Versioned Object Base - VOB) is totally separate from Engineering.

(If your data store is integrated with and versioned with the actual software, our method likely won't work for you. You may have no choice.)

We have found that we have more problems working directly in Clearcase (building help and PDFs remotely from inside Clearcase has caused problems) than by copying source out and working locally.

Typically, we copy out the previous published version of a source project to a local work area, and we establish a new folder in Clearcase for the version we are working on.
We use Frame for books and Flare for help currently.

During a project we periodically check in a version of our Frame files or Flare project, and at the end of the project we check-in the final version.
Our work areas are backed up nightly, so we cannot lose more a a day's work if a hard drive crashes or another disaster occurs.

What we do with our Flare projects to simplify check in and check out is that we zip the entire project and check in just one file. We do run the risk of a corrupted zip file, but like I said we have daily backups and multiple versions checked in so the risk is lessened (I won't say minimized). We have never had data loss on a Flare project.

We separate versions that we check in rather than overlaying version 2 projects on top of version 1.
We do this because pulling prior versions of projects from Clearcase is more difficult if you have them stacked on top of each other. It is easier and less confusing to have a folder that says Version 1 and contains only increments of version 1, and a separate folder that contains only version 2.

This method leaves a bigger footprint for sure, but disk space is cheap and plentiful in our neighborhood.
Part of the rationale is that Clearcase (at least as implemented by our Release Engineering) can't effectively diff binary files, and diffing doc source to find spelling and punctuation differences is a waste of time. Source control that diffs ascii program files makes sense. Diffing doc files doesn't.

I am sure that what I am saying is anathema to a source control professional, advocate, or zealot. It will never make best practices. But it is simple and effective, and we have not had any data loss. For us the bottom line is that because it is simpler, our writers, none of whom consider source control a religious calling, make fewer mistakes than occurred when we were checking stuff in file-by-file.

I hope this helps (and only tars slightly what little reputation I have).

[Peter Breeden]

Hi Matt:

This is great feedback. Thank you very much for all your efforts and detailed answers. I have passed on your post to the Developers and Quality Assurance teams to assess. Based on your answers to my questions, it looks like my concerns regarding bringing the Help system into Clear Case were justified. I am glad we did not just jump into the Help migration to Clear Case.

I might have more questions next week. If I do, I'll post them here and hopefully you will check back from time to time.

Many thanks and have a good weekend!

Cheers,
Peter

[mattbnh]

One Postscript to this.

In our environment, our books and help are not directly built during the nightly builds of our software products.

Instead, the versions we build are grabbed from Clearcase by a Release Engineering script and put in the correct location in the install kit, which installs them to the location to which they are linked. We are producing chms and pdfs.

This "grab" is separate from our source control source location (where the zips are). We have an area for the source and a folder for the build chms and PDFs for Release Engineering to include in the product.

This may need to differ in your environment (if you are producing web help that is integrated into a web product, for example) and our method may not be appropriate.

In my case, I have one webhelp system. What I do with that is send a zip of the completed help to the developer. He unpacks it and places it in with his code and checks that into Clearcase. I then zip the entire Flare project and put it in Clearcase in our Tech Pubs VOB.

[Peter Breeden]

So, we are following your recommendation by zipping the Help system and uploading that file to Clear Case on a rugular schedule.

Many thanks for your help!

Cheers,
Peter

[kkelleher]

I'm planning to commit my help project to SVN, and was curious about what exactly you experts out there would recommend that I check in. I was thinking:
1 - FM source files that are being imported into the Flare project
2 - The entire help project, zipped up as suggested by Matt
3 - The contents of the folder where I'm publishing, also zipped

Now, I think 3 might be somewhat redundant with the output folder that will be in the project ZIP, but it does ensure (in practice) that I have the exact set of files we've distributing with our product.

I think I'm just looking for a sanity check here, so any comments or advice would be appreciated. Am I missing anything? Is this the simplest approach?

Thanks,

Kristen

[kkelleher]

I was going ahead with checking this in, but I noticed that the archive created from my project was nearly 70 MB. It's not something I want to deal with through svn. So I poked around the project folder, and noticed two sizable elements that I don't think need to be checked in (since they can be recreated if nesc.):
Output - This contained several help systems I had built long ago that are no longer relevant. I just deleted the targets that aren't useful.
Analyzer - This contained a 100+ MB SQL database that I don't think I need in source control.

I'm wondering what happens if I re-create my project file structure without the Analyzer folder. Will Flare re-create the database and etc.?

Any thoughts?

Thanks,

Kristen

[LTinker68]

You don't need the Output folder or the Analyzer folder in source control.

The Output folder itself is retained (unless you delete it manually yourself), but the contents of a specific subfolder are always deleted when that target is built. So you might see subfolders for MyWebHelp, MyPDF, MyWordOutput, etc. If you build the MyPDF target then the contents of that folder are deleted before the build. So the contents of the Output folder are constantly being blown out and rebuilt. Plus, you can build an output at any time, so you don't lose anything if you don't put the Output in source control.

Likewise, the content of the Analyzer folder changes every time it runs a scan. And if you delete the Analyzer folder in a project, Flare will recreate it automatically the next time you open that project in Flare. So it also can be left out of source control.

[kkelleher]

Thanks for the information, Lisa -- very helpful as always!

-Kristen