GPL, External resources and source control. Who does what?

[Msquared]

Hi there,

I must get my head round Global Project Linking (or Global Imports, as Flare refers to them) and External Resources. From day one of my Flare usage I identified (largely correctly) which files would be global/shared and prefixed them all with GLOBAL. But after a few hiccups with GPL, and after discovering it was really global project COPYING, I put all attempts to use it on the back burner and up till now, I've been manually keeping all my GLOBAL* files in sync using Beyond Compare until I decide what to do. I've got so much shared content now that I can't really put it off any longer.

I think I understand how to set up both GPL and External Resources, and think I've grasped some of the pros and cons, but before I go for my proposed scheme, I'd be interested to know what others do, and what others think.

Some shared files rarely change, and when they do, I'm happy to change them in the context of a master project and reimport in each project that uses them (for example, page layouts, copyright text snippets, standard company graphics). I propose to use GPL for these, and select the Auto-Reimport before Generate Output option. I propose to exclude all project local copies from source control and just put the files from the master project in source control, since the project local copies are just that, copies.

Is this how others use GPL?

Then I have some common files that I think aren't so suitable for GPL. These files may change more frequently, and will usually need to be changed in the context of a working project, so I can see that the changes are correct and/or work on the changes as I go (for example, stylesheets, the dictionary file, shared topics or snippets). It's a real pain to work on these if they are global imports (that's the main reason I abandoned attempts to set this up earlier) as you either need to break the import link, or move to the GPL project to make your changes, taking any necessary working context, topics etc with you. So I thought I'd set these up as External Resources, since I can make changes in the project or in the external files as required, then synchronise later. But that poses more questions.

It seems that although two-way synchronisation is supported, it is completely manual, and I must always remember to synchronise my local project changes back to the copy in the external file when I'm ready, and also then manually synchronise the other projects to the copy in the external file. Is that so? Sounds tedious. Is there any way to specify an automatic downward sync (I don't mind having to manually sync up to the external file, but having to manually sync many projects to the latest external files could be painful).

And it seems that the synching will only work provided I remember to select the option to set up the sync mapping (which I note is stored in the .flprj file!) when I added the external file in the first place. If not, then I've effectively got a once-only copy, and no obvious way to see that it's an (unmapped) copy of an external file?

Finally, do those of you that use external files include the local copies in source control? It seems that there is a far greater chance of losing changes than with GPL, since there is nothing to stop you changing the local copy, and nothing to remind you to sync the changes back to the external files and check in the changes there.

Who used External Resources, and how to you make them work for you?

[ChoccieMuffin]

Quick response to the GPL point - you've described exactly how we do it here.

1. Our Globals project contains anything that needs to be used by lots of child projects, and anything that comes in from the Globals project is not included in source control.
For the Content folder that includes the following:
* master pages
* page layouts
* global_data variable set (each child project has its own local variable set if needed)
* common content files, either topics or snippets (such as copyright information, company addresses etc)
* images that are used throughout (mainly company logos and branding "pretties" to go on covers but also elsewhere)
* stylesheets and table stylesheets
and for the Project folder:
* TOCs (for example a Front Matter TOC that includes cover page, TOC proxy topic, copyright and other boilerplate stuff)
* Condition tag sets containing conditions that would be commonly used throughout all projects
* Global glossaries (I remember you mentioning you have several flavours of glossary in your stuff)

TIP: Particularly if you link to SVN manually, in your Globals project don't put content at the same level as your content usually goes in your child projects, or you could end up confused about which files are the result of an import and which files are new ones you've added. So if you put topics in your child projects directly in the Contents folder, in your Globals project put them in a Content\Globals folder. It's not such an issue in the Projects folder because those bits are less likely to change often and there are fewer files to get mixed up about.

2. Each child project contains one (or more) import files. These ARE included in source control, but to be a pain they do change every time you import the Globals project. (Don't know how that affects the automatic SVN thing, we do it manually, so when checking in we just exclude those files from check-in.)

3. Stylesheets. Yes it's a bit of a pain, but it helps to keep the stylesheet in the Globals project. If I need to play around with changes in a child project, when I try to change a file that has come in as part of an import Flare points this out. I can still make the change locally (so I can see the result) and then I copy the changed CSS file back into my Globals project once I'm happy with it, then check the Globals project back in as normal.

4. As you say, keep the "Automatically import" check box selected, so your globals stuff comes in when you build.

We have some child projects that are also used in a couple of bigger manuals, so we do the same with the child projects.

[NorthEast]

I use project imports (GPL) quite a lot, mainly to import a standard project 'template' - stylesheets, page layouts, master pages, etc.

One of the key things I like about project imports is the flexibility in what you do and don't import; which I tend to control by importing only files marked with a certain condition. I don't think you can have that degree of control in External Resources - you can just map folders and hand-pick individual files, but that requires manual setting-up by the user and scope for error. By comparison, I can make project imports foolproof for anyone to set up, as I can just supply a project import file (flimpfl) with preconfigured settings.

Project imports are a one-way street though; so if you really need to update and sync files in two locations, then External Resources is the only option for these files in Flare. I would just make sure that if multiple people are using the projects, that you all know how this is set up and used.

I propose to exclude all project local copies from source control and just put the files from the master project in source control, since the project local copies are just that, copies.

Why would you want to exclude local copies from source control? (Also, how do you actually do it with Flare's binding?)
Just I can think of a few reasons why it's a bad idea, but not why it'd be good to.

[Msquared]

Thanks for the replies guys. More food for thought. Perhaps I should leave external files well alone then. I was hopeful they may be more flexible than my experiments today and your comments have indicated. Two way sync is a wonderful thing where it's required, but I can see the beginnings of a real sync mess if I have to handle it all manually - I may as well stick with Beyond Compare.

Jenny (aka Choccie)

I have indeed identified almost exactly the same common files as you (and in fact identified them as I set my first Flare project up). But you say

in your Globals project don't put content at the same level as your content usually goes in your child projects,

I've prefixed all my global files with GLOBAL (again when I set my first Flare project up). Is there any reason why that's not enough to distinguish them?

Dave,
You ask

Why would you want to exclude local copies from source control? (Also, how do you actually do it with Flare's binding?)

I don't like duplicated information - good practice from my software development days. If two things really are meant to be identical, and should never be different, then they should be stored as a single item in a single place and referenced as required. It seems that's the rule that Flare's GPL forces on you, by not letting you edit the local copy. In this case, if you have a local copy checked in, and when you sync with the latest global version, you forget to check in the new local copy, someone else could get that project from your source control system not do the sync, then build it without errors, thinking they have the latest versions. Whereas if you don't have any local copies of your GPL files, the build will fail unless the import has been done, I think? I may be wrong - this is all theory so far . . .

How do you exclude files from bound projects? In Perforce at any rate, you create an "ignore" file of a specified name. This file contains the list of files you don't want to check into Perforce, so mine for example, already includes
Analyzer\*.*
Output\*.*
Project\Reports\*.*
Project\Users\*.*
SourceControl\*.*
I was just proposing to add GLOBAL*.* Most source control systems will have similar functionality.

[NorthEast]

In this case, if you have a local copy checked in, and when you sync with the latest global version, you forget to check in the new local copy, someone else could get that project from your source control system not do the sync, then build it without errors, thinking they have the latest versions.

I don't really follow that scenario. Getting the latest from source control is not the same as a re-import, they're not related, so it isn't a guarantee that everything is up-to-date. No errors would be reported about the import either, whether or not source control is used.

If you're relying on people doing a manual re-import, then your potential for problems are actually worse if you don't use source control.
If the files are in source control, it means only one person has to remember to re-import and check-in. If they forget to check-in, at least the version in source control is a base-line of the file version.
If the files aren't in source control, then everyone can potentially have different local versions of the files to each other, as it will depend entirely on when they last ran the re-import. Therefore, there's a lot more potential for building an out-of-date target if it's not in source control.

Something also worth bearing in mind, is how you re-import; because it works on the basis of the file dates of the local and source files, not the file versions of the source project. We found that it was quite easy to be in a situation where Flare indicates the local files are newer (by date), but are actually out-of-date in terms of the file version in the source project. Therefore, it matters a lot what options people use when they re-import; and if the files aren't in source control, then people may re-import using different options and get different results.

[Msquared]

Gosh. What a minefield!

Something also worth bearing in mind, is how you re-import; because it works on the basis of the file dates of the local and source files, not the file versions of the source project. We found that it was quite easy to be in a situation where Flare indicates the local files are newer (by date), but are actually out-of-date in terms of the file version in the source project. Therefore, it matters a lot what options people use when they re-import; and if the files aren't in source control, then people may re-import using different options and get different results.

That worries me, and I'll come back to that in a minute. But first . . .

Getting the latest from source control is not the same as a re-import, they're not related, so it isn't a guarantee that everything is up-to-date. No errors would be reported about the import either, whether or not source control is used.

Yes, I know. But whether you have anything in source control to get will affect whether there is any chance of potentially getting something different from source control and from a re-import. If you don't have any copies of global files in source control, then for a clean build, the content must be reimported (or there will be build errors) so you will always know which versions of global files are going to be included in the build, because there is only one version to choose from.

That's why I would NOT want to put a duplicate copy in source control. If the intention is that the (product) build always uses the global version, then that is what should happen. If you want to use a local copy while working on changes, then you should have to do something explicit to make this happen, and then not be tempted to check in these changes by mistake. You could do this by editing your local copy of the GPL file (which is outside of source control, so won't be offered for check in), and by temporarily clearing the "Auto reimport before generating output" option in the import file, and not checking in that change either. Otherwise, as you say "the potential for problems is actually worse" - what happens in a production build won't be what you were seeing when you were working on your content locally, and crucially, may not be what you expect to happen either.

But if there are no local copies of your GPL files in source control, should you inadvertently check in a modified import file with "Auto reimport before generating output" cleared, or the link to the imported version broken (so you could edit the local copy of a GPL file), a clean build from the content in source control will fail because some or all of the GPL files will be missing.

Coming back to your other point about file date being significant for a reimport, you seem to be saying if I made some change to a local copy of a global file, then reimported, specifically to get the original copy of the global file back, Flare would decide that there was nothing to reimport as the copy in the global project wasn't newer (in this case, it's actually older)? Scary!

So, I'm still thinking that only safe build is a clean build, direct from source control. And the only safe clean build is one where there are no pre-existing copies in your project of any global files, since then Flare must reimport them all. Although of course, while working locally, you can build however you like - here I'm concerned with building deliverables in a predictable and repeatable manner.

The corollary appears to be that the safest, least error-prone implementation for working with GPL files is to always make changes directly to the version of the file in the global project then reimport them into your working project.

Unless of course, someone can convince me otherwise? I'm very willing to be convinced, since the process I've described is a pain for editing stylesheets, which are the one kind of file that I always seem to be tweaking in the content of a specific project, and the above is rather tedious.

[ChoccieMuffin]

Marjorie, re stylesheets, does point 3 in my earlier post do the trick?

[NorthEast]

If the global files aren't kept in source control, then the very first time you get the 'clean' project and build it, it will ensure that you have to run the project import and get the latest global files.
But the second time you open the project, you'll now have a copy of the global files in your local project, so from that point onwards your project isn't 'clean', and you still have the potential for the local global files to be out-of-date (and actually to a greater degree than using source control).
So I don't see how that works, unless your workflow is that you have to manually delete all the local global files every single time you open the project.

If the imported global files are in source control, yes someone could break the link, edit the file and check it in - but that's not really going to happen by accident. It would also be rectified the next time anyone does a re-import, gets the latest files, and checks them in.

Anyway, the issue I mentioned about the import is a real one - it's caught us out. As a rule now, if a project import reports local files as being newer (in red), we make sure that we re-import those too; in addition to files where the source is newer (in green).

[NorthEast]

Marjorie, re stylesheets, does point 3 in my earlier post do the trick?

The structure that I use for stylesheets is to have separate global/local stylesheets in the project.

The topics/targets link to a single main stylesheet (e.g. main.css), which itself just links to other stylesheets.
So main.css might just contain:

Code: Select all

@import url('global.css');
@import url('project.css');

Where:
- global.css is a stylesheet imported from a global project, that I use in all my projects (this will not frequently change).
- project.css is a stylesheet in the local project, with any styles used in this project only.

[Msquared]

What a great discussion with some thought-provoking comments. Thanks guys!

Hi Jenny. Yes, sort of - thanks. I'm still playing with this, so I may be wrong, but I think in order to edit the local copy of the imported file, I have to explicitly break the reimport link in the actual import file (which worries me somewhat, as then I have to remember to reinstate it and/or remember not to check in the change, or this file won't be reimported again).

It's a shame that the import file can't import itself! Or perhaps it can? Perhaps I need another import file, just to import this one? Not sure what order the imports would occur in though - I'd need the import file that imports my main global imports file to be run first. I wonder . . . That would be cool!

Hi Dave. Thanks - I would do that where necessary, except I'm making changes that I would want to port to my main stylesheet eventually, but which are being made initially for a specific project.

For example, I'm currently defining a whole load of drop-down styles, because I'm working on something that is hoping to use them heavily. But I'd like those styles to be available to all projects now that I've got them. I'm a sole author, so things like stylesheets get developed incrementally, as and when I have the need, then reused heavily once I get the hang of them.

[ChoccieMuffin]

No, you don't have to break the import link, but you do get reminded when you try to edit the copy in the child project that it's a linked file, and also when you try to save it. (But if you work in Notepad++ of course you don't get the reminder.) The big thing is remembering to copy it to your Globals project before you build your project with the automatic imports option still on, or if you turn the option off you to check it you have to remember to turn that back on again once you've copied the stylesheet back into your Globals. I've done both a couple of times so with the first oops I lost my changes, but I hadn't completely ripped it to shreds so it was a minor annoyance more than a massive problem. And with the second oops the build failed.

[Nita Beck]

Interesting discussion which (I admit) I've only skimmed. (I've been kinda silent on the forums lately 'cause I'm working multiple gigs with pressing deadlines.)

Regarding global vs. local stylesheets, my clients and I do exactly as Dave describes. Each local project has its own stylesheet that is linked to the global stylesheet. If we need to invent some new styles, we tend to invent them locally, working out all the kinks. Once satisfied, we then transfer those new local styles into the global stylesheet and delete them from the local stylesheet.

Regarding GPL and source control: My philosophy is to put everything that constitutes a local project, including any global stuff imported into to, into source control, as I want to be able to "go back in time" at any point. (I'm about to use Subversion terminology, as that's the source control I'm familiar with.)

Consider this scenario: Say we've done a release and so we create a tag in Subversion that announces that the local project at this point in time is for V1.0 of our product. Now we start working toward V2.0. Months go by, perhaps many months. Changes are being made to the local project but also likely to the global project (like maybe even a whole new branding that is being slated for public release a year from now!).

Now suddenly we have a request to output documentation for an interim product release, V1.1. We start in Flare by importing a fresh copy of the project from the V1.0 tag (well, we might do branching here, but I don't want to make this scenario more obscure), not from the current point where we happen to be in our V2.0 development. That V1.0 copy will have in it the imported global items as they existed months ago, which is exactly what we want. We only want to update local content with the V1.1 changes. We purposefully don't want to reimport from our current working copy of the global project.

I hope this makes sense. Marjorie, I hear you re your desire to have there be only one instance of a global something, but I think you have to give that up, or perhaps think of the "one instance" a little differently, as in, there's only one instance of a global something that is maintained.

[Msquared]

Hi Nita!

But if you tagged the global content with the V1 tag too, you would get exactly what you wanted when you extracted all the content associated with that tag. That's the way that we would work. Then, when the V1 project is built, it auto-imports from the V1 global project and all is good.

I'd feel quite uncomfortable about having anything that contributed to the final output (whether code or documentation) that wasn't tagged with that version. In fact, we use Perforce streams not tags to manage the versions (we have dozens of them!), but, as you say, the principle is the same.

The potential problem I can see with local copies in your project is this (bear in mind some of this is theory to me at present so I may just have misunderstood something):

• You build your V1 project (perhaps via an automated build process).
• The latest versions of the global files are pulled into your project, overwrite the local copies and are used to build the output.
• These imported files are NOT checked into source control as the versions used for that build - your build is automated and hands-off.
• The build is repeatable, but depends on the current version of the files in the global project, not the copies in the local project as they were overwritten during the build.
• When you rebuild the V1 version months later you say you will use the local sources. But they may not be the ones used originally

All this assumes you have your project set up to auto-import from the latest version of the global project when it builds. My (even) longer reply to Dave earlier today originally enumerated more options, depending on whether you wanted:

• one item to be the master, with second item as an exact duplicate which is updated immediately and automatically the master changes
• one item to be the master, with the second item as an duplicate, but changes to the master do not need to be reflected immediately in the duplicate, and can be applied at some convenient later date

I'd be scared not to use the first option (which requires an auto-import to work as expected). I agree that if you must use the second option, you would need a local copy in source control, as the two versions of the files may well be out of step. It would also need a manual import when you are ready, as an automated imports would overwrite your preferred local copies for the build. But I see several possible problems with this, because . . .

• If you have to deliberately holding off taking copies of global content, then the content isn't truly global. That's fine, except then I'd question whether GPL is the correct mechanism. When you do eventually re-import from that import file, you will get new copies of everything in that import file, which may be more than you expected or wanted if you really are in a scenario where not all changes can be taken on immediately.
• If the global content is used across multiple projects, and manually re-imported into each, you have to remember to manually reimport into every project at some point, not just the project you are currently working on. Otherwise, you could inadvertently find yourself producing a series of documents, for example, with some old-style page layouts, and some new-style page layouts. From a software engineering point of view, this is a *very bad thing* because the global project has to know about everything that uses it for updates to it to be propagated as expected. The global project shouldn't care about where its artefacts are used - the importing projects should go get the latest versions when they need to use them.
• If you have to specify a (manual and hence potentially error-prone) process to keep all projects in step that says that every time you start work on a project, you have to remember to import the latest global files and check them in first, then how is this different (or better) than never keeping local copies, and always automatically re-importing on build? I'm wary of anything that needs you to "remember" some manual step.
• I'd say that holding off the global import should be the exception, not the norm, so again, why not set it up to re-import automatically, and temporarily break the auto-import while you are working on something that isn't ready for it?
• If you are going to reimport for every build, then why confuse things with extra copies of files in source control, which may or may not be the ones actually used for the build?

I've convinced myself even more that my original conclusion is correct. I'm thinking that the only safe way to use GPL is with auto-import as the default setting, and no duplicated local copies in source control. But then, I'm an ex-software engineer, and I know how easy it is to screw up when you have a large number of interconnected little bits.

[Nita Beck]

<wave>Hi Marjorie!</wave>

But if you tagged the global content with the V1 tag too, you would get exactly what you wanted when you extracted all the content associated with that tag. That's the way that we would work. Then, when the V1 project is built, it auto-imports from the V1 global project and all is good.

No, we do not tag the global project as some specific version/tag, ever. It is a Flare project that has nothing in it that is product-specific so never needs to be tied to any particular product's version number. Besides, for which product at version 1 would we tag it for???? Our global project supports 30-40 local projects.

Also, as a side note 'cause I realize that you and I are using different source control platforms with different terms, a "tag" in Subversion is, unfortunately, not a label, a bit of meta data as one might think. Actually, a "tag" in Subversion is a stored COPY of a project at the point in time when the "tag" was created. So when I need to reproduce a Flare project from a Subversion "tag," I'm really saying, "go grab all of the files from this place in Subversion". It'll grab the .flprj file, the Content folder, and Project folder. Those folders will contain both local files and imported global files.

You build your V1 project (perhaps via an automated build process).

And perhaps that's a big difference in our workflows. We do not do automated builds.

When you rebuild the V1 version months later you say you will use the local sources. But they may not be the ones used originally.

In my scenario, they will be the ones used originally, no question about it. That's why I will have acquired a working copy of the Flare project from that tag in Subversion. It is IDENTICAL to the project as it existed whenever that tag was created.

All this assumes you have your project set up to auto-import from the latest version of the global project when it builds.

And we do not. We do imports from the global project only on demand. Besides, in my scenario, if all I'm trying to do is make a minor .dot revision to a project, I don't even need to re-import from the global project. It already has all the imported global items it in, 'cause those were put in Subversion as part of the local project.

If the global content is used across multiple projects, and manually re-imported into each, you have to remember to manually reimport into every project at some point, not just the project you are currently working on.

Part of our workflow is, prior to build, we re-import from the global project. We may or may not find that something needs to be re-imported, as our global project does not often change. Also, we don't worry about re-importing into each and every local project each and every time the global project changes. Rather, we re-import to the local projects as needed. We don't have all those 30-40 local projects in an update cycle at the same time. For example, we have one product that is at the end of its life and its Help system gets updated maybe twice a year. When we start a batch of updates, we re-import from the global project and away we go. When we're ready to do a build, we do another re-import, just in case anything else changed, and then we build.

A last word: Anyone reading this thread needs to be very clear about the distinction between importing from a global Flare project into a local Flare project and creating a Flare project by importing from source control. I fear that folks are conflating these separate processes. I'm not saying that you are, Marjorie, but I do think that folks need to distinguish between them.

Sorry... a little more...

If you are going to reimport for every build, then why confuse things with extra copies of files in source control, which may or may not be the ones actually used for the build?

I really don't understand this question...

[Msquared]

Thanks for the detailed reply Nita,

I love this forum - it's almost as good as having a team of folks to kick ideas around with. As a sole author, I do miss the informal group problem-solving sessions from my development days.

Sorry, you are quite correct. I was a bit sloppy with terminology - all references to "importing" are indeed related to global importing of globally linked files, not importing projects from source control (well off my radar - I tried it once in my early days, and it didn't work for Perforce projects so I just get initial copies of my projects using the Perforce client then open it in Flare).

I can see that your "use case" is indeed the second of my two options for using GPL - you don't necessarily want changes in the master (global) project to be applied to all projects that use it immediately, so you don't always want to automatically get (or import) the latest copies from the global project before you build. Hence you do need local copies of the particular version of each global file that is used in that project.

But I do want changes to the master (global) project to be applied to every project that uses it, with as little manual intervention as possible. The best way for me to do that is to let Flare handle the automatic import from the global project when a target is built. And in this case the local copies in source control are redundant, as they will be overwritten with the automatically imported versions from the global project when a target is built. So my gut feeling is to avoid any scope for confusion or error and not add them to source control in the local projects. That's the mechanism I'm about to implement, I think.

I now have my docs auto-built as part of the software build (at last, hurrah!), so I need to always make sure that everything is consistent and ready to go with the latest content - that fact will go part of the way to explaining our different scenarios. The other difference is that we produce a single product (with three different variants, each with different release cycles, multiple components and a number of documents). The software, like the documentation, is built from a single source. Each product release will include a full set of documents for that product, generated from multiple Flare projects, and they will always need to use the same global files.

I think your tags are very like our streams. We create a new stream shortly before we ship a product release. Similar to your tag, this is a copy of all the code and documentation sources at that point. We then stabilise the content in that stream (while on-going development continues in the main stream). When the content is stable, we have our release, and we create a release stream from the stabilised stream. We then use the stabilised stream (or a new stream created from it) for subsequent patch releases for that product variant and version. The difference between us is that my global project will be part of this steam, so I will always have access to the global files at the state they were in when the release stream for that release of that product variant was made. So long as I know the stream where any doc patches need to be applied, like you, I will know I have everything required to rebuild exactly what was shipped.

If I ever get to the point where we have multiple products with widely differing release cycles, and perhaps other differences, then I suppose I may need two levels of global project, a company-wide one for content that applies for all products, and a local one for content that only applies to a single product. In that case, presumably I could use two different global import files, with different import mechanisms. But again, I suspect I will have everything I need in a product stream provided it is set up to include the correct components from the Perforce depot.

I'm happier now that I understand why you have chosen the option you did, and why I think I need to make some different choices. I'd be very wary of doing something different from what folks like you and Dave do without understanding exactly why my needs are different. Jenny seems to do it more my way, but if I remember correctly, her projects are also built automatically as part of a software build.

I really don't understand this question...

That's probably because it doesn't relate to the workflow you are using. I was simply saying that if you have your project set up to auto-import from the latest version of the global project when it builds (which I realise you don't, but I will), then the local copies of the global files don't need to be under source control because whatever is stored in source control will never be required. It will always be overwritten with the newly-imported version if it is not present, or if it is present and different. So my question was why bother maintaining files in source control that won't ever be used, and may potentially get out of step with the ones that are actually used at build time, and hence cause confusion for future maintainers?

Related to this, Dave also makes an interesting point earlier on in this thread about "newer" not being based on file version, but on file date when Flare decides whether an import is needed. I've not yet experimented with this, so I may have misunderstood, but I can imagine various scenarios involving rollbacks (where a file is reverted to a previous version to back out a change, but in Perforce it's actually implemented as a new version containing the older content, on top of the version being rolled back) or merges (my developers are always merging changes into streams, and sometimes include unnecessary doc files by mistake, usually because they choose the wrong merge options - almost always no actual harm is done, but sometimes they do end up adding a new version of an existing file that is identical to the previous version). Now, if the affected file is the local copy (which is necessary for you, but unnecessary for me), the presence of this apparently newer file may stop a global import that is really required because the global file has the more up-to-date content but an older file date. And once a merge is done by mistake, I can't even fix the date issue by a rollback, as that will just add another even newer file version! The safest option seems therefore to avoid the situation, by not having unnecessary local copies.

Thanks for helping me debate and understand this.

[lauraj]

I'm reactivating this thread because I've been using external resources for quite awhile now and have just discovered a new "Gotcha" which, I think, means I won't be using them anymore. This looks like a great set of people to kick it around with.

I never used GPL; I'm not sure why. I investigated it and found it didn't suit my needs, but I don't remember the specifics. I use TFS, but my Flare projects are not bound to TFS (early on, the binding didn't work very well, and later I was a lone author and didn't bother). So I check files in and out manually. My .chm files are built automatically every night, based on whatever source I've checked in, as part of the software build.

When external resources were introduced (Flare 9?), they seemed to solve a problem for me. I am working on a software product which (for fairly good reasons ) has about 15 Flare projects associated with it. They all should share some resources, like a copyright snippet and a CSS, and there are a number of topics, snippets, and images that are shared among several of the projects.

I set up external resources in one directory, and subdirectories called ExternalResourceCopies in all the projects, so I could keep track of what came from where. I checked this all into TFS -- the copies MUST be checked in or the build can't find them and fails. I realized early on that a disadvantage is the need to manually sync the resources, which means I STILL need to periodically open all fifteen projects and do things to them - I'd hoped to avoid that, but whatever The external resource concept was still proving helpful to me for a long time.

Fast forward through a LONG project - we haven't shipped this documentation yet, although I put in the external resources more than a year ago. We're getting close and just branched the code base and ... oh, snap.

All the external resource mappings (visible in the .flprj files) use absolute paths. That means if I sync, the doc on the branch will be grabbing the external resources off the Trunk - NOT what I want!! I hand-edited the .flprj to change the paths to relative paths, but now when I synchronize, Flare just tells me the external files are "missing".

Point the first: If anyone has a fix or workaround for me, GREAT! Tell me quick!
Point the second: I think the only way to maintain the integrity of my branch documentation now is to delete all the external resource mappings and treat the copies as the "real" source. That's a shame, because there's a complex topic in there that will be changing before we release, but I think I have to do this. In all fifteen projects.
Point the third: I probably never should have used the external resources feature. GPL might have suited me better; I'll investigate it next time I have time.

[Msquared]

GPL has exactly the same problem - it stores both the relative AND absolute file names of each linked file.

Fortunately I realised very early on in my Flare journey that absolute file names were used, and we use branches too, so we had the same problem. I have set up all my projects so that the branch root is mapped to a logical drive P:. I only ever open the project from the P-mapped drive, which means that all absolute references always start "P:\Docs\Source\Flare\", whichever branch I'm working on.

That's the solution I'd advise. I have a batch script to map the branch to P: that I run as soon as I switch to a branch, and our autobuild runs the same script before it starts building documents. It does make for some very fancy footwork in our source control system (Perforce) as we have many branches, and I often need to work on several concurrently. I need workspaces mapped to the logical P: drive to work on projects, but ordinary workspaces to get the files in the first place so I can run the mapping. You also need to be careful that you have the right P: mapping before you start work!

If you opt for this solution, you will need to go through and do a global search and replace for all your absolute references and change them to the new absolute path using the new mapped drive. Once you've done that, so long as you map the right branch to the logical drive, you should be good.

I've been hovering between GPL and external files for a while now, and have just started implementing all my shared files as GPL. I'd like to say it works well, but in all honesty, it doesn't. The reimport doesn't work reliably, and doesn't always select all changed files, so you still need to manually sync each project. You also need to have duplicate copies in each project that uses the files, which seems to negate the reason for using global files in the first place. And because the reimport doesn't work reliably, the copies need to be checked into source control too or you may not get the correct versions on an auto-build. I also find the one-way synch a real nuisance, which is why I'm still considering external files. But they seem to have serious limitations too, like the fact that you need a copy in your project before you can use them for most things you'd want to use them for, and I seem to remember that I discovered they had to be synched one at a time, although I may have remembered that wrong.

[lauraj]

Thanks for the answer! That's not a bad workaround - I hate to force that mapping step on our build scripts, but it is probably feasible. I need to consider whether the shared files are worth the dance, honestly - there's not such a big difference between these manual sync processes (external resources or GPL) and just copying common files around. I won't be popular if I try to impose that on the branch build in the final days of the project, at any rate, so I've just changed all the references in all the .flprj's to the branch path and I have a while to decide what I'll do on Trunk. (I did the search & replace in Notepad++... that was very easy.)

You can sync your whole set of external resources at once (you don't have to do it one at a time). If that helps. I'm disappointed with MadCap in this area; they usually seem to understand the demands of industrial software (like repeatable builds), but these features just don't seem designed for our use case at all.

[Msquared]

I'm a bit disappointed too with the limitations and general bugginess of both GPL and External files, and I would agree that MadCap don't seem to understand the requirements here at all. GPL Sync is so buggy it needs be done manually, and External Files sync is only manual, although it did seem to be a bit less buggy.

But I'm a bit concerned by the statement in the Flare help for External Files that the "paths of these files are written to the registry so they will be available for all your Flare projects". I don't want anything at all, ever, to be stored in the registry if my project needs it to build. I want it all to be to be stored somewhere with the project so it is available for anyone who wants to build the project, on any machine with Flare installed.

I'm just having a little play with External Resources now, and it seems they only work for files that live below /Contents. Is that correct? If so, does no-one else have common conditions, variables etc?

[lauraj]

Well, ironically, the External Resources info is NOT needed to build, because all the synchronization happens manually on your dev machine. So the build only cares about the "project copies" of the files, which look like regular project files. I hadn't noticed the mention of registry storage and agree that it's suspect, though. What if another engineer works on these projects at some point? Seriously, this feature is starting to remind me of something Innovasys would implement.

I would like to use central resources of some sort for variables and conditions, yes. I haven't tried that - I think I meant to look into it once I got my external topics, stylesheets, and snippets working to my satisfaction, and that hasn't happened, so...

[Msquared]

So, in a project environment with external files and several authors, someone makes some changes to a global file and wants to propagate it to every project that uses that file. They will have no way of knowing what other projects have used those external files. And the changes won't be picked up by the other projects unless an author who worked on that project thinks to open that project and synchronise it, and they are still working on the same PC that they set up the external files mapping on.

The help says "The external resources feature is ideal for shared files that you expect to change over time (e.g., logo images, PDFs, stylesheets), as opposed to, say, a template file that is simply copied into your project and changed only in that project." If I've understood correctly, "ideal" isn't quite the word I'd use.

I'm hoping I've misunderstood . . . someone tell me that's the case, please.

[lauraj]

I'm not sure about the "same PC" caveat. I mean, the actual file mappings are stored in the .flprj file, so I don't know what the registry entries are used for. Maybe it's nothing important... I hope...

[Msquared]

After a bit of experimentation, these seem to be the limitations of Flare's External Files feature.

You can only map whole folders to a location in or below /Contents. if you want to put an external file somewhere else, for example, if it's a Variables file which needs to be below /Project, you can only map individual files not folders (although you can select multiple files).

If you map files rather than folders, and select the Sync option, there is no icon, either in the External Files pane or the Contents Explorer/Project Organiser panes to show that that file is synched. So if you forget to select the sync option when you set up the mapping, and don't spot this, you may be blissfully unaware that you have not linked your project copy of the file to the intended external file.

The external file to project file mapping is local to you on the PC you set it up on. If someone else works on your project or if you work on the project on another PC, you will either need to know there was a mapping, then manually set up exactly the same mapping again, or you can continue working, blissfully unaware that a mapping ever existed.

I think I may report the first two as bugs and see what MadCap says.

[lauraj]

WOW. Thanks for the report, Marjorie. The limitation to one PC means that I will, indeed, be eliminating this construct from my projects.
(And I've just checked with a fellow writer and demonstrated that you're correct about that. He cannot see "my" mappings.)

[Msquared]

CAVEAT! I may be misrepresenting Flare's capabilities here.

I'm getting the bit about being limited to one PC from the fact that MadCap's own help says the info is stored in the registry, and from my own observations yesterday while doing some limited experiments.

I have not yet confirmed this limitation, simply inferred it. I will be contacting MadCap today to see if my understanding is correct.

What I did notice was that if I opened another project, my external files window containing the files external files available for use had persisted from the previous project. Perhaps that's what they mean by the saving in the registry bit? Perhaps the external files used by a project are saved with the project (hence there should be no limitation on who can access the external files mapping for that project) but also, once you've located the external files you need to use, they are saved for you for all projects? Let's hope that's it.