Global Project Linking reimporting not working

This forum is for all Flare issues not related to any of the other categories.
Post Reply
Msquared
Propellus Maximus
Posts: 848
Joined: Mon Aug 06, 2012 10:19 am
Location: Southampton, UK

Global Project Linking reimporting not working

Post by Msquared »

I've finally taken the plunge and decided to set up Global Project Linking for my projects. The trouble is, I can't get it to reimport changed files predictably. Any ideas?

This is where I'm at. I identified my global files long ago, put them in a separate project, created an import file for them in my ordinary projects, and got them into my projects initially via an import. But for various reasons, every time I needed to make a change to one of the global files, I broke the import link, then manually reconciled the various versions using file comparisons.

Now I've got a bit of a break between deliveries, so I thought I'd sort this properly. One of the things I wanted to do was impose a bit more structure on my global project, which meant moving a few global files around. So I did that in one of my projects where I had previously broken the import links, checked everything still built, then updated my global project with the changed and moved files. So far, so good. then it all goes wrong.

I restored the removed links, and tried a reimport. Different projects do slightly different things, but none of them are right. Some imports correctly tell me there are changes, but the files that are supposed to be updated just aren't. Some update the changed files, but don't delete the stale files (files that no longer exist at the original location in the Global project because I've moved them elsewhere), although I have that option set. The files that need to be updated or deleted are all writable, as is the global import file.

On some occasions, several files are flagged in green as newer, but some of them are automatically selected for re-import, and some are not, although they may be reimported if I explicitly tick them (or they may not, depending on the project). :-(

The only predictable import was done in a brand new test project that I set up, with a brand new import file. That works as expected. I've tried creating a new import file in an existing project, but that finds nothing at all to import/reimport even when there is - it's like Flare knows there is another import file that relates to the same files so ignores the repeat request. I've come to the conclusion there is some "gotcha" that I've missed which is causing the unpredictable behaviour. I can't trust auto-reimports as part of my builds if it's going to be this flaky.

Any ideas?
Marjorie

My goal in life is to be as good a person as my dogs already think I am.
Msquared
Propellus Maximus
Posts: 848
Joined: Mon Aug 06, 2012 10:19 am
Location: Southampton, UK

Re: Global Project Linking reimporting not working

Post by Msquared »

I've done some more investigation on a test project and I think there are some bugs here. Firstly, I'll say I'm in the UK, so my PC uses UK date formats (that is DD/MM/YYYY not MM/DD/YYYY). You'll see in a minute why I mention this.

This is what I observed.
  • The initial import is fine.
  • If you delete all your globally imported files before you reimport, it works as expected.
  • If you leave Flare to make its own mind up, as far as I can tell, files are flagged correctly in the list of files to be imported most of the time, but the flagged files are not always selected. So you need to select all the flagged files manually to ensure the import works properly.
    • Source Newer seems to be set as required but sometimes is selected for import and sometimes not. I've seen cases where there are two adjoining files in the list are both flagged "Source Newer" and one is selected and the other isn't.
    • Source Missing -where the source has been deleted or renamed - is sometimes flagged, and sometimes not detected at all, although I have the "Delete Stale Files" option selected, If flagged, I have never seen it selected - the only way this will be actioned is if I manually select it
    • New files are flagged as required, but again not always selected in the list.
Most significantly, if you look in the import file where Flare records each imported file with a timestamp, the dates are completely screwed. Sometimes I get entries like this

Code: Select all

      Source="../../Content/Resources/Images/CiscoBranding/GLOBAL CiscoTopicIcon.png"
      TimeStamp="01/01/0001 00:00:00" />
and sometimes, the date is in the UK format, like this

Code: Select all

    <Url
      Source="../../Content/Resources/Stylesheets/GLOBAL Cisco Print Styles.css"
      TimeStamp="06/08/2014 15:59:27" />
and sometimes in the US format like this

Code: Select all

    <Url
      Source="../../Content/Resources/Stylesheets/GLOBAL Cisco Print Styles.css"
      TimeStamp="08/06/2014 15:59:27" />
These last two are entries FOR THE SAME FILE in successive versions of the .flimpfl file. The actual date of last modification was the 6th August 2014, at 15:59:27. The first file (with the 01/01/01 date) was actually last modified on 16th July 2014. It's hardly surprising Flare can't make the right import decisions if the dates are wrong.

So, my working conclusion is that for UK users at least, the only safe way to do an reimport is to delete all global files from your project before importing. I'll speak to MadCap support later today and see if they can shed any light on what is going wrong here.
Marjorie

My goal in life is to be as good a person as my dogs already think I am.
NorthEast
Master Propellus Maximus
Posts: 6359
Joined: Mon Mar 05, 2007 8:33 am

Re: Global Project Linking reimporting not working

Post by NorthEast »

Msquared wrote:So, my working conclusion is that for UK users at least, the only safe way to do an reimport is to delete all global files from your project before importing. I'll speak to MadCap support later today and see if they can shed any light on what is going wrong here.
Interesting, I'd never checked the date format in the file.

I've never had to delete imported files, but I've had issues when importing from projects that are bound to source control, because Flare can indicate files as 'local is newer' (by date) which are actually out-of-date in terms of the file version in the source project. Anyway, I make a point of re-importing all files marked 'source is newer' (green) and 'local is newer' (red); i.e. any file that is different.
Msquared
Propellus Maximus
Posts: 848
Joined: Mon Aug 06, 2012 10:19 am
Location: Southampton, UK

Re: Global Project Linking reimporting not working

Post by Msquared »

Ideally, I'd like this to be able to work reliably as part of my auto-build. My experiences so far are that it won't, even if we ignore the source control date issue (which I think you'd mentioned before, and I was aware of and hence looking out for).

However, I'm really keen not to store a duplicate copy of my global files in source control with each project. I'd rather have just the one copy with the global project. Others on here seem to be OK with duplicates, but I don't really like the idea, unless you may have a need for the copy to diverge from the original, in which case you will need to store a copy. I know that's not the case for me.

So, if I don't store any global files with the projects that use them, my autobuild will always run from a fresh "get" from source control. My indications are that scenario works as intended, and can be trusted to give the correct results automatically. If not, I'll soon see build errors as the project will report missing files when it builds. But at least I know my project isn't building with out-of-date files.

That just leaves me with an issue when I'm working interactively with Flare and need to import/reimport global files. In that case, so long as I understand the issues, I can cope. I now know I need to "Select All" for a reimport, as I can't trust Flare's decisions. And if there are deleted files in the global project, and Flare doesn't give a warning about deleting files, then I need to reconcile those manually.

Here's a recent screenshot of a section of Flare's import decisions. You will see that although it correctly detected an newer source file, it didn't actually select it for me. it also correctly detected one of my missing source files, and again didn't select it for me. But there were also about 40 other missing source files from another folder that it didn't even detect. :-(
2014-08-07_11-31-12.png
Now I've looked a bit deeper into the structure and content of the .flimpfl file, I have come to the conclusion that once things go wrong (perhaps because of date issues) or if the file gets out of step with reality for other reasons (perhaps because you manage your imports manually for a bit - as I did) the global imports process is unable to get back in step with the new reality because Flare thinks the imports file contains a full and complete record of what has gone before, and it doesn't. Rather than looking at the state of the global project and the state of the importing project, then deciding what needs to be reimported or deleted, it looks at what the import file thinks the import history was.

And I think it may actually be more complicated than that. Even if I create a brand new import file, without any preconceived ideas about collected files, skipped files etc, Flare still makes decisions based on stuff it thinks it knows about my import history rather than the current contents of the file system, so I wonder whether it's caching something elsewhere too.
You do not have the required permissions to view the files attached to this post.
Marjorie

My goal in life is to be as good a person as my dogs already think I am.
NorthEast
Master Propellus Maximus
Posts: 6359
Joined: Mon Mar 05, 2007 8:33 am

Re: Global Project Linking reimporting not working

Post by NorthEast »

Msquared wrote:However, I'm really keen not to store a duplicate copy of my global files in source control with each project. I'd rather have just the one copy with the global project. Others on here seem to be OK with duplicates, but I don't really like the idea, unless you may have a need for the copy to diverge from the original, in which case you will need to store a copy. I know that's not the case for me.
Wouldn't that make the situation worse?

If you don't include imported files in source control, then you have greater potential for the imported files to be different on each individual's local drive, as it depends on when they run the import and what options they select.

If the imported files are kept in source control, then at least if one person runs the import correctly, then everyone will get the correct files.
Msquared
Propellus Maximus
Posts: 848
Joined: Mon Aug 06, 2012 10:19 am
Location: Southampton, UK

Re: Global Project Linking reimporting not working

Post by Msquared »

If the imported files are kept in source control, then at least if one person runs the import correctly, then everyone will get the correct files.
I was hoping (rather naively, perhaps) that if I could set GPL up in the way it reads as if it should work, then the latest versions of the global files would always be pulled in for every build. And there would be no need for any user to manually set up anything (with the potential for errors, as with any manual process), because Flare would handle it all automatically.

Alas, that seems to be far from the case. I've just spent over an hour on the phone to a very patient MadCap support person, and, even in a completely clean project, importing from a completely clean project, we've seen some of the issues I've been struggling with - files not being identified as needing reimport, files being identified for import but not being selected without manual intervention, stale files not being deleted when the option is set, and bizarre as it sounds, files that are successfully imported into one project not being identified as needing import in another project, as if Flare is confusing files from one project with files from another project.

Sorting out the global imports manually, checking each project carefully, then checking it all back into Perforce for each project is exactly what I've been doing anyway, using Beyond Compare. The only difference now will be that I have the extra step of first letting Flare try to decide what to copy and getting it wrong. I can afford to be indulgent if I only have a few projects to keep in step, but I can't believe that people with hundreds of projects work in this way.
Marjorie

My goal in life is to be as good a person as my dogs already think I am.
sprockets
Jr. Propeller Head
Posts: 2
Joined: Fri Jun 08, 2018 4:48 pm

Re: Global Project Linking reimporting not working

Post by sprockets »

Apologies for resurrecting an old thread, but did you ever get this working as desired? We are having the same issue with Global Project LInking as you have described:
  • Source Newer seems to be set as required but sometimes is selected for import and sometimes not. I've seen cases where there are two adjoining files in the list are both flagged "Source Newer" and one is selected and the other isn't.
    Source Missing -where the source has been deleted or renamed - is sometimes flagged, and sometimes not detected at all, although I have the "Delete Stale Files" option selected, If flagged, I have never seen it selected - the only way this will be actioned is if I manually select it
    New files are flagged as required, but again not always selected in the list.
Often, even if I manually select a file to import, it does not import, even if the source is newer. Sometimes this can be resolved by opening the associated imports file with a text editor and manually removing references the file name, but this is a tedious process.
NorthEast
Master Propellus Maximus
Posts: 6359
Joined: Mon Mar 05, 2007 8:33 am

Re: Global Project Linking reimporting not working

Post by NorthEast »

Yeah, I'm not sure why some files are not automatically selected, but it's a right pain.
I have a feeling that if you manually de-select a file once, then Flare will remember it - there is presumably some logic behind it?!

Anyway, you need to check the import file list to make sure everything you want to import is actually selected.
Sorting the import list by Status makes that a lot easier.

And if you're using a project import from a project that's in source control, then you need to also select any 'local is newer' (red) files, which are never selected by default.
Msquared
Propellus Maximus
Posts: 848
Joined: Mon Aug 06, 2012 10:19 am
Location: Southampton, UK

Re: Global Project Linking reimporting not working

Post by Msquared »

Hi Sprockets,

No, I didn't ever get GPL working as I hoped it would. But I've not used Flare very much in the last couple of years, and in that time, there have been lots of changes and new features, so I'm not really the best person to give advice about the current situation.

I do miss Flare though. Its powerful single sourcing and ease of creating good quality HTML and PDF outputs knock spots off anything else I've used in the meantime.
Marjorie

My goal in life is to be as good a person as my dogs already think I am.
sprockets
Jr. Propeller Head
Posts: 2
Joined: Fri Jun 08, 2018 4:48 pm

Re: Global Project Linking reimporting not working

Post by sprockets »

@MSquared: Thanks for responding. I do love Flare as well and hope you get the chance to return to it in the future if you wish.

@Dave: Don't quote me on this, but I think when you manually de-select a file, it ends up in the SkippedFiles list (visible when you open the import file in a text editor). I've sometimes had luck working around these problems by clearing out URLs in the import files, as well as deleting the "LastImport" time stamp.
NorthEast
Master Propellus Maximus
Posts: 6359
Joined: Mon Mar 05, 2007 8:33 am

Re: Global Project Linking reimporting not working

Post by NorthEast »

sprockets wrote:@Dave: Don't quote me on this, but I think when you manually de-select a file, it ends up in the SkippedFiles list (visible when you open the import file in a text editor). I've sometimes had luck working around these problems by clearing out URLs in the import files, as well as deleting the "LastImport" time stamp.
Cheers - clearing out all the skipped files did the trick.
Post Reply