Importing directly from SVN repo versus local project

This forum is for all Flare issues related to using Source Control.
Post Reply
ashtontreadway
Propeller Head
Posts: 11
Joined: Tue Mar 04, 2014 10:51 am

Importing directly from SVN repo versus local project

Post by ashtontreadway »

Hi:

Are we doing something wrong here?

We have a "Library" project that contains content we share across multiple projects.

What we want is for that project, when imported and/or updated, to pull directly from the latest copy of the SVN root. That is, when the project is included or imported, to have it pull directly from the latest copies in the SVN repo.

What we're finding in practice is that the only way to get that content into projects is by having a local version of the Library project, and then importing that local version.

This is a profound disadvantage, because it forces the Flare user to:

1. Open, and update, their local copy of the Library.
2. Close the LIbrary and open their project.
3. Update the Library import in their project.

This is clumsy, and error-prone, and is in practice making our idea of shared content unusable.

My question is: are we doing something wrong? Is there some way to generate an import directly from SVN, without the intermediate step of having a local project?
Robotman
Sr. Propeller Head
Posts: 185
Joined: Sat Mar 04, 2006 3:05 am
Location: Melbourne, Australia
Contact:

Re: Importing directly from SVN repo versus local project

Post by Robotman »

What if you stored the Library project on the server and not in source control? I've seen that recommended elsewhere and it's how we're implementing our GPL project.
\m/ Gary \m/
Flare 2020 / Windows 10 64-Bit
Screaming Symphony
NorthEast
Master Propellus Maximus
Posts: 6359
Joined: Mon Mar 05, 2007 8:33 am

Re: Importing directly from SVN repo versus local project

Post by NorthEast »

You're not doing anything wrong - a Flare project import can only get files from a local or network drive (not source control).

Whilst the import will only have to be run once by one person (who then checks-in the changed files), you are right in saying it is error prone.

We follow this procedure:
1) Do a 'get latest version all' in both projects.
2) Run the project import, and import all files that are different - both files that are 'source is newer' (green) and files that are 'local is newer' (red).

This second point is really important. The import will mark files being imported as 'source' or 'local' is newer, but you shouldn't just import 'source is newer' files (green). That's because the import is just comparing the date stamp on files from both projects, and that's unreliable because the the date stamp is just when you last got the file from source control; it's not related in any way to whether it's actually the latest version of that file from source control.
ashtontreadway
Propeller Head
Posts: 11
Joined: Tue Mar 04, 2014 10:51 am

Re: Importing directly from SVN repo versus local project

Post by ashtontreadway »

Dave Lee wrote:You're not doing anything wrong - a Flare project import can only get files from a local or network drive (not source control).
I'm speechless. This isn't even bad design. This is stupid design.

Off to file an RFE, but I'm not optimistic about my chances.

Thanks for the answer.
HiyaKath
Propeller Head
Posts: 35
Joined: Thu Nov 17, 2011 12:07 pm

Re: Importing directly from SVN repo versus local project

Post by HiyaKath »

I am interested in this workflow because when recently troubleshooting an issue with MadCap support I was astounded to hear that Flare does not work properly if projects are stored on a Network drive. :shock:

When I asked how I could manage a multi-author environment support told me to use source control.

But if the information in this thread is accurate, then source control is only feasible as long as project linking is not used? Is this true or have I just not found the proper setup to support both multiple authors and linking?

Thanks,
Kath
Nita Beck
Senior Propellus Maximus
Posts: 3667
Joined: Thu Feb 02, 2006 9:57 am
Location: Pittsford, NY

Re: Importing directly from SVN repo versus local project

Post by Nita Beck »

HiyaKath wrote:But if the information in this thread is accurate, then source control is only feasible as long as project linking is not used? Is this true or have I just not found the proper setup to support both multiple authors and linking?
One most certainly can have workflows that involve global project linking and source control. My clients and I do it all the time.

In this situation, the global project is bound to source control, by itself, and each of the authors who will co-author the global project or who will need to import from it into child projects will have his or her own local copy of the global project on his or her computer.

Likewise, the child project is bound to source control, and each of the authors who will co-author it will have his or her own local copy of the child project on his or her computer.

When an author needs to pull updated items from the global project into a child project, he or she will import from the local copy of the global project into the local copy of the child project. Thereafter, the updated, imported global files in the child project will need to be checked into source control from within the child project.
Nita
Image
RETIRED, but still fond of all the Flare friends I've made. See you around now and then!
MattyQ
Sr. Propeller Head
Posts: 136
Joined: Tue Sep 30, 2014 7:10 am
Location: Roanoke, VA

Re: Importing directly from SVN repo versus local project

Post by MattyQ »

Nita Beck wrote:
HiyaKath wrote:But if the information in this thread is accurate, then source control is only feasible as long as project linking is not used? Is this true or have I just not found the proper setup to support both multiple authors and linking?
One most certainly can have workflows that involve global project linking and source control. My clients and I do it all the time.

In this situation, the global project is bound to source control, by itself, and each of the authors who will co-author the global project or who will need to import from it into child projects will have his or her own local copy of the global project on his or her computer.
Another workflow is to have a copy of your global project stored in a network location accessible to all your Flare users. Then you can either automate the updating of this global project (for example, having a script get the latest version of the global project n times a day and copies the latest to the network location) or assign a gatekeeper user who is responsible for keeping that network version up to date.

With a managed version of the global project on the network, you then don't need to trouble your writers with always getting the latest version of the global project, and (if desired) you can set the "Import when you build the project" flag on the import file and automate it completely.
HiyaKath
Propeller Head
Posts: 35
Joined: Thu Nov 17, 2011 12:07 pm

Re: Importing directly from SVN repo versus local project

Post by HiyaKath »

MattyQ wrote:Another workflow is to have a copy of your global project stored in a network location accessible to all your Flare users. Then you can either automate the updating of this global project (for example, having a script get the latest version of the global project n times a day and copies the latest to the network location) or assign a gatekeeper user who is responsible for keeping that network version up to date.

With a managed version of the global project on the network, you then don't need to trouble your writers with always getting the latest version of the global project, and (if desired) you can set the "Import when you build the project" flag on the import file and automate it completely.
This would be my preference because I'd like to set up automated scheduled builds. I guess I was thrown by MadCap Support's claim that Flare projects should not be stored on network drives, since I've never seen any indication (either in this forum or in their own documentation) of that limitation.

However, I guess what I need to keep in mind from the original question/reply is that both the global and the child projects have to already be stored on a drive (local or network) and updated before the (re)import, as Flare will not dip into SVN to import files.

Thanks for the clarification!
Kath
HiyaKath
Propeller Head
Posts: 35
Joined: Thu Nov 17, 2011 12:07 pm

Re: Importing directly from SVN repo versus local project

Post by HiyaKath »

And thanks also Nita, for reassuring me that what I'm attempting is possible!

Thanks,
Kath
Nita Beck
Senior Propellus Maximus
Posts: 3667
Joined: Thu Feb 02, 2006 9:57 am
Location: Pittsford, NY

Re: Importing directly from SVN repo versus local project

Post by Nita Beck »

I think the issue with Flare projects on network drives has to do with performance, and there might be issues too with the project being open by two people at the same time...
Nita
Image
RETIRED, but still fond of all the Flare friends I've made. See you around now and then!
MattyQ
Sr. Propeller Head
Posts: 136
Joined: Tue Sep 30, 2014 7:10 am
Location: Roanoke, VA

Re: Importing directly from SVN repo versus local project

Post by MattyQ »

Nita Beck wrote:I think the issue with Flare projects on network drives has to do with performance, and there might be issues too with the project being open by two people at the same time...
Yeah, I've definitely seen some performance issues when using Flare with a project in a network location.

In the case of the workflow I've used, I would not suggest ever editing the project directly in the network location, just copying it there manually or via a script after you've made changes locally. The network location just exists as a most current, unified point for the other projects to import files from. Because you're just copying from it, you never have issues with performance, and you don't need to worry about multiple people importing the files at the same time (there's no conflicts since it's just a copy operation).

In my case, I was the global project gatekeeper, so the actual flow was:
  • 1. Edit my local version of the global project.

    2. Commit my changes to Source Control.

    3. Copy the latest and greatest global project to the network location.

    4. Whenever one of the company projects is built, it automatically pulls the latest files from global.
In my case, for the third step, I had two scripts in play:
  • • The first script was always running, and detected when I closed the global project after editing it in Flare. It would then prompt me to copy my changes to the network location with a Yes/No dialog box. Clicking Yes would automatically copy the necessary files to network location. (By necessary, I mean excluding FileSync, Analyzer, Output, Project\Exports, and Project\Users)

    • The second script was set up as a Windows task, and ran twice a day. It would get the latest version from Source Control, and then copy the files to the network location, excluding the same folders as above.
NorthEast
Master Propellus Maximus
Posts: 6359
Joined: Mon Mar 05, 2007 8:33 am

Re: Importing directly from SVN repo versus local project

Post by NorthEast »

We import content from a 'master template' project on a network folder, which is also in source control (TFS).
I've not experienced any problems with the project being on a network folder, however it is absolutely essential that it's bound to source control as this prevents simultaneous editing by multiple users.

We also import projects that are on our local drives, which are also bound to source control.
The problem with importing from local projects is that the project import needs to be to a fixed location, but people will commonly use different drives and folders on their PC.
We got round this by setting up a new drive letter (using subst), e.g. so drive P:/ could be mapped to C:/Users/MyUser/Projects/ on one person's PC and maybe D:/Projects/ on another PC.
Then when we set up the import, we set up the project import relative to drive P:/, so it's is consistent for all users.

And to re-iterate my previous post, it is really important that you are careful about how you import and what you import.
HiyaKath
Propeller Head
Posts: 35
Joined: Thu Nov 17, 2011 12:07 pm

Re: Importing directly from SVN repo versus local project

Post by HiyaKath »

Dave Lee wrote:And to re-iterate my previous post, it is really important that you are careful about how you import and what you import.
Dave I know you mentioned above that you include both kinds of files that are indicated as "newer" because the date/time stamp is only for when they were copied from source control (and not the actual file date/time). Do you manually run the import each time?

I can't find a setting on my import file that would always bring all files in each time I generate. I'm trying to set up some automatic builds so I want Flare to copy all the files from the original import regardless of their timestamps.

Thanks,
Kath
MattyQ
Sr. Propeller Head
Posts: 136
Joined: Tue Sep 30, 2014 7:10 am
Location: Roanoke, VA

Re: Importing directly from SVN repo versus local project

Post by MattyQ »

HiyaKath wrote:I can't find a setting on my import file that would always bring all files in each time I generate. I'm trying to set up some automatic builds so I want Flare to copy all the files from the original import regardless of their timestamps.
I did some testing before I set up the workflow at my last organization and found that if you enable the option to automatically import when you generate, it brings in all files (like Dave advises). It appears to take the greedier option, rather than the interface you get when you manually import in Flare, which defaults to only the newer files. I'm interested to retest this in a new environment, but, again, it did copy all files regardless of timestamps, for us.
HiyaKath
Propeller Head
Posts: 35
Joined: Thu Nov 17, 2011 12:07 pm

Re: Importing directly from SVN repo versus local project

Post by HiyaKath »

Excellent news! Thanks for the info.

Kath
NorthEast
Master Propellus Maximus
Posts: 6359
Joined: Mon Mar 05, 2007 8:33 am

Re: Importing directly from SVN repo versus local project

Post by NorthEast »

HiyaKath wrote:Do you manually run the import each time?
Yes, we do it manually, because I couldn't see how you can have a reliable automatic build.

In order to get a reliable build, the process is:
1. Do "Get latest all" in the project you're importing from (to make sure you're actually importing the latest files).
2. Do "Get latest all" in the project you're importing to (to make sure your build project is up to date, prior to import).
3. Run the project import, and import anything both newer and older.
4. Build the project.

So if you had an automated build, it's going to omit (1), so files in the project you import from might not be the latest.
In your build project, I'm not sure what order the target would run the "automatically get latest" and the import auto-sync; so I'm not sure if (2) and (3) would happen in the right order.
MattyQ
Sr. Propeller Head
Posts: 136
Joined: Tue Sep 30, 2014 7:10 am
Location: Roanoke, VA

Re: Importing directly from SVN repo versus local project

Post by MattyQ »

Dave Lee wrote:In order to get a reliable build, the process is:
1. Do "Get latest all" in the project you're importing from (to make sure you're actually importing the latest files).
2. Do "Get latest all" in the project you're importing to (to make sure your build project is up to date, prior to import).
3. Run the project import, and import anything both newer and older.
4. Build the project.
We found that, in our case, ensuring in an automated fashion that the network version of the global files were always the latest and greatest reliably eliminated steps 1 and 3. Our writers (we had 12 working on 40 separate projects) had consistent results when they would build their projects in Flare (steps 2 and 4 still being key).

If you're worried about the integrity of the network version, you can restrict access to the network directory to Read privileges, and provide Write/Modify access only to the individuals who would be responsible for modifying the global project. You can also automate the process outside of Flare, setting up a script to first leverage the svn command line to cover steps 1 and 2 for you, and then calling madbuild to generate the output for you. (That said, I think further breakdown of that sort of automation is probably out of scope for the thread.)
NorthEast
Master Propellus Maximus
Posts: 6359
Joined: Mon Mar 05, 2007 8:33 am

Re: Importing directly from SVN repo versus local project

Post by NorthEast »

Yeah, the projects we're importing from are on a local drive rather than a network, hence the need for step (1), and why the process couldn't be automated using Flare alone.
We do step (1) manually, but it sounds like you're automating this outside of Flare.

If we put the project on the network, it'd mean the files (should) be up to date with source control, but then it also opens up the problem that you can import files that are currently being edited (saved) and not yet checked-in. (This is also an issue with local projects, for your own edits.)
Presumably that's why you don't import from the 'live' network project, but a copy that you make after you finish editing.
Post Reply