Using Source Control, Ajax and PHP for contents editing.

[Eric Lachance]

Hello All!

I am going to start by writing this as a small introduction to what I will soon place here, before I actually write anything. The reason I do this is that I want to make sure that no one at MadCap is going to put on the Mad Cap (heh) and scream at me for giving people a way around the use of Contributor/X-Edit or Flare licenses. So, MadCap, if you're reading this before I actually post any code, please feel free to remove this post completely and I will understand completely.

So what am I talking about, how does this apply to you and why do you care? I'll start with a bit of background (I have to warn you at this point, I love writing, so this is and will remain a huge wall of text. If you're afraid, turn back now!). I edit the documentation for 2 different software suites plus a standalone software and a knowledgebase, for a grand total of 9 Flare Project - all of them are separately edited and published, as they currently have very little in common. This will change in our next major release as both suites will be merged together with the same base code, but will remain separate suites aimed at different clients.

Now with 9 different projects for a single guy with absolutely no technical writing experience (except the one year since I've started doing this), it's a huge endeavor to keep everything updated and error-free, especially with the Knowledge Base, which has around 1500 FAQ/Error Definitions and How To's! But I'm not alone in this huge company, and we have a full department of technical support agents that encounter issues every day and are best trained to deal with at least describing the solution. So, what initially thought is to provide a select number of people with a Contributor licence so that, if any page needed to be edited or corrected, they could simply request that I send them that page (or a couple of pages) as a Review package which they could then edit and send back to me. I would then have to verify spelling, phrasing and essentially beautify their text. The major issue with that was that if a technician without Contributor had a change to make, they would have to explain this change to whoever had it, which had to request a copy of the page from me. I would then have to wait for the edit, receive it and immediately deal with it lest it became lost in a sea of incoming emails.

So I had a brilliant (I think) idea: since all our Flare source XMLs were on Subversion and since I had some very solid understanding of both PHP and Javascript, I would build us our own little editing tool that would let anyone with the proper access (username and password, only from internal IPs) to edit a page directly from the browser, then submit that change directly without ever needing special software. The whole thing would be front-ended by jQuery and a bunch of Ajax calls, and a back-end written in PHP with SVN command line calls and database interaction. I've been working on it for about 50 hours (officially; I actually put in a little of my personal time on this) and I've nearly completed everything with the only exception of creating a new page from an existing HTML template (for example, a new knowledge base article) and I've passed through a bunch of hurdles, some of them being related simply to the fact that my PHP backend was on a different subdomain server (if you know ajax, you probably know the hassle of cross-domain ajax).

I'm all for sharing experiences and open source, and I really want to share the result of this experiment with everyone here. What I want to do is to "generiticise" (I know it's not a word) the whole thing and make it accessible, probably as a single working Flare project. This project will assume that you have PHP on the server as well as an SQL Server with the appropriate plugin for PHP on IIS. I know that may sound weird (PHP on IIS with MS SQL!) but the reason is that my PHP code is located on our Feedback Server machine. Since the machine is already publicly accessible (we plan on giving access to other satellite offices so that was a requirement) as well as a database ready to be used, it made that decision easy.

As I said from the get go, if MadCap does not want me to put this online, they have plenty of time to remove this post as it will take me at least another week to completely finish, document and package this "solution".

In the meantime, I'm interested in some feedback about the idea, specifically:
- Is PHP and SQL Server alright, or would you like to have it available in another server-side and database language (possibly, the end result would support multiple DBs)
- Would you have any use for cross-domain requests (this actually requires a proxy on the same domain/subdomain where the documentation is located!), or would this all be on the same server for you?
- Would you take this solution and adapt it to other project (not flare related) for your own use?
- If you were to really need this, would you be willing to pay for this? (Assuming that doesn't break MadCap's EULA, I don't even know if it would, I'm just asking out of curiosity!)

Thanks for sticking out through this ordeal of a wall of text

[Eric Lachance]

== Reserved for possible future use ==

[RamonS]

I think it is a neat idea, but if you are concerned about MadCap's reaction you should talk to them directly. This is a peer-to-peer forum.
Typically, PHP pairs with MySQL, which I find to be the better database overall unless you are hooked on the MSSQL specific stuff, which typically costs.
Lastly, make sure you document the source code well, provide tech specs, and add at least another 50 hours for testing and bug fixing before releasing anything. Feel free to use my now rather dusty PHP CSH module to add help to your front end (see here: http://forums.madcapsoftware.com/viewto ... =12&t=5232). Also, since you already use PHP dial down the client side scripting to an absolute minimum. The server has more processing power than some browser client. I've seen web clients that are JSed to the max. While they are slick they are also dog slow.
Uh, and at least run a few tests to make sure that someone clicking the back button does not send the whole thing into a tailspin. The icing would be to include an install script (PHP based) that collects the database connection strings and SVN user name and password and then creates the database and tables as well as writes out the ini files to keep the settings, plus add a database backup routine. Maybe that's for version 2.
And I mention it again, document and comment your code! I think that a 2:1 ratio of code to comments is necessary...unless you want other not understand what you are doing, including yourself half a year from now.

[aforsyth]

Yes - sounds like a great idea, and some wise advice in the last post. I think there are probably quite a lot of us -- developers turned doc writers -- who would be interested in working with this. Most non-developers (i.e. normal people ) would probably not understand the use or potential of your work at this stage.
I am actually at a stage where I'm considering the purchase of Contributor for some product managers, and that will probably still go ahead, since I have been a developer long enough to appreciate *not* having to support my own tools and code for other people, unless it's part of my job description / salary (doesn't fit in so easily to senior doc writer duties these days!). However, I'd be glad to run this on a simple PHP/MySQL setup for my own use - I'm guessing that your SQL is fairly generic (you haven't used tons of T-SQL procedures?), which would make it fairly easy to port to MySQL.

Regarding a commercial license, I think that would be an option in the future, but you need a good well-tested product and a good user community before this could be viable. I am generally happy to pay something for a self-contained web application if I can see a good user community pushing it forward and effective support (updates, user help, tech support) provided by the company/individual. Therefore it's usually best to start in Open Source land, as I'm sure you're well aware.

Finally, I think MadCap won't have any great problem with this tool, as long as you are not continually pushing it as a rival commercial (or viable open source) alternative in these forums. As was suggested, I think you should proactively check with them directly, but their choice of standard XML-based technologies for Flare content clearly shows that they expected (and hopefully welcome) the integration of other tools in the documentation process.
Before I chose Flare to replace our ageing Word documentation, I was originally hired to move the docs to a Confluence Wiki, which was great apart from the classic Wiki weaknesses of single sourcing (minimal re-use possible without getting hugely complicated admin/organisation, difficult to get content translated, difficult to organise successive product versions in the context of single sourcing, issues with screenshot management, etc.). When I first approached documentation writing as a change from the web development world, I was sure that a fully web-based solution could be found -- a cross between Flare and a Wiki. No such luck so far (although MindTouch have told me that this is their eventual goal in a few years' time), but it's a huge opportunity. If your tool eventually became a web-based Flare, I'm sure MadCap would be interested in an acquisition

Anyway - great idea. Looking forward to trying it out!

Alan