Out of Memory Exception when compiling to HTML

[helen]

After weeks of everything working just fine (including today), I'm now sporadically getting an Out of Memory Exception on my CHM target. I have 4GB of RAM - surely this is plenty? I can't reproduce it everytime either, it will work 1 minute and not the next. I've seen this error cropping up on other Targets in the forum too - has anyone got to the bottom of it? Is it really memory related? I have all the latest .NET patches, XP, Flare 4.0, but I've not rebooted as it's sporadic and only just started happening. Is there something I can clear out?

[RamonS]

There are things you can clear out, such as temporary files. Two things to try check come to mind:
- delete the contents of the output folder and try again
- check the size of the page file and increase it. or even better set it so that Windows can choose which size it thinks is best. Despite the 4 GB of RAM Windows always makes use of the swap file and when it runs out of space it starts complaining.
Does it always generate the error at the same topic? If yes, maybe excluding that topic and trying again for test purposes may give some more clues. These out of memory errors not necessarily cover only RAM issues, but as far as I know also other things such as pointers.

[helen]

Cheers Ramon

I checked my virtual memory size - it's already maxed out at 4GB, so unless Flare has turned into a monster, I think the error message is indicating a different problem as you suggest.

The annoying thing (or good thing) is that it doesn't happen every time. It can happen once, I can change nothing (not empty Output folder) and then it works perfectly 30 seconds later. As far as I can tell it's completely random. It's not always generating the error on the same topic either - it's as and when it feels like it, maybe 1 in 10 compilations currently.

[RamonS]

Do you have any other applications running during the times it fails? Maybe it is not Flare that takes up all the memory, but some other process and it is just Flare showing the error. As I mentioned many many times before, the error message in Flare are totally useless. And as I did many many times before I call again on the MadCap developers to once and for all fix this. It is just bad coding showing an utter disrespect towards the customers.

[helen]

Oh yes, Outlook 2007 for one. I agree on the error messages, it's like hunting for a needle in a haystack!

[RamonS]

Yuck! Outlook 2007! I don't know if that is the cause, but I am very tempted to blame the stupid ribbon for it, hehe. I am sure there are more applications running, such as a real-time virus scanner, an indexer, etc. They may not have a GUI, but they can still mess things up.

[helen]

Oh yes there were more, I just listed the one I assume is most memory hungry.

I'll defend the ribbon, for the little I do in Word (now I have Flare) it's not too bad in all honesty.

[RamonS]

It takes too much screen space and does not cut back on complexity. I still have three or four levels of menus, just that the second level (the ribbon) is permanently expanded and has a bunch of unintelligeable icons sometimes without text. I don't think that permanently exposing the second level of menus is worth loosing about 1/3 of the screen just for the navigation. I know I can autohide the ribbon, but that negates the benefit of the easier access. Rather than waste more space I wished Microsoft would have cut out all of that bloat from Office and turned that into add-ins. That would have made the applications leaner by default and the need for the ribbon wouldn't be there - and corporations would save insane amounts of money since they don't have to retrain their employees (actively in classroom settings or passively through the self-paced training courses). It is a well meant attempt that was crappily implemented and doesn't do anything convincing enough to accept the disadvantages it brings. What is even worse, Microsoft didn't allow any alternatives such as the old style menus. This is just bad software.

[helen]

So, not a ribbon fan then?

I barely open Word these days, thankfully I have no need to. I just start furiously chanting under my breath if any of my work colleagues ask for help with Word 2003/2007 - it seems to scare them off.

[Richard Ferrell]

If you open your Index Explorer do you show any subentires that show a blank line?

[helen]

If you open your Index Explorer do you show any subentires that show a blank line?

Thank goodness for the Expand All button! No - no blank lines in the subentries of the Index Explorer.

[Andrew]

Have you ever checked on your current memory when this happens? You can do it quick 'n' dirty using Task Manager, or much more fine-grained using something like the performance monitor.

[helen]

Well I found a 'geek' to talk to and have been informed that I can whack my virtual memory paging size up to roughly 2.5 x the amount of RAM I have, so 10GB in my case. Apparently Windows is pretty rubbish at managing it's own page size - it only realises it's out when it really is out, not some time before hand!

Andrew - no I've not been that astute as it usually catches me off gaurd. I've got performance monitor so I'll go digging next week and see what I can find.

[Andrew]

Well I found a 'geek' to talk to and have been informed that I can whack my virtual memory paging size up to roughly 2.5 x the amount of RAM I have, so 10GB in my case. Apparently Windows is pretty rubbish at managing it's own page size - it only realises it's out when it really is out, not some time before hand!

That's odd; in theory, Windows should automatically expand its pagefile if it needs to (as long as it is set to let Windows manage the size).

[KevinDAmery]

Well, in theory Windows should never BSOD either....

[Andrew]

Well, in theory Windows should never BSOD either....

Yes, but we hear about it frequently -- but I've *never* heard about the pagefile not auto-expanding when it was set to let Windows manage it.

[RamonS]

Unless it runs out of drive space. I don't know in which steps Windope resizes the swap file, but if it does that in 512 MB increments then maybe it runs out of drive space after a while.

[Andrew]

That's true. Running out of drive space is one possible reason it wouldn't expand. And that's very easy to check

[KevinDAmery]

Well, in theory Windows should never BSOD either....

Yes, but we hear about it frequently -- but I've *never* heard about the pagefile not auto-expanding when it was set to let Windows manage it.

I don't think it's a case of it not auto-expanding: I suspect it's more along the lines of it not auto-expanding very efficiently, causing programs that want to write data *now* to time out and throw an error before Windows is ready to give them access to the new page file space. This is why it usually works if you try the operation again: Windows has had time to get off its duff and actually give you some page file space.

[helen]

Well, in theory Windows should never BSOD either....

Yes, but we hear about it frequently -- but I've *never* heard about the pagefile not auto-expanding when it was set to let Windows manage it.

I don't think it's a case of it not auto-expanding: I suspect it's more along the lines of it not auto-expanding very efficiently, causing programs that want to write data *now* to time out and throw an error before Windows is ready to give them access to the new page file space. This is why it usually works if you try the operation again: Windows has had time to get off its duff and actually give you some page file space.

Yes that was how it was explained to me. I'm definitely not running out of drive spac either, but I've changed my page size so we'll see if it happens again.

[NorthEast]

I see this problem fairly often in WebHelp builds using Flare v4, and I'd never seen it with the same projects in v3.

I've got 4GB RAM in my PC and I've set Windows to manage virtual memory itself, which allocates about 3.5GB. According to Task manager, Flare always seems to be using about 1.2GB when this error occurs. Even when I close the project in Flare, Flare will still use 1.2GB. By comparison, when I start Flare without a project open it'll use about 0.09 GB.

[Andrew]

That's interesting...perhaps it's running out of virtual address space (VAS) for its process. In 32-bit Windows (and for all 32-bit Windows applications compiled without the /3GB flag, which I would guess includes Flare, since very few Windows programs use that flag), the maximum per-process VAS (if I recall correctly) is 2 GB. So if the Flare process hits that limit, you might see an out-of-memory exception.

One way to check would be to monitor the VAS for each process...I'd bet there's a counter for it in perfmon, but I've never tried to monitor that before.

[helen]

I see this problem fairly often in WebHelp builds using Flare v4, and I'd never seen it with the same projects in v3.

I've got 4GB RAM in my PC and I've set Windows to manage virtual memory itself, which allocates about 3.5GB. According to Task manager, Flare always seems to be using about 1.2GB when this error occurs. Even when I close the project in Flare, Flare will still use 1.2GB. By comparison, when I start Flare without a project open it'll use about 0.09 GB.

I too have 4GB RAM. However I have set my virtual memory to 10000MB and the maximum to 10000MB and since then I've not had the error appear at all - not once. Previously I would estimate it popped up once every six builds or so of various targets. The rule of thumb I was told is 2.5 times the size of your actually memory - hence my setting.

I agree though - previously when I saw the error Flare v4 was using 1.2-1.4GB (or in the region of) and that just doesn't leave enough for the rest of Windows and other programs I have running (given that I only have 32-bit XP). I never saw the error in V3 either.

[RamonS]

Maybe the error was caused when Windows increased the swap file size. During that operations requests for virtual memory are not accepted and Flare or more likely .NET does not handle that correctly.

[helen]

Maybe the error was caused when Windows increased the swap file size. During that operations requests for virtual memory are not accepted and Flare or more likely .NET does not handle that correctly.

Could well be - all the more reason not to let Windows "manage" its Virtual Memory.