performance - Why does Delphi's compilation speed degrade the longer it's open, and what can I do about it?

Question

My company has been running a large project on Delphi for more than a decade. Our codebase has been growing over the years and now stands at around 4 million lines of code. Compilation speed is becoming an issue. We've spent time weeding out unit circular referencing (a known cause of slow compilation) and examined every aspect of the setup. It gets to a point we can't improve it any further materially with what we can control.

At the moment, on a state of the art PC with 4 cores running Windows XP SP3 and Delphi 2006, start Delphi fresh and do a full build, it takes ~40 secs. Then, if we do another full build in the same Delphi session immediately, it will take 1m 40s. Do another full build again, it will get worse. So on and so forth.

(We are well aware Windows itself caches files and this has a big impact on the compilation speed. The above figures are based on that the files are cached. We set up such scenario by getting Delphi to compile the project once, terminating it and then starting a new Delphi session. So while 40 secs doesn't see to be slow, it's only because the files are cached by Windows. And we do this in order to have an apple-to-apple comparison.)

What puzzles us is why the compilation speed gets worse. (We have observed in the past the slow down was worse if the project had a lot of unit circular referencing.) If we terminate Delphi and start a new session, the compilation time will get back to 40 secs. An even more interesting thing we've observed is, we can achieve the same speed "improvement" by clicking the "Cancel" button to abort the compilation and then do the full build right away. Compilation time will get back to 40secs too.

It appears to us Delphi's own cache of unit dependency isn't as efficient as building it from scratch and it is getting worse over time. And it also appears the Cancel button somehow clears this cache. What we are thinking is, if we can tap into the Delphi IDE subsystem that does this clearing, we can always maintain the compilation speed at its peak performance. But we don't know how.

Does anyone know what we can do?

We are still using Delphi 2006 as we haven't yet found a feasible way to port our large project to Unicode. I read on forums that the latest Delphi XE exhibits similar compilation speed issue with unit circular referencing. Anyone knows if Delphi XE has addressed the problem?

p.s. We are also aware that splitting the project into runtime packages can reduce compilation time. But for deployment and administrative reasons, we try to avoid using runtime packages.

question from:https://stackoverflow.com/questions/6577993/why-does-delphis-compilation-speed-degrade-the-longer-its-open-and-what-can-i

Answer 1

If you build your application, here are some tricks to speed up the process:

• Erase all *.dcu before the build (del *.dcu /s);
• Run a good defragmenter on your corresponding hard drive;
• Put most of your source files in the same directory, and try to leave the IDE and Project library paths as short as possible, with the most used entries at first;
• Install DelphiSpeedUp.

Delphi 2007 should compile faster than Delphi 2006.

Delphi 2009/2010/XE would probably be slower: from user experiment, the implementation of generics and new RTTI made the compilation process more complex, and the actual implementation was found out to be slower e.g. than with Delphi 2007.

Update:

Did you try enabling the ProjectClearUnitCacheItem hidden menu entry?

I've this entry enabled either by the CnPack, either by DDevExtension (I don't know which one do this, probably the later). This could be used to clear the internal unit cache.