winapi - Accessing files beyond MAX_PATH in C#/.NET

Question

BACKGROUND

I need to write a tool using .NET version 2.0 at highest (using something off the shelf is not an option for this client for political, commercial, and confidentiality/trust reasons) to migrate files from one server to another over the network. The servers are file servers for local teams, and certain team folders need to be migrated to other servers to facilitate a reorganisation. The basic idea is we read each file and stream it over the network out of hours and after a few days the data will be migrated. File permissions need to be preserved. As this will take a few days (we are talking up to several gigabytes of data, for some teams), we need to iterate over the files each night and compare the modification dates and update those that have changed. The theory is that eventually the new server will have an up to date copy of the files and the users can switch over to the new server. It is of course not quite this simple, but we have a design we think should work :)

THE PROBLEM

So in theory we just open the file, stream it over the network, and write it at the other end, right? :)

Unfortunately, on the servers themselves, the file shares were created at folder paths such as:

D:DataTeam SharesDIVISIONDEPARTMENTNAME OF TEAM - COULD BE FAIRLY LONG

For each user, this path is mapped to a drive, for example it would be shared as \SERVERTEAMNAME and mapped to the T: drive.

This has caused the situation where the files as visible from the T: drive are within the MAX_PATH limitation, however when viewed locally on the server itself, they go way beyond it. We cannot access the files using the network shares as this tool needs to be generic, to run on hundreds of these servers, and there is no standard way to tell which file shares are ones we should be moving and those which are not - there is no naming convention standard even. Also, occasionally there are sub-shares of other shares and so we exceed the MAX_PATH limit twice over!

I am aware of the workaround to specify paths using the "\?" prefix, which treats the path as a UNC path and allows a theoretical maximum of 32k characters.

This workaround is implemented at the Win32 API level, the System.IO namespace is (mostly) basically just a thin wrapper around native Win32 API functions, however Microsoft have "helpfully" implemented extra (incorrect) validation before handing the call off to the API. In this case, the .NET Framework is rejecting the path because it claims that '?' is an invalid path character.

So my question is... is there a way I haven't thought of that will allow me to work around this without having to completely rewrite the almost the whole System.IO namespace, making a load of P/Invoke calls, just to remove this annoying validation?

Answer 1

The BCL team did a 3 part series on exactly why these choices were made and what the work arounds are. If you haven't read that yet I suggest you do as it's a great source of information on the subject

• http://blogs.msdn.com/bclteam/archive/2007/02/13/long-paths-in-net-part-1-of-3-kim-hamilton.aspx