Thursday, March 23, 2006
This message is about where dlls go when they are part of “isolated assemblies.” Everyday C++ users may run into this when they move their code from a development machine to a node on a compute cluster.
1. When you copy an executable to the cluster, it now needs not only dlls but also manifest files copied with it, or we have to install them on the cluster for the users.
2. There is a common occurrence where a failure in the linker can cause an executable to fail ever to load. There is a quick fix for this.
When compiling unmanaged visual C++ applications in VS2005, you have to pay attention to how you distribute the dlls. Normally, you just put Dlls beside the exe. That isn’t working for some good reasons. I’m going to give you an outline here of where the bodies lie, trying to show you pertinent directories, tools, and project settings.
Google keywords: Isolated applications, side-by-side assemblies, winsxs
About isolated apps and assemblies
blog with details on how to do installation with isolated apps
Windows Installer XML – use xml file to create an msi.
There is a section in VS under Linker->Manifests that says you can
- embed manifest – yes/no
- allow isolation – yes/no
Visual Studio now distributes the CRT and debug CRT and versions of ATL and MFC libraries in VS
C:\Program Files\Microsoft Visual Studio 8\VC\redist\x86
C:\Program Files\Microsoft Visual Studio 8\VC\redist\Debug_NonRedist\x86
Here you find the directory Microsoft.VC80.DebugCRT\ with the files
Executables, under various conditions, are supposed to be able to find dlls in various directories if you put the manifest with them
Part II: The Common Deal-breaker: Oldnames.lib
When you compile code which links to libraries using older, deprecated methods, the linker finds those methods in an implicit library called Oldnames.lib. Because of a flaw in the linker, any time it pulls in oldnames.lib, it will also pull in a copy of the wrong CRT. That is, if you link against the debug CRT, it will pull in the release CRT, and vice versa. This wouldn’t normally cause the executable to fail to run, except that the linker includes only the correct assembly. For instance, when linking against the debug CRT, the following manifest is included in the executable, even though the linker mistakenly linked to the runtime CRT, as well.
<assembly xmlns="'urn:schemas-microsoft-com:asm.v1'" manifestversion="'1.0'">
<assemblyidentity type="'win32'" name="'Microsoft.VC80.DebugCRT'" version="'8.0.50608.0'" processorarchitecture="'x86'" publickeytoken="'1fc8b3b9a1e18e3b'">
The solution is to tell the linker to ignore the other CRT. In other words, if you link with msvcrtd.lib, you have to explicitly tell the linker to ignore msvcrt.lib, and vice versa. That makes the linker think longer about where it can find the references it needs, and it comes back around to the correct library.
Mind that Microsoft would not consider this problem a linker flaw because I have only seen it happen when linking to libraries older than my current version of Visual Studio, and linking Visual Studio 2005 projects to those of Visual Studio 2003 or Visual Studio 6 is not supported. This is likely confounded with the fact that I am linking release and debug always to a release library, which is not supported. Does any company really distribute their development libraries as MT, MD, MTd and MDd? I've only seen VRCO do this.
At this point, I’ve also created an installer for the C runtimes, as described in the blog link above, so that’s what is loading. I just run vccrt.msi, then my application runs. I used this tool called WIX to install it. It’s a bit of open source from Microsoft. The project doesn't appear to be under development any longer, but it's still working for now.
Typical deployment of isolated assemblies on a cluster node
We have already covered the typical story for isolated assemblies. If an application uses a dll which was compiled as an isolated assembly, then the application has to contain a manifest which specifies those assemblies. Visual Studio automatically embeds that manifest in the application. When you deploy that application to a new machine, Microsoft recommends that you use an installer to install that assembly as a shared assembly, meaning that the installer places the assembly in the WinSxS folder in the Windows folder, which functions like a GAC. Visual Studio even includes a vcredist.exe program to install all of the release versions of Visual Studio isolated assemblies into the WinSxS folder.
Because we are running on cluster nodes, we want to know how to deploy those isolated assemblies, instead, as private assemblies. This is usually a simple matter. You find directory for the isolated assembly, such as Microsoft.VC80.DebugCRT, in C:\Program Files\Visual Studio 8\VC\redist\Debug_NonRedist\x86. Copy the whole directory to the same directory as your executable on the cluster node, and everything should work.
By the way, Dave installed the redistributable, release version, isolated assemblies on one of the clusters. We have to figure out whether we will install all Visual Studio assemblies on the clusters for the users. The problem will still exist for other isolated assemblies, although we don’t, at this point, know of any coming from companies other than Microsoft or products other than Visual Studio.
How OpenMP requires another step to deploy
When you try the above technique for OpenMP, the executable still fails to find the assembly on the cluster node. When Visual Studio compiles your OpenMP program, it embeds a manifest file that refers to version 8.0.50608.0 of the OpenMP assembly, but the actual assembly is version is 8.0.50727.42. Why would this work on your machine? It works because your computer has a machine-level policy, sitting in %windows%\WinSxS\Policies, which states that, if any application asks for version 8.0.50608.0, it’s okay to give them 8.0.50727.42.
The version mismatch between the distributed assembly and the assembly required by the compiler is a known problem. It happens because different parts of Visual Studio are frozen at differnt times before delivery. What we have to do to make this run on the cluster is to deploy that same policy as an application configuration file.
Application configuration files for native application files are very similar to those for managed files. If your program is myprog.exe, you make a text file, with UTF-8 encoding (with or without a byte-order mark), called myprog.exe.config. In it, you redirect the program’s binding to the newer version of the debug OpenMP assembly. Note that, when you identify the DebugOpenMP assembly, you leave out the version attribute because you will specify it in the bindingRedirect node.
<assemblyidentity name="myprog.exe" processorarchitecture="x86" version="188.8.131.52" type="win32">
<assemblyIdentity type="win32" name="Microsoft.VC80.DebugOpenMP"
<bindingredirect oldversion="8.0.50608.0" newversion="8.0.50727.42">
If we had deployed the OpenMP assembly to WinSxS, so that it was a shared assembly, I think that there would have been a machine-level policy to take care of the version mismatch. As is, we had to make this file for ourselves and deploy it in the same directory as myprog.exe.
How a little bug made this hard to figure out
Yesterday morning, we had figured out that there was a version mismatch and that an application configuration file was necessary. We had copied the debug OpenMP assembly directory into the same directory as our application, but nothing worked. After five hours, we noticed that the debug OpenMP assembly directory was called Micrsoroft.VC80.DebugOpenMP. If you look in your Studio 2005 VC\Debug_NonRedist\x86 directory, you’ll find that the directory name is wrong.
It seemed like a great chance to contribute to the project, so I've created wrappers that could fold into the source tree.
- Managed C++ wrappers for VTK.
- Allow use in C#, managed C++, VB, and others.
- .NET 2.0 only, meaning Visual Studio 2005.
- Builds within the typical CMake build system for VTK.
- Includes a .NET Windows Forms Control for drag-and-drop in Visual Studio designer.
We'll have to see how successful these are. Because the CMake build system does not handle managed C++ projects, the build isn't entirely hands-free. Just before hitting Build in Visual Studio, the user has to execute a macro to convert the projects to managed code. That problem means this wrapper couldn't be built in nightly dashboard tests. I'd be willing to extend CMake to handle managed projects if that's what it takes.