Configuration Data is such a Pain

I have finally got my computer to my new house in Bangalore. I dont have access to the internet, so I will essentially by offline. I am yet to install SharpDevelop and .NET Framework SDK v1.1. Hopefully, I should be able to get it done today. I had some time today morning but I was too busy playing NFS Hot Pursuit 2 and Quake 3. (No real souls to frag. Only bots. )

Currently one of our installation packages does something that it should not be doing. It is handling configuration data for several applications. Clearly this created several issues during upgrades. Now, for the next schedule major release, we (the packaging team) decided that we would let the applications handle the configuration information. This was duly presented as a proposal to other "application development" teams in a meeting. The people were a little hot under the collar during the meeting. Other development groups suspected that we were doing this to brush off our responsibility. I am sure that other setup developers would also have faced such similar predicament.

I was recently browsing through the InstallShield Community forums and found that mosth setup developers were invariably including configuration data as a part of their installation. I even saw one post where a setup developer had used a VBScript custom action with the FileSystemObject and tried to replace certain place holder texts in the installed configuration file. I am sure that many of us, who have burnt their fingers with VBScript and FSO, would agree with me. Logically speaking, there is nothing wrong with this approach. But we have to realize that we are introducing a element that can go wrong. Having deferred VBScript custom actions with FSO is a lot of fun while debugging.

My recommended approach would be to have a small configurator utility, which would accept command line parameters and perform the same tasks. This serves two purposes.

1. The configuration data is effectively taken out of the MSI. You can avoid slicing and dicing the CustomActionData property to get the desired values. There is one element less in the project that can go wrong.
   
2. The user need not run the installation, if he has specified incorrect configuration. He has the option of configuring the software after installing it. While installing on locked down environments, the setup author has to ensure that the permissions table is authored such that the current user has no problems writing on the the specified resources. This should'nt be a problem for registry keys or folders as their permission tokens can always be altered by the installer.
   

Finally, Riko is back. It seems that he is going to submit his tallow source to the community. I can't wait to get my hands on it.

My 2 cents

If people were following the the WIX-Users Mailing list, they would find that it has been a little hot recently. The first instance (as far as my knowledge goes) was with Michael creating a validation tool for the WIX Toolset called 'smoke'. I personally found it very useful as it could easily be included as a part of the build process. The output is in a structured XML format that makes it easier for us to crunch and build reports. But, some of the members in the group did not share the same level of enthusiasm. Michael seemed to be put off by that and included a disturbing update to his blog entry. Rob immediately acknowledged it and posted a blog entry about being thick skinned.

Recently, history repeated with Marc Bogers starting to ask about the ICE 33 warnings as Tallow populates the Registry table instead of the Registry Tables group. Firstly, Tallow is only a helper utility and is anything but perfect. It is useful but does not exactly produce usable WIX code. Heck, it doesn't even write to a file. Rob had said that ICE33 warnings can be ignored as it created some "strange behavior". More precisely, the resiliency associated with advertisement would prompt the end user for the media when Windows Installer detects some inconsistencies with the CLSID registration. Michael dismissed the argument that it arose from sloppy authoring and had nothing to do with MSI itself. Follow Michael's blog entry on COM registration for more information.

As we all know, Rob does not agree on a lot of things like inserting GUIDs while generating output with Tallow, using the Registry Tables Group and so on. I am sure that he has good reasons behind them. I am also sure that many people (including me), have modified Tallow to suit their requirements. Some people are using Riko's Tallow that does write to the Regsitry table's group using the Class and TypeLib elements. (On a totally different note, Riko, when are you going to release that code of yours?) I have a version of Tallow that creates components with GUIDs and names them appropriately. I am pretty sure that I am aware of the consequences and I have written code to persist the component information for future reference.

This freedom is the inherent beauty of open-source software. People can use their own ingenious imagination to tailor the tool to best suit their requirements. Of course, some of our opinions differ and we need a healthy debate to analyze these opinions and make the best of it. We should refrain from flaming and should try to embrace new ideas. Rob may be the BDFL for the WIX Toolset but I am sure that he would like the WIX Toolset to be successful and that helps developers create better installers without breaking much sweat. I tried hard not to write about this topic but after reading Michael's blog and comments with certain conspiracy theories, I wanted to voice my opinion too. I am sorry if I have hurt anybody's feelings.

Using an InstallScript application as the setup launcher

This, I guess, is my first InstallShield specific post. MSI is good in a lot of things but not when dealing with more than one package in a single transaction. MSI engine is designed to be run as a single instance and cannot be used effectively to install more than one package. Nested installations are possible but often lead to undesired consequences. Having a setup chainer is good but it involves writing code and handling reboots. So, for all who are using InstallShield DevStudio and up can benefit by using the InstallScript project as their setup launcher and monitor. We can effectively "script" a launcher application and have complete control over the installation process using the available libraries.

Why InstallScript....? I would not usually recommend InstallScript for any of the custom actions in the project, as the MSI-Scripting bridge is a little weak and has certain limitations. However, InstallScript libraries would cut down most of the code that you would have to write with a brand new C++ application. Also the scripting engine supports the silent mode. So you do not have to code separately for that either. But on the flip side, you would have an overhead of 1 Meg for the InstallScript engine. But this is okay considering the fresh set of bugs that you would introduce with the first version of the launcher application. For those not concerned about silent mode, you can check out the free open source setup launcher (dotNetInstaller).

The trick here is to use the InstallScript's program....endprogram syntax instead of the event based model. This makes sure that you do not register the application and does not call unknown events in between. Good old block scripting. <smile/> This also means that you need to take care of exceptions. If you are a setup developer from the good old InstallShield Professional 5.x days, you would immediately be at home with this kind of approach. So a simple script to just display a message box would be as below.

#include "ifx.h" //The standard header
//Our program block starts here
program
 MessageBox("Hello World!",INFORMATION);
endprogram

InstallScript engine just executes the statments present between the program and endprogram statements. As you can see, this approach gives you the flexibility of a complete scripting language with libraries custom made for creating installations. InstallScript projects are known for their flexibility but miss on the other aspects of MSI such as Elevated Privileges, Install on Demand, Source Resiliency and Self-healing.

Spying on Registry Entries

I am probably letting out the best kept secret of installation industry. I always used to wonder and have now discovered how most of the installation tools spy on registry entries that are created during COM registration or similar processes without actually affecting the build system. The spying program creates temporary registry keys for each of the registry hives HKCR, HKLM, HKCU, HKU and it maps the registry hives to these temporary registry keys. It then triggers the registration function which creates registry entries withing the registry keys specified instead of including it in the hives. I came across this revelation while I was wading through the source code for Tallow in the WIX toolset.

The core of this spying exercise relies on functions exposed by the advapi32.dll. The actual hive to key mapping is performed by the RegOverridePredefKey() function. The handle to the registry key is passed by using the RegCreateKeyEx() or the RegOpenKeyEx() function. After the mapping is done, you can invoke the DllRegisterServer() function after loading the library using the LoadLibrary() function. This mapping would be active for all the registry entries created by that particular process. So out of process registration for exe files may not directly work with this method. For the sake of simplicity, I am going to extract COM Interop settings from a given assembly. So let us write a simple console app in C# to do this. This app would perform the mapping and write the registry entries to a REG file and wipe out the key after the file is written.

Extracting registration information from a DLL file is similar but involve the importing other functions from Kernel32.dll. So I am giving that a raincheck now. You can download the Wix toolset's source package if you are interested. To start off with, let us write a class with static members to import the functions from advapi32.dll and create wrappers for them. Ensure that you have System.Runtime.InteropServices namespace included.

    class OverRideRegistry
    {
        //We first declare some stuff required. The are defined in winreg.h and windows.h
        public static readonly UIntPtr HkeyClassesRoot = (UIntPtr)0x80000000;
        public static readonly UIntPtr HkeyCurrentUser = (UIntPtr)0x80000001;
        public static readonly UIntPtr HkeyLocalMachine = (UIntPtr)0x80000002;
        public static readonly UIntPtr HkeyUsers = (UIntPtr)0x80000003;

        public const uint Delete = 0x00010000;
        public const uint ReadOnly = 0x00020000;
        public const uint WriteDac = 0x00040000;
        public const uint WriteOwner = 0x00080000;
        public const uint Synchronize = 0x00100000;
        public const uint StandardRightsRequired = 0x000F0000;
        public const uint StandardRightsAll = 0x001F0000;

        public const uint GenericRead = 0x80000000;
        public const uint GenericWrite = 0x40000000;
        public const uint GenericExecute = 0x20000000;
        public const uint GenericAll = 0x10000000;

        #region The Interop Import Stuff
        //we now import the functions exposed by advapi32.dll
        //Use RegCreateKeyEx to get handle to the openedKey
        [DllImport("advapi32.dll", EntryPoint="RegCreateKeyExW", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
        internal static extern int RegCreateKey(UIntPtr key, string subkey, uint reserved, string className, uint options, uint desiredSam, uint securityAttributes, out IntPtr openedKey, out uint disposition);

        //This does the actual hive to key mapping
        [DllImport("advapi32.dll", EntryPoint="RegOverridePredefKey", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
        internal static extern int RegOverridePredefKey(UIntPtr key, IntPtr newKey);

        //Like good programmers, we release our handles.
        [DllImport("advapi32.dll", EntryPoint="RegCloseKey", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
        internal static extern int RegCloseKey(IntPtr key);
        //Our interops are done :-)
        #endregion

        //Now we actually write wrapper functions to use the imported functions
        //Wrapper for creating Registry Keys
        internal static IntPtr OpenRegistryKey(UIntPtr key, string path)
        {
            IntPtr newKey = IntPtr.Zero;
            uint disposition = 0;
            uint security = StandardRightsAll | GenericRead | GenericWrite | GenericExecute | GenericAll;
            int error = RegCreateKey(key, path, 0, null, 0, security, 0, out newKey, out disposition);
            return newKey;
        }
        //Wrapper for the mapping
        internal static void OverrideRegistryKey(UIntPtr key, IntPtr newKey)
        {
            int error = RegOverridePredefKey(key, newKey);
        }
        //Wrapper for freeing the handle
        internal static void CloseRegistryKey(IntPtr key)
        {
            int error = RegCloseKey(key);
        }
    }

Now that we have the helper class with the static methods, we can define our main Console App class, which would use the functions in the helper class to register the assembly and steal the registry entries. Let us define the functions that Map and Map the registry hives. I had initialized the RegLocation to a string "Software\\Vagmi\\RegInterop\\". So the extracted registry entries would be put in the HKLM hive within the specified path. Also not that I have mapped HKLM as the final mapping. Else all keys that are subsequently created would be created within our registry key.

        void MapRegHives()
        {
            MapRegHive(OverRideRegistry.HkeyClassesRoot,this.RegLocation+"HKCR");
            MapRegHive(OverRideRegistry.HkeyCurrentUser,this.RegLocation+"HKCU");
            MapRegHive(OverRideRegistry.HkeyUsers,this.RegLocation+"HKU");
            MapRegHive(OverRideRegistry.HkeyLocalMachine,this.RegLocation+"HKLM");
        }
        void MapRegHive(UIntPtr key, string location)
        {
            Console.WriteLine("Mapping " + key + "  to " + location);
            IntPtr createdKey=IntPtr.Zero;
            try
            {
                createdKey=OverRideRegistry.OpenRegistryKey(OverRideRegistry.HkeyLocalMachine,location);
                OverRideRegistry.OverrideRegistryKey(key,createdKey);
            }
            catch(Exception e)
            {
                Console.WriteLine("Caught exception: " + e.Message);
            }
            finally
            {
                //close key like responsible programmers
                OverRideRegistry.CloseRegistryKey(createdKey);
            }
        }
        //Functions to unmap registry hives.
        void UnMapRegHives()
        {
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyClassesRoot,IntPtr.Zero);
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyCurrentUser,IntPtr.Zero);
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyLocalMachine,IntPtr.Zero);
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyUsers,IntPtr.Zero);
        }

I then parse two command line arguments one for the DLL and another for the REG file to export. I then pass these as constructors to my class, which calls these functions.

        public RegSpyCOMInterop(string AssemblyPath, string RegFile)
        {
            try
            {
                Assembly a=Assembly.LoadFrom(AssemblyPath);
                RegistrationServices regServices=new RegistrationServices();
                //map hives to registry keys
                    MapRegHives();
                    //Register assembly for COM interop
                    regServices.RegisterAssembly(a,AssemblyRegistrationFlags.SetCodeBase);
                    //Unmap hives
                    UnMapRegHives();
                WriteToRegFile(RegFile);
            }
            catch(Exception e)
            {
                Console.WriteLine("Caught Exception : " + e.Message);
            }
        }

The WriteToRegFile() function launches regedit and exports the hive that contains our keys. It then uses simple text replacement to change the values of the exported file to make the REG file functional.

        public void WriteToRegFile(string regfile)
        {
            RegistryKey r=Registry.LocalMachine.OpenSubKey(RegLocation,false);
            PathToDelete=r.Name + "\\";
            Process pr=new Process();
            pr.StartInfo.FileName="regedit";
            pr.StartInfo.Arguments=" /e " + regfile + " " + r.Name;
            pr.Start();
            pr.WaitForExit();
            string line;
            StreamReader reader=new StreamReader(regfile);
            StreamWriter writer=new StreamWriter("TempFile.txt");
            while((line=reader.ReadLine())!=null)
            {
                writer.WriteLine(processRegistryName(line));
            }
            reader.Close();
            writer.Close();
            File.Copy("TempFile.txt",regfile,true);
            File.Delete("TempFile.txt");
            r.Flush();
            r.Close();
        }

        public String processRegistryName(string regname)
        {
            string newRegName="";
            newRegName=regname.Replace(PathToDelete,"");
            newRegName=newRegName.Replace("[" + PathToDelete.Substring(0,PathToDelete.Length-1) + "]","");
            newRegName=newRegName.Replace("HKCR","HKEY_CLASS_ROOT");
            newRegName=newRegName.Replace("HKCU","HKEY_CURRENT_USER");
            newRegName=newRegName.Replace("HKLM","HKEY_LOCAL_MACHINE");
            newRegName=newRegName.Replace("HKU","HKEY_USERS");
            return newRegName;
        }

There you have it. A command line utility to extract COM Interop Registry entries. Hope this answered some of your queries regarding registration without affecting the target system. You can find the entire source code at http://www.geekswithblogs.net/vagmi.mudumbai/articles/16581.aspx.

Microsoft's Whitepaper on Patch Sequencing

While browsing, I came across this excellent whitepaper on MSI 3.0 Patch Sequencing.

http://www.microsoft.com/downloads/details.aspx?FamilyId=AD7AC91E-2493-4549-AE6F-BF5E007C12A3&displaylang=en

Using WIX Toolset for Distributed Development

When I talk to my fellow developers and other installation developers about the WIX Toolset, they do not fully understand the implications of the WIX Toolset in its own right but map its features to their favorite installation development tools. The WIX Toolset mostly ensures that you follow the best practices while authoring packages. The biggest advantage of the WIX Toolset over other applications is the support for distributed development. I have raved and ranted about this in my previous blog entries but I would substantiate it with actual WXS code this time. For starters, WXS file is an XML file that has to follow the schema as specified in the wix.xsd file in the 'doc' folder of your distribution. Rob has an excellent demo on distributed development with the WIX toolset. But the quality of video is very poor and the sample files are not yet available for us to tweak around with. So I have come up with my own example.

Let us assume the following scenario. There are group of installation and application developers in the organization working on a setup project. Let us allocate roles (purely hypothetical. I am not a manager <grin/>) for our convenience. Let us assume that the application developers take care of maintaining a catalog of components and resources that go into the product. The second developer takes care of assigning the appropriate components into the respective merge modules or features in a product. Let another developer be responsible for creating UIs. Additionally, each application developer creates his own fragments of files by hand or by using a simple tool. Let us assume that we have to package three files.

1. Notepad.exe


2. ReadMe.txt


3. Calc.exe

Notepad.exe and Readme.txt are a logical unit and hence we would put them in a single component called Notepad.exe and set Notepad.exe as the key file of the component. We would also have Readme.txt as a companion file for Notepad.exe such that the versioning logic of Readme.txt is controlled by the versioning logic of Notepad.exe. Calc.exe is a standalone application that does not have anyother files. So here we go. The application developer codes his file named Dev1.wxs. Pretty simple isn't it?

<!--Dev1.wxs-->
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
  <Fragment>
    <DirectoryRef Id="INSTALLDIR">
      <Component Id="Notepad.exe" Guid="12CC008F-89A6-422d-868B-B066606FFD99" DiskId="1">
        <File Id="Notepad.exe" Name="Notepad.exe" LongName="Notepad.exe" KeyPath="yes" src="$(env.BUILDPATH)\Notepad.exe">
            <Shortcut Id="LaunchNotepad" Advertise="no" Description="Launches Notepad" Directory="DesktopFolder" Name="Notepad" LongName="Launch Notepad" WorkingDirectory="INSTALLDIR" Icon="Notepad.ico"/>
        </File>
        <File Id="Readme.txt" Name="Readme.txt" LongName="Readme.txt" CompanionFile="Notepad.exe" src="$(env.BUILDPATH)\Readme.txt"/>
      </Component>
    </DirectoryRef>
    <Icon Id="Notepad.ico" src="$(env.BUILDPATH)\Notepad.ico"/>
  </Fragment>
</Wix>

Now the second application developer creates his file dev2.wxs.

 

<!--Dev2.wxs-->
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
  <Fragment>
    <DirectoryRef Id="INSTALLDIR">
      <Component Id="Calc.exe" Guid="2C5ECB67-E585-4301-BAF4-5380FE6C26AB" DiskId="1">
        <File Id="Calc.exe" Name="Calc.exe" LongName="Calc.exe" KeyPath="yes" src="$(env.BUILDPATH)\Calc.exe"/>
      </Component>
    </DirectoryRef>
  </Fragment>
</Wix>

By now, the UI guy would have created a new WXS file for UI or stole the UI from a premade package. I normally steal UI from premade(I just coined this word) packages like Orca or some UI generated by freeware installers. Writing UI code for MSI using the WIX toolset is not exactly a pleasurable experience. <sigh/> So here goes the UI file. I am not going to include the WXS file here but if you need it, you can send me an email at vagmi.mudumbai@gmail.com. Now we have a file called dialogs.wxs which just has a bunch of dialogs, binaries, error texts, UITexts and properties defined for the UI.

Now we have to piece together each of these files and link them to our product. This is the MyProduct.wxs file which contains the feature-component mapping and the rest of the installation logic.

<!--MyProduct.wxs-->
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
  <Product Id="148E08FF-D7C4-46ed-8D4D-601C67FE0AFD" Language="1033" Name="MyProduct" Version="1.0.0" UpgradeCode="B7FE793A-800D-4c14-8CB4-B00AA84FF685" Manufacturer="Vagmi">
    <Package Id="D1192FCD-BA01-4d8f-BA7B-663CE9934BDE" Compressed="yes" Description="My WIX Installation" InstallerVersion="200" Languages="1033" Manufacturer="Vagmi"/>
    <Media Cabinet="Data.cab" EmbedCab="yes" Id="1"/>
    <Directory Id="TARGETDIR" Name="SourceDir">
        <Directory Id="ProgramFilesFolder" Name="PFiles">
            <Directory Id="INSTALLDIR" Name="Vagmi"/>
        </Directory>
        <Directory Id="DesktopFolder" Name="DTFOLDER"/>
    </Directory>
    <UI>
        <InstallUISequence>
            <Show Dialog="SetupCompleteError" OnExit="error" />
            <Show Dialog="SetupInterrupted" OnExit="cancel" />
            <Show Dialog="SetupCompleteSuccess" OnExit="success" />
            <Show Dialog="SetupInitialization" After="LaunchConditions" />
            <ResolveSource Before="CostFinalize"><![CDATA[Not Installed And Not PATCH]]></ResolveSource>
            <Show Dialog="InstallWelcome" After="MigrateFeatureStates"><![CDATA[Not Installed And (Not PATCH Or IS_MAJOR_UPGRADE)]]></Show>
            <Show Dialog="SetupResume" After="InstallWelcome"><![CDATA[Installed And (RESUME Or Preselected) And Not PATCH]]></Show>
            <Show Dialog="MaintenanceWelcome" After="SetupResume"><![CDATA[Installed And Not RESUME And Not Preselected And Not PATCH]]></Show>
            <Show Dialog="SetupProgress" After="MaintenanceWelcome" />
        </InstallUISequence>
    </UI>
    <Feature Id="TheOnlyFeature" ConfigurableDirectory="INSTALLDIR" Level="1" Title="TheOnlyFeature" Description="TheOnlyFeature">
        <ComponentRef Id="Notepad.exe"/>
        <ComponentRef Id="Calc.exe"/>
    </Feature>
  </Product>
</Wix>

As you can clearly see, the complex job of creating an installer package is split among the developers. This is a great feature as it does away with Merge Modules and all the associated hassles with modularization. Further changes or corrections can be made independently without affecting other parts of the installation. The only part that needs automation is the generation of WXS fragments for the developers. Although it is a simple piece of code to write, we cannot expect all developers to be conversant with XML and learn the WIX toolset. All we need is for them to specify the files and the destinations in a tool and a tool that should generate minimal WXS code for these fragments. I do not think that should be a problem considering the level of automation installation and build teams adopt for other tasks.

Building these is a straight forward task.

• Compile the files using the following command.
  candle dev1.wxs dev2.wxs dialogs.wxs myproduct.wxs
  
  This would produce dev1.wixobj, dev2.wixobj, dialogs.wixobj and myproduct.wixobj


• You can then link the generated object files using light.
  light -out build/product.msi dev1.wixobj dev2.wixobj dialogs.wixobj myproduct.wixobj

The myproduct.msi file would be generated in a folder named 'build' under the current directory.

WIX Tool

I have become quite lazy off late. I caught up with the WIX chat only after Rob posted the link on his blog. I was glad to find out that WIX was going to get integrated into Visual Studio.NET and there would be a MSBUILD task for WIX in Visual Studio .NET 2005. Very interesting indeed. Read the Windows Installer chat transcript for more information.

There has been a lot of commotion in the wix-users list about a good tool that generates WIX code for us. It was really encouraging to find out that there are a lot of developers just like me who feel that it is important to have a WIX code gen tool at least for the painful UI parts. Also it would help us get the overall view of the Setup project. I would visualize the tool to be some sort of a modeler than a full blown tool like InstallShield. I would still like to edit the XML code by hand and use the tool only as a layer to automate the redundant tasks. This would of course save a lot of time for more interesting things for more important stuff. (Yes. Gaming does count as one of the important stuff.) <smile/>

I am no C# guru and I really have started learning C# and .NET only for the past 6 months. Yeah! I have been a little slack in joining the .NET bandwagon. I have started coding in C# for fun and I have started to play around with the WIX code right about now. I have been trying to serialize and de-serialize WIX objects into XML contents using the Serialize namespace. But reality is just dawning on me. XML Serialization support in the System.XML namespace is not up to the mark. At least not up to my expectation. I spend most of my code type checking stuff and writing my own wrapper classes to expose the public members as properties. As I said, I am no tools guru but I was hoping to get each of the WIX object into the property grid. This would ease out most of the work and present a decent WXS reader if not a tool to write WXS file. Gabor has started working on a C# based WIX Dialog editor. I cant wait to get its sources. I am now looking for a good free refactoring plug in for Visual Studio .NET 2003 to convert public variables into public properties. I thought of writing the addin myself but I really don't want to re-invent the wheel. But if I end up writing it, I promise to expose the code to the community. <smile/>

Designing Good Setups

I have been mulling over the essentials of designing good setups and I found that most of it boils down to the component rules and the way you identify the data that goes to the setup. Microsoft enforces a set of rules that you should follow while authoring components. So, if the components are well managed, your product is well managed to a certain extent. Once you have identified the "Resources" in the setup, you would have to segregate them into components. Resources are bits installed by the setup, which do not change during the lifetime of the application, unless changed by a better version of the product by means of an MSI operation (small update/ minor upgrade/ major upgrade). Rob has blogged about what setup is and explains Resources, Application Data and User Data in good detail. I have my own opinions on handling application data. User Data is never installed, managed or removed and hence does not bother us.

Components are logical group of resources like Files, Registry entries, COM information, ODBC Data, Shortcuts and so on. Component is considered the atomic unit of installation. This statement is quite misleading and I have queued up a blog for Windows Installer's wierd behaviors to deal with that <grin/>.

• As a rule of the thumb, you should place PE files (EXE, DLL, OCX and so on) individually in a component. Ensure that the key file that you select for a component is versioned. If the developer has not versioned the file, let him know that versioning a PE is very important to have a reliable upgrade. There are certain situations in which the setup developer has to assert his/her position to ensure the quality of the software shipped. After all if the installation fails, the customers would have no chance to further evaluate or use the product. If you have non-versioned resources that need to be upgraded, ensure that they are made companions to the key file of the component. This would ensure that the versioning logic for the component is consistent.


• Never remove a resource or add a resouce to the component. If you have a file in the component, the only thing you can do is to bump up the file version or change the contents of the file itself. You can never add a file or remove a file from a component. This would result in incorrect reference counting and might prematurely remove resouces from the system or leave resource behind during uninstallation.


• You cannot include resources going to multiple locations in the same component. Also you cannot change the destination of the component once declared. You would have to ensure that once the component design has been finialized, its destination's position in the directory product tree structure does not change.


• If you have same file, especially a key file included in two different components that go to the same destination directory (e.g., 32 bit and 64 bit files or different language versions) ensure that these components are mutually exclusive by setting appropriate conditions on the components.


• Component code and name of the component must be the same.


• Make sure that there is only one COM server per component. Also ensure that the component is not linked to multiple features and is linked only to a single feature.


• Ensure that shortcuts are included in the same component that has the file. Thus, the shortcuts are installed only when the file is installed. This might seem very elementary but people from other installation worlds like InstallScript, find this particularly hard to digest. 

Microsoft has also included a nice little article on things that could happen when component rules are broken in the Platform SDK.

WIX features

After my recent cribbing about my wish-list for WIX, it is only fair that I talk about the advantages of WIX over other tools like InstallShield. I am primarily comparing it with InstallShield, as it is the only tool that I am aware of to an extent and is also the current market leader (in my opinion) in the installation tools market place.

1) XML Format: WIX source files are in a programmer friendly format. You are allowed to comment the code at any place, that makes the installation project source files as readable as the application's source files. Tools like InstallShield X allow you to save your file in the XML format but editing the XML and adding comments to it is not really easy. Furthermore, the changes in the source code across versions can be easily tracked by diffing from the source control, which would be very inconvenient with binary files.

2) Distributed Development: Each setup developer or even the application developer can edit a small fragment of the project to produce multiple WXS files which can be compiled independently and linked in the final process to produce MSI/MSM files. To my knowledge, this feature is currently available only with WIX. Of course, you could use other tools to create Merge Modules, but they would only add to the confusion with the modularization of the columns than to aid the process of distributed development. You can use WIX to maintain the readability of code. You could include comments within the code to actually point to the WXS file from which the element is referenced.

3) Clean Installation: You do not have to go through a whole big deal of the installation process just to get the package authoring system available on the build and the design environments. The WIX package decompressed is only 3.56 MB including the documentation and comes completely with custom actions to work with IIS, SQL Server, hypercharged LockPermissions functionality and much more. There are no DLLs to be registered, no registry entries to be created. Just unzip it into a machine with .NET Framework 1.1 installed and WIX would run without any problems. You would need Platform SDK on the machine to work with merge modules. Or at least the mergemod.dll registered on the build machine.

4) Custom Action Library: WIX comes with a custom action library to install/configure IIS websites and virtual directories. It is also present in InstallShield. WIX has support for SQL Server 2000 which is also present in InstallShield X. So I guess that levels the playing field. What WIX currently does not have is the support for installing COM+ Applications. But people following the Wix-Users list would know that these are already been worked upon and there have been a couple of users who have implemented the code and are currently contributing it back to the community.

What does WIX need?

After a long time, I have got some time to get to my blog. After a long day of presentations, I guess it is only fair enough that I relax by concentrating sometime on my long forgotten blog. We in our organization primarily use InstallShield for most of our setup development needs. I and one of my colleagues were discussing the topic of distributed development. InstallShield is good in a lot of things but distributed development is not one of them.

As a common sense approach, distributed development is done using merge modules. Although these may be discrete chunk of non reusable installation units, merge modules are the only way to enable Distributed development with InstallShield. I then truly realized the power of WIX which inherently supports distributed development. I then mentally started preparing a list of features that WIX ought to have to replace tools like InstallShield.

1) GUI - I guess WIXStudio need to be pushed a little harder. I personally do not believe that a development tool such as WIX really needs a GUI. But if the world were to run only on my beliefs, it would be a far better place than it is right now . WIX language is pretty straight forward and the tool nudges the developers on to the right track. Where the tool fails to point to the right direction, wix-users list does the job.

2) Bells and Whistles - Tallow is a cool tool but is unfortunately very buggy. As Rob suggested it is a fertile field for development and is constantly improving. I have seen some pretty encouraging messages in the Users list. I still have not become a part of the dev-list as yet, as I really cannot contribute at this point in time. I would however like to do so as soon as I have access to my computer at home.

3) Dialogs - It would be great to have a dialog editor. If not a full blown IDE for WIX, the least setup developers expect would be to have a dialog editor. Rob would argue that setup need not have a user interface, but in reality, dialogs are required for a enterprise setup of a decent quality. I have posted a message on WIX-Users-list for a list of free/open source MSI editors with visual Dialog editors. We could then dark the setup and create a WXS fragment.

4) Documentation – Last but not the least, WIX really needs a lot more documentation. I have been really busy off-late and have not been able to contribute as much to WIX as before. I hope the WIX Wiki would grow to fill this need.

IMHO, WIX is still the best tool to create installations for large projects.

Good Bye!

Imagine a blog entry where I talk about my last day in my organization. Incidentally Rob is moving to the Longhorn Windows Setup team.

Riko's Blog Entry on "Resources and Data"

Imagine a blog entry where Riko speaks on the Resources, User Data and Application Data. As noted in his blog, he is yet to make his point. It is an excellent blog entry. Also read Rob's entry on what setup is. There are a couple of links to the same entry in Riko's article too.

Handling Application Data

Recently, I had initiated a thread in the WIX users list regarding the inclusion of application data in the MSI package. My thanks to Riko, who shared his views on that. It has always been tricky and will continue to be a tricky situation to deal with application data. Most applications include their application runtime and configuration data as registry entries, INI or XML files. An unassuming developer might package these registry entries with the Main application's component, as they are dependent on the main application. This would work fine during the first install, but would cause significant issues during an upgrade. Geoff has blogged one such scenario. This blog tries to capture some of my thoughts regarding such Application Data.

For starters, lets look at what happens when you include the application data in the MSI package. The user, to configure the software the way they want, usually modifies these entries. Many times, it is the application itself that changes or writes data to the registry, INI or XML files to persist the configuration state of the application. Assuming we have included such files and registry entries grouped in the same component as the main executable. We can argue that it would be a good design as these entries are logically bound to the application and have no sense existing as a standalone unit. The application would roll out pretty smooth and will have no problems until you hit a point when you have to upgrade. Let us consider a minor upgrade, no major changes, a few DLLs modified and couple of executables added. Let main executable file have its version bumped. As the executable is the key file of the component, it qualifies for an upgrade. Let us also consider changes made to the XML files and a few Registry entries to accommodate the change. During an upgrade, all hell would break loose. The registry would be overwritten and the registry might be left with conflicting entries and some XML files would updated, the others may be left untouched as it would have a greater modified date. If you are really unlucky, the testing team might not quite get the tests done right. The Result: Total Chaos.

So how should these things be handled? The answer to this question is not straightforward and seems to be pretty tricky. The most ideal scenario would be to let the application handle its own configuration. The application should be able to create these registry entries and XML files during the first run. The setup developer then has to merely author the RemoveFile and RemoveReigstry tables to get the files out during uninstallation. As far as XML handling is concerned, it is best left to the application to handle it. MSI does not support XML handling and we need robust custom actions to do it for us. Handling XML data/SQL Data during upgrades can only be done using custom code. It would be desirable if this code is present on the application's end and not on the installer's end. If you do have to include the application data in the MSI package, the following guidelines may be useful.

1) Identify and Isolate the Application Data:
Clearly identify the resources like registry entries, INI Files and other configuration files that are to be used by the application. Use the INIFile table to author INI Files. Avoid including the INI file itself in the File table. Once identified, include the application data in a separate component. Use multiple components, if necessary.

2) Minor Upgrade and Uninstall Scenario:
If the application creates extra files or registry entries, ensure that the RemoveFile and the RemoveRegistry tables are populated. The problem is that these tables are referred to even during installation of the component. So you would have to set the Never Overwrite bit of the component to make sure that this component is never marked for an upgrade. Code the upgrade logic for the application data in to the application itself. In the unlikely event of it not being feasible, write custom actions to achieve the same. Avoid writing custom actions for tasks that can be handled by the application.

3) Major Upgrade Scenario:
Certain applications like InstallShield X, schedule the RemoveExistingProducts action right before the InstallInitialize action. This is not the most efficient placement for this action. The most efficient placement for this action would be after the InstallFinalize action. I will talk about this in detail another time, if you haven't figured it out as yet. If the authoring tool places the RemoveExistingProducts action before InstallInitialize, manually schedule it after InstallFinalize. There are certain custom actions in InstallShield X (I guess Component Services) that are not very happy with this placement, although I am not very sure about this. In these cases, you would have to use related locator (AppSearch, RegLocator, IniLocator, etc.,) and signature tables to get save the information into properties and reuse them in the install. This is a very elaborate and time-consuming exercise. The shorter and a devilish way to get around this is to write custom actions to read the values in the registry and store them in a text file, back up your application data files and restore them back with another set of custom actions. As unscientific as it might sound, a lot of people have used it to get out of this mess.

Rob is currently working on a similar blog entry. I can't wait to read on what he has to say about this.

Designing Upgrades - Part I

The previous week had been really busy. With five new recruits on board, I was asked to give them a quick introduction to InstallShield X and Windows Installer. It was broken up into 5 sessions, 3 hours each. And of course, I had my routine to take care of. So, after all this hard work, I decided to pamper myself with a movie the weekend. Before that I have decided to write something about designing upgrades with Windows Installer and WIX. I have split this article into two for easy consumption.

Designing upgrades are fairly basic and I have seen a lot of people in the forum having a little trouble getting started with the upgrades. But once they understand the concept of upgrades from the Windows Installer's perspective, it becomes a cakewalk. So, for starters, let me just talk about the different types of upgrades that you can perform with Windows Installer and implement the same using WIX. This article is designed only to be a quick start guide and is no means a complete guide for an upgrade. If you want to know more about upgrades, read the Upgrades and Patching section of the Windows Installer SDK documentation.

Windows Installer keeps track of products and packages using GUIDs. There are three important GUIDs that you need to know to understand upgrades.

• ProductCode - This uniquely identifies a product. This value is written to the Property table under the name ProductCode.


• UpgradeCode - This GUID is used to logically bind related products. This value is written to the Property table under the name UpgradeCode.


• Package Code - This uniquely identifies a package. Almost any change in the MSI package, mandates a new Package GUID. This value is written to the summary information sream under the name of 'Revision Number'.

And then there is the ProductVersion property, which specifies the version of the product. Windows Installer recognizes three types of upgrades. In all these upgrades, the package code will always change.

• When the upgrade just changes the application files but does not change the Product Code or the Product version, it is termed as a small update.


• When the upgrade changes only the ProductVersion but does not change the ProductCode, it is termed as a minor upgrade.


• When both the ProductCode and ProductVersion changes, it is termed as a Major Upgrade.

The small update and the minor upgrade can be installed over the existing installation. Usually, they only make changes to the parts of applications that have been changed. There are usually no major design changes to the product tree, excepting a few additions and modifications.

The major upgrade however, installs a completely new product and uninstalls the existing version of the product(s). Minor upgrade and small update can be targetted only at a particular product but Major upgrades can target more than one product. Major upgrade usually would have major changes made to the product tree. There might be situations that you might have to change the product code. In those cases, you would have no other choice but to perform a major upgrade.

So, as long as you do not have to change the product code, you can perform a small update or a minor upgrade. Minor upgrade would allow you to track the upgrade applied by looking at the ProductVersion. You can apply a minor upgrade or small update using the following command line.

msiexec.exe /i REINSTALL=ALL REINSTALLMODE=vomus

The value of the REINSTALL property is a list of features delimited by commas that are to be reinstalled. The features listed must be present in the Feature column of the Feature table. The REINSTALLMODE property is a string containing letters specifying the type of reinstall to perform. Options are case-insensitive and order-independent. This property should normally always be used in conjunction with the REINSTALL property. However, this property can also be used during installation, not just reinstall.

To design a major upgrade, you would have to author the Upgrade table. The upgrade table allows you to filter products and subsequently features based on the UpgradeCode, ProductVersion and Language. The Remove column of the upgrade table can be used to specify the list of features to be removed. Windows Installer will remove all the features, if the column is null. The ActionProperty column can be used to specify the name of a property. For the major upgrade to work, you need to have FindRelatedProductsAction and RemoveExistingProducts action in the InstallExecuteSequence table. The FindRelatedProducts action reads the settings in the upgrade table and stores the matching ProductCode(s) in the property specified by the ActionProperty column of the Upgrade Table. The RemoveExistingProducts action removes the product during installation. The most efficient placement for the RemoveExistingProducts action is after the InstallFinalize action. However, tools like InstallShield X, sequence this action in between InstallValidate and InstallInitialize actions.

To be continued...

Installing Windows Services (Created with .NET) with WIX

This is a pretty obscure topic and there is not enough literature on this. Installing a .NET Service is a fairly simple task. The Visual Studio.NET interface allows you to add the System.Configuration.Installer class to your assembly that enables managed installations. Developers often test their services by using the installutil.exe command line tool. But this tool is not the most appropriate for packaging as it shows an ugly command window during installation, which no setup developer would desire. Microsoft, includes a DLL named installutillib.dll with a single MSI entry point function named ManagedInstall. This function can be called via a MSI custom action to handle the four overridable functions exposed by the Installer Class. I am an InstallShield X user and installing .NET service is as easy as setting a property for the component from the IDE. I was a little lost when I wanted to achieve the same with WIX. Although, I knew that I could create these custom actions myself, I was hunting for a way by which I could do it in an easier fashion with WIX. After hours of searching the WiX.chm file, I decided to get on with it myself. I still am not sure if its hidden somewhere in the Wix.chm file.

I was a little skeptical about this and hence started of with an empty .NET enabled Windows Service that does nothing. I added the Installer class to the service and built it. I was too lazy to change the name. So my service was just called Service1 as christened by VS.NET 2003. Once we have our service executable ready, we have to create a little configuration file in XML that specifies the supported frameworks. I called it the IUConfig.XML For people wondering what IU stands for, it is short for InstallUtil <smile/>. The file is fairly simple and it goes something like this.

<?xml version="1.0"?>
<configuration>
<startup>
<supportedRuntime version="v1.1.4322"/>
</startup>
</configuration>

I put my InstallUtilLib.dll, FirstWindowsService.exe (My Windows Service) and IUConfig.xml in a folder called src. You can find the InstallUtilLib.dll in your [WindowsFolder]\Microsoft.NET\Framework\v1.1.4322\ directory. Once you have these three files, its time to start coding the WXS file. Again, just for the sake of simplicity, I am going to install only this service and nothing else. So here is my WXS file. As you can see it is not much. It has only a feature with one component, containing the service executable and the iuconfig.xml file. I have added the InstallUtilLib.dll as a binary.

<?xml version='1.0'?>
<Wix xmlns='http://schemas.microsoft.com/wix/2003/01/wi'>
<Product Id='F47A6F48-86C1-47A8-B404-35656C908BEB' Name='DotNetService' Language='1033' Version='1.0.0.0' Manufacturer='Vagmi' UpgradeCode='5BDA92CF-5D2B-4638-8550-4B8BE5BA8F24'>

<Package Id='????????-????-????-????-????????????' Description='Dot Net Service' Comments='Creating a .NET service' Manufacturer='Vagmi' InstallerVersion='200' Compressed='yes'/>
<Media Id='1' Cabinet='dotnet.cab' EmbedCab='yes' />

<Directory Id='TARGETDIR' Name='SourceDir'>
<Directory Id='ProgramFilesFolder' Name='PFiles'>
<Directory Id='DOTNETSERVICE' Name='DotNet' LongName='DotNetService'>
<Component Id='TheService' Guid='FF15180D-B296-4F30-9385-0F15B8ACC1FF'>
<File Id='WindowsService' Name='Firstw~1.exe' LongName='FirstWindowsService.exe' KeyPath='yes' DiskId='1' src='src\FirstWindowsService.exe' />
<File Id='ConfigFile' Name='IuConfig.xml' LongName='IuConfig.xml' DiskId='1' src='src\IuConfig.xml' CompanionFile='WindowsService'/>
</Component>
</Directory>
</Directory>
</Directory>

<Feature Id='TheOnlyFeature' Description='Feature contains the single component' Level='1'>
<ComponentRef Id='TheService'/>
</Feature>

<!-- Including the InstallUtilLib.dll. This file does all the magic of installing the services. -->
<Binary Id='InstallUtil' src='src\InstallUtilLib.dll' />
</Product>
</Wix>

To perform a managed install of the service, we need four custom actions - two deferred custom actions for installing and uninstalling, one commit custom action and one rollback custom action. As these custom actions execute in the higher security context, we need to pass data to these custom actions using four separate 'Set Property (Type 51)' custom actions. The ManagedInstall function expects the following parameters. The exact functionality of the parameters is still a mystery to me and I have to yet reasearch on it. But for now, we would take this for granted.

/installtype=notransaction /action=(install/uninstall/commit/rollback) /LogFile= "PathTo\Assembly" "PathTo\iuconfig.xml"

So the code for custom actions would look something like this.

<CustomAction Id='InstallServiceSetProp' Property='InstallService' Value='/installtype=notransaction /action=install /LogFile= "[#WindowsService]" "[DOTNETSERVICE]iuconfig.xml"'/>
<CustomAction Id='InstallService' BinaryKey='InstallUtil' DllEntry='ManagedInstall' Execute='deferred' />
<CustomAction Id='UnInstallServiceSetProp' Property='UnInstallService' Value='/installtype=notransaction /action=uninstall /LogFile= "[#WindowsService]" "[DOTNETSERVICE]iuconfig.xml"'/>
<CustomAction Id='UnInstallService' BinaryKey='InstallUtil' DllEntry='ManagedInstall' Execute='deferred' />
<CustomAction Id='CommitServiceSetProp' Property='CommitService' Value='/installtype=notransaction /action=commit /LogFile= "[#WindowsService]" "[DOTNETSERVICE]iuconfig.xml"'/>
<CustomAction Id='CommitService' BinaryKey='InstallUtil' DllEntry='ManagedInstall' Execute='commit' />
<CustomAction Id='RollbackServiceSetProp' Property='RollbackService' Value='/installtype=notransaction /action=rollback /LogFile= "[#WindowsService]" "[DOTNETSERVICE]iuconfig.xml"'/>
<CustomAction Id='RollbackService' BinaryKey='InstallUtil' DllEntry='ManagedInstall' Execute='rollback' />

You would not have to sequence these custom actions such that the uninstall custom actions run before the RemoveFiles action, and install, rollback & commit custom actions are scheduled after the InstallFiles action in the same order. So your <InstallExecuteSequence> would look something like this.

<InstallExecuteSequence>
<Custom Action='UnInstallServiceSetProp' After='MsiUnpublishAssemblies'>$TheService=2</Custom>
<Custom Action='UnInstallService' After='UnInstallServiceSetProp'>$TheService=2</Custom>
<Custom Action='InstallServiceSetProp' After='StartServices'>$TheService>2</Custom>
<Custom Action='InstallService' After='InstallServiceSetProp'>$TheService>2</Custom>
<Custom Action='RollbackServiceSetProp' After='InstallService'>$TheService>2</Custom>
<Custom Action='RollbackService' After='RollbackServiceSetProp'>$TheService>2</Custom>
<Custom Action='CommitServiceSetProp' After='RollbackService'>$TheService>2</Custom>
<Custom Action='CommitService' After='CommitServiceSetProp'>$TheService>2</Custom>
</InstallExecuteSequence>

Putting all this together, we would have a file like this.

<?xml version='1.0'?>
<Wix xmlns='http://schemas.microsoft.com/wix/2003/01/wi'>
<Product Id='F47A6F48-86C1-47A8-B404-35656C908BEB' Name='DotNetService' Language='1033' Version='1.0.0.0' Manufacturer='Vagmi' UpgradeCode='5BDA92CF-5D2B-4638-8550-4B8BE5BA8F24'>

<Package Id='????????-????-????-????-????????????' Description='Dot Net Service' Comments='Creating a .NET service' Manufacturer='Vagmi' InstallerVersion='200' Compressed='yes'/>
<Media Id='1' Cabinet='dotnet.cab' EmbedCab='yes' />

<Directory Id='TARGETDIR' Name='SourceDir'>
<Directory Id='ProgramFilesFolder' Name='PFiles'>
<Directory Id='DOTNETSERVICE' Name='DotNet' LongName='DotNetService'>
<Component Id='TheService' Guid='FF15180D-B296-4F30-9385-0F15B8ACC1FF'>
<File Id='WindowsService' Name='Firstw~1.exe' LongName='FirstWindowsService.exe' KeyPath='yes' DiskId='1' src='src\FirstWindowsService.exe' />
<File Id='ConfigFile' Name='IuConfig.xml' LongName='IuConfig.xml' DiskId='1' src='src\IuConfig.xml' CompanionFile='WindowsService'/>
</Component>
</Directory>
</Directory>
</Directory>

<Feature Id='TheOnlyFeature' Description='Feature contains the single component' Level='1'>
<ComponentRef Id='TheService'/>
</Feature>

<!-- Including the InstallUtilLib.dll. This file does all the magic of installing the services. -->
<Binary Id='InstallUtil' src='src\InstallUtilLib.dll' />

<!--Write custom actions to install, uninstall, commit and rollback the changes-->

<CustomAction Id='InstallServiceSetProp' Property='InstallService' Value='/installtype=notransaction /action=install /LogFile= "[#WindowsService]" "[DOTNETSERVICE]iuconfig.xml"'/>
<CustomAction Id='InstallService' BinaryKey='InstallUtil' DllEntry='ManagedInstall' Execute='deferred' />

<CustomAction Id='UnInstallServiceSetProp' Property='UnInstallService' Value='/installtype=notransaction /action=uninstall /LogFile= "[#WindowsService]" "[DOTNETSERVICE]iuconfig.xml"'/>
<CustomAction Id='UnInstallService' BinaryKey='InstallUtil' DllEntry='ManagedInstall' Execute='deferred' />

<CustomAction Id='CommitServiceSetProp' Property='CommitService' Value='/installtype=notransaction /action=commit /LogFile= "[#WindowsService]" "[DOTNETSERVICE]iuconfig.xml"'/>
<CustomAction Id='CommitService' BinaryKey='InstallUtil' DllEntry='ManagedInstall' Execute='commit' />

<CustomAction Id='RollbackServiceSetProp' Property='RollbackService' Value='/installtype=notransaction /action=rollback /LogFile= "[#WindowsService]" "[DOTNETSERVICE]iuconfig.xml"'/>
<CustomAction Id='RollbackService' BinaryKey='InstallUtil' DllEntry='ManagedInstall' Execute='rollback' />

<!-- Now to sequence these CAs in the execute sequence -->

<InstallExecuteSequence>

<Custom Action='UnInstallServiceSetProp' After='MsiUnpublishAssemblies'>$TheService=2</Custom>
<Custom Action='UnInstallService' After='UnInstallServiceSetProp'>$TheService=2</Custom>

<Custom Action='InstallServiceSetProp' After='StartServices'>$TheService>2</Custom>
<Custom Action='InstallService' After='InstallServiceSetProp'>$TheService>2</Custom>

<Custom Action='RollbackServiceSetProp' After='InstallService'>$TheService>2</Custom>
<Custom Action='RollbackService' After='RollbackServiceSetProp'>$TheService>2</Custom>

<Custom Action='CommitServiceSetProp' After='RollbackService'>$TheService>2</Custom>
<Custom Action='CommitService' After='CommitServiceSetProp'>$TheService>2</Custom>


</InstallExecuteSequence>

<!--Now we're done-->

</Product>
</Wix>

Despite all my skepticism, the above code ran perfectly fine. Now, I have to work on the real services. Hope you find this useful.