Single no more

There have been a lot of changes with my life lately. The most prominent one of all is that I am single no more. I am now happily wedded to my long time love Shanthi. I stole a few minutes off my busy schedule to post this .

I am going drop off the web for a couple of weeks now. Bye Bye web. Cya later.

6006l3 4 h4x0r

I just came across this. 1337 5p34k is a form of hacker language that was very popular on BBSes in the good old days. I did this in my college days. I was pleasantly surprised to find this page on Google. :-)

The Observer Pattern using Ruby Meta- Voodoo

The ruby standard library already has an implementation for the observer pattern. The implementation of the pattern was not very rubyish. It seemed a direct implementation of the Java equivalent of it. Ruby is far more powerful. This is the more rubyish version of the of the code goes something like this.

require 'observerext'

class Person
include Observer #include the observer related methods
observable :name,:address # the metavoodoo to create the accessor methods and fire events
end


admin=Person.new
admin.name='vagmi'
admin.address='admin@somesite.com'

# first observer
admin.observe :name do |old,newvalue|
puts %{The administrator changed from #{old} to #{newvalue}}
end

# the second observer
admin.observe :name do |old,newvalue|
puts 'there can be multiple observers'
end

admin.name='root'
#this prints the following statements
# > The administrator has changed from vagmi to root
# > there can be multiple observers

You can find the code for observerext.rb here. What I have basically done is to extend the Module class, which defines the observable method. The observable method injects the attributes' getters and setters. The setter method notifies the observers if there are any. The Observer module does the leg work of registering and unregistering the blocks and the notifyobservers method.

Update: Forgot to credit Greg Houston for explaining some of the ruby voodoo.

Moving back to blogger

I have moved back to blogger from my previous blog on geekswithblogs.net. I know that frequently changing the URL of a blog will hurt the traffic. But I have no other choice. Geeks with blogs admin application does not support unicode content and that really sucks. Apart from that fact, I have been working on phonetic layouts for both हिन्दी and தமிழ் on Ubuntu. I have just completed them . The first version can be downloaded here.

Most people like me are living the g-life. I use almost all the services that google exposes. Search, Calendar, Blogger, CodeHosting, Adsense, Docs and Spreadsheets, Picasa web, Youtube (although not technically a google service yet) and maps to name a few. Google has long since become the synonym for the word search. It is now pioneering in the applications space. Google Apps for your domain will give products like M$ Exchange and Lotus Domino a run for their money. It makes perfect business sense for SMEs to base their groupware on Google's infrastructure than maintain their own. This is of course given that you must have the kind of bandwidth and the infrastructure to remain connected and ensure that the business is not affected. But I do not see this as a huge problem. GMail exposes your inbox via POP3 and hence it is possible to have your mails both offline and online. Google Calendar can be used to sync up appointments. What is missing from Google right now is a Wiki. That would make it a complete groupware solution. Google would probably integrate its docs and spreadsheets application when they are mature with your domain. Google is taking web to the next level. And it is way more than Web 2.0. But let us refrain from naming it Web 3.0. Just imagine if we have enough bandwidth and google lets us host our databases on its infrastructure. I am sure google would like it. :-)

We are now using the Web as its creator Tim Berners Lee envisioned it to be. A readable and a writable medium. Web is no more a place for companies to advertise their business. It is now a formidable platform on which they conduct business.

Its midnight and I am very tired. Its about time I hit the sack. Good night world.

Rolling with RubyCLR

I downloaded RubyCLR today to play around with it and found that it was much more productive than IronPython. The reason is quite the same as what I mentioned in my previous blog entry. The Ruby-CLR bridge, which is essentially a library, is implemented as a native Ruby extension. So, this gives the developers the choice of using both C based modules and .NET based assemblies from the same script. And thus, in my opinion, RubyCLR is better than JRuby. We do not have to rewrite most of the native wrappers that are already written. In fact, some of them can be brought to .NET. I am not sure on how to embed Ruby as a scripting language for .NET applications. The other way is pretty clear from the samples that are distributed with the download.

Go ahead. Try it out. You will be surprised on how productive you become.

Madai Thiranthu - A really cool Tamil Hip Hop Remix

While I was browsing the net, I came across this amazing remix. As Yuvan is busy butchering his father's masterpieces in the name of remixes, this group of Malaysian hip hop artists have produced another masterpiece remixing the older master piece by Ilayaraja.

http://www.youtube.com/watch?v=uQSDr4609BU

When Cross Platform is not good enough...

I have been an avid fan of programming languages like C#, Python and Java as they take away most of the clunkiness of C/C++ code. Java, .NET/Mono, Python, Ruby and all other similar languages make claims that they are cross platform compliant. But the truth is, the library developer is bound to a specific technology.

For instance, the bzr version control system is written entirely in python. It has quite some promising features but it can never measure up to Subversion. Just imagine integrating bzr with Eclipse or using it with Visual Studio 2005. I am sure that I am not the first person to raise this as a concern. It is a part of the FAQ here. But my concern is not about the performance but the portability. It might, at the outset, seem as a funny argument as Python is supposed to be an extremely portable language so why should anyone have such a concern? The problem is not with the platform portability but with the technology portability. You would have to jump through several hoops before you can get bzr working with Java or .NET/Mono. Another argument might be that people could simply port bzr to Jython or IronPython. This might be possible with open source software but might not be the case with the closed source counterparts. There are a lot of good number of libraries written in Java, which unfortunately have to be rewritten in .NET. A testimony to this is the J* open source projects being ported to .NET and called as N*, like xUnit, xAnt, xVelocity, etc.

I understand that Java, C#, Python or Ruby make developers insanely productive. But library developers have to resist the urge to develop libraries in these languages and try to write their libraries in C/C++. By libraries, I also mean the tools and toolkits that are associated with them. Cross platform portability is definitely a concern. But it can be addressed but portable libraries like Boost or APR. You can later provide bindings to higher languages in the .NET stack, Java, Python or Ruby using interface generators like SWIG. There are numerous projects that have followed this approach and are hugely succesful. For instance, Subversion, wxWidgets, GTK , SQLite, MySQL are all glorious examples of toolkits and libraries that have core C/C++ components and have language bindings to higher level languages.

I am not saying that programming in Python or Ruby is bad. I absolutely love the way they make you productive. But high level languages are not the panacea for all programming problems. There are certain categories of software that ought to be written in C/C++ and not with higher languages. You can then write wrapper/helper libraries to make the access even simpler. But if the library developer ensures compliance across technologies, higher the chances of the library being usable and consequently popular.

Just my 2 cents on library development.

Configuration Data is such a Pain

I have finally got my computer to my new house in Bangalore. I dont have access to the internet, so I will essentially by offline. I am yet to install SharpDevelop and .NET Framework SDK v1.1. Hopefully, I should be able to get it done today. I had some time today morning but I was too busy playing NFS Hot Pursuit 2 and Quake 3. (No real souls to frag. Only bots. )

Currently one of our installation packages does something that it should not be doing. It is handling configuration data for several applications. Clearly this created several issues during upgrades. Now, for the next schedule major release, we (the packaging team) decided that we would let the applications handle the configuration information. This was duly presented as a proposal to other "application development" teams in a meeting. The people were a little hot under the collar during the meeting. Other development groups suspected that we were doing this to brush off our responsibility. I am sure that other setup developers would also have faced such similar predicament.

I was recently browsing through the InstallShield Community forums and found that mosth setup developers were invariably including configuration data as a part of their installation. I even saw one post where a setup developer had used a VBScript custom action with the FileSystemObject and tried to replace certain place holder texts in the installed configuration file. I am sure that many of us, who have burnt their fingers with VBScript and FSO, would agree with me. Logically speaking, there is nothing wrong with this approach. But we have to realize that we are introducing a element that can go wrong. Having deferred VBScript custom actions with FSO is a lot of fun while debugging.

My recommended approach would be to have a small configurator utility, which would accept command line parameters and perform the same tasks. This serves two purposes.

1. The configuration data is effectively taken out of the MSI. You can avoid slicing and dicing the CustomActionData property to get the desired values. There is one element less in the project that can go wrong.
   
2. The user need not run the installation, if he has specified incorrect configuration. He has the option of configuring the software after installing it. While installing on locked down environments, the setup author has to ensure that the permissions table is authored such that the current user has no problems writing on the the specified resources. This should'nt be a problem for registry keys or folders as their permission tokens can always be altered by the installer.
   

Finally, Riko is back. It seems that he is going to submit his tallow source to the community. I can't wait to get my hands on it.

My 2 cents

If people were following the the WIX-Users Mailing list, they would find that it has been a little hot recently. The first instance (as far as my knowledge goes) was with Michael creating a validation tool for the WIX Toolset called 'smoke'. I personally found it very useful as it could easily be included as a part of the build process. The output is in a structured XML format that makes it easier for us to crunch and build reports. But, some of the members in the group did not share the same level of enthusiasm. Michael seemed to be put off by that and included a disturbing update to his blog entry. Rob immediately acknowledged it and posted a blog entry about being thick skinned.

Recently, history repeated with Marc Bogers starting to ask about the ICE 33 warnings as Tallow populates the Registry table instead of the Registry Tables group. Firstly, Tallow is only a helper utility and is anything but perfect. It is useful but does not exactly produce usable WIX code. Heck, it doesn't even write to a file. Rob had said that ICE33 warnings can be ignored as it created some "strange behavior". More precisely, the resiliency associated with advertisement would prompt the end user for the media when Windows Installer detects some inconsistencies with the CLSID registration. Michael dismissed the argument that it arose from sloppy authoring and had nothing to do with MSI itself. Follow Michael's blog entry on COM registration for more information.

As we all know, Rob does not agree on a lot of things like inserting GUIDs while generating output with Tallow, using the Registry Tables Group and so on. I am sure that he has good reasons behind them. I am also sure that many people (including me), have modified Tallow to suit their requirements. Some people are using Riko's Tallow that does write to the Regsitry table's group using the Class and TypeLib elements. (On a totally different note, Riko, when are you going to release that code of yours?) I have a version of Tallow that creates components with GUIDs and names them appropriately. I am pretty sure that I am aware of the consequences and I have written code to persist the component information for future reference.

This freedom is the inherent beauty of open-source software. People can use their own ingenious imagination to tailor the tool to best suit their requirements. Of course, some of our opinions differ and we need a healthy debate to analyze these opinions and make the best of it. We should refrain from flaming and should try to embrace new ideas. Rob may be the BDFL for the WIX Toolset but I am sure that he would like the WIX Toolset to be successful and that helps developers create better installers without breaking much sweat. I tried hard not to write about this topic but after reading Michael's blog and comments with certain conspiracy theories, I wanted to voice my opinion too. I am sorry if I have hurt anybody's feelings.

Using an InstallScript application as the setup launcher

This, I guess, is my first InstallShield specific post. MSI is good in a lot of things but not when dealing with more than one package in a single transaction. MSI engine is designed to be run as a single instance and cannot be used effectively to install more than one package. Nested installations are possible but often lead to undesired consequences. Having a setup chainer is good but it involves writing code and handling reboots. So, for all who are using InstallShield DevStudio and up can benefit by using the InstallScript project as their setup launcher and monitor. We can effectively "script" a launcher application and have complete control over the installation process using the available libraries.

Why InstallScript....? I would not usually recommend InstallScript for any of the custom actions in the project, as the MSI-Scripting bridge is a little weak and has certain limitations. However, InstallScript libraries would cut down most of the code that you would have to write with a brand new C++ application. Also the scripting engine supports the silent mode. So you do not have to code separately for that either. But on the flip side, you would have an overhead of 1 Meg for the InstallScript engine. But this is okay considering the fresh set of bugs that you would introduce with the first version of the launcher application. For those not concerned about silent mode, you can check out the free open source setup launcher (dotNetInstaller).

The trick here is to use the InstallScript's program....endprogram syntax instead of the event based model. This makes sure that you do not register the application and does not call unknown events in between. Good old block scripting. <smile/> This also means that you need to take care of exceptions. If you are a setup developer from the good old InstallShield Professional 5.x days, you would immediately be at home with this kind of approach. So a simple script to just display a message box would be as below.

#include "ifx.h" //The standard header
//Our program block starts here
program
 MessageBox("Hello World!",INFORMATION);
endprogram

InstallScript engine just executes the statments present between the program and endprogram statements. As you can see, this approach gives you the flexibility of a complete scripting language with libraries custom made for creating installations. InstallScript projects are known for their flexibility but miss on the other aspects of MSI such as Elevated Privileges, Install on Demand, Source Resiliency and Self-healing.

Spying on Registry Entries

I am probably letting out the best kept secret of installation industry. I always used to wonder and have now discovered how most of the installation tools spy on registry entries that are created during COM registration or similar processes without actually affecting the build system. The spying program creates temporary registry keys for each of the registry hives HKCR, HKLM, HKCU, HKU and it maps the registry hives to these temporary registry keys. It then triggers the registration function which creates registry entries withing the registry keys specified instead of including it in the hives. I came across this revelation while I was wading through the source code for Tallow in the WIX toolset.

The core of this spying exercise relies on functions exposed by the advapi32.dll. The actual hive to key mapping is performed by the RegOverridePredefKey() function. The handle to the registry key is passed by using the RegCreateKeyEx() or the RegOpenKeyEx() function. After the mapping is done, you can invoke the DllRegisterServer() function after loading the library using the LoadLibrary() function. This mapping would be active for all the registry entries created by that particular process. So out of process registration for exe files may not directly work with this method. For the sake of simplicity, I am going to extract COM Interop settings from a given assembly. So let us write a simple console app in C# to do this. This app would perform the mapping and write the registry entries to a REG file and wipe out the key after the file is written.

Extracting registration information from a DLL file is similar but involve the importing other functions from Kernel32.dll. So I am giving that a raincheck now. You can download the Wix toolset's source package if you are interested. To start off with, let us write a class with static members to import the functions from advapi32.dll and create wrappers for them. Ensure that you have System.Runtime.InteropServices namespace included.

    class OverRideRegistry
    {
        //We first declare some stuff required. The are defined in winreg.h and windows.h
        public static readonly UIntPtr HkeyClassesRoot = (UIntPtr)0x80000000;
        public static readonly UIntPtr HkeyCurrentUser = (UIntPtr)0x80000001;
        public static readonly UIntPtr HkeyLocalMachine = (UIntPtr)0x80000002;
        public static readonly UIntPtr HkeyUsers = (UIntPtr)0x80000003;

        public const uint Delete = 0x00010000;
        public const uint ReadOnly = 0x00020000;
        public const uint WriteDac = 0x00040000;
        public const uint WriteOwner = 0x00080000;
        public const uint Synchronize = 0x00100000;
        public const uint StandardRightsRequired = 0x000F0000;
        public const uint StandardRightsAll = 0x001F0000;

        public const uint GenericRead = 0x80000000;
        public const uint GenericWrite = 0x40000000;
        public const uint GenericExecute = 0x20000000;
        public const uint GenericAll = 0x10000000;

        #region The Interop Import Stuff
        //we now import the functions exposed by advapi32.dll
        //Use RegCreateKeyEx to get handle to the openedKey
        [DllImport("advapi32.dll", EntryPoint="RegCreateKeyExW", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
        internal static extern int RegCreateKey(UIntPtr key, string subkey, uint reserved, string className, uint options, uint desiredSam, uint securityAttributes, out IntPtr openedKey, out uint disposition);

        //This does the actual hive to key mapping
        [DllImport("advapi32.dll", EntryPoint="RegOverridePredefKey", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
        internal static extern int RegOverridePredefKey(UIntPtr key, IntPtr newKey);

        //Like good programmers, we release our handles.
        [DllImport("advapi32.dll", EntryPoint="RegCloseKey", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
        internal static extern int RegCloseKey(IntPtr key);
        //Our interops are done :-)
        #endregion

        //Now we actually write wrapper functions to use the imported functions
        //Wrapper for creating Registry Keys
        internal static IntPtr OpenRegistryKey(UIntPtr key, string path)
        {
            IntPtr newKey = IntPtr.Zero;
            uint disposition = 0;
            uint security = StandardRightsAll | GenericRead | GenericWrite | GenericExecute | GenericAll;
            int error = RegCreateKey(key, path, 0, null, 0, security, 0, out newKey, out disposition);
            return newKey;
        }
        //Wrapper for the mapping
        internal static void OverrideRegistryKey(UIntPtr key, IntPtr newKey)
        {
            int error = RegOverridePredefKey(key, newKey);
        }
        //Wrapper for freeing the handle
        internal static void CloseRegistryKey(IntPtr key)
        {
            int error = RegCloseKey(key);
        }
    }

Now that we have the helper class with the static methods, we can define our main Console App class, which would use the functions in the helper class to register the assembly and steal the registry entries. Let us define the functions that Map and Map the registry hives. I had initialized the RegLocation to a string "Software\\Vagmi\\RegInterop\\". So the extracted registry entries would be put in the HKLM hive within the specified path. Also not that I have mapped HKLM as the final mapping. Else all keys that are subsequently created would be created within our registry key.

        void MapRegHives()
        {
            MapRegHive(OverRideRegistry.HkeyClassesRoot,this.RegLocation+"HKCR");
            MapRegHive(OverRideRegistry.HkeyCurrentUser,this.RegLocation+"HKCU");
            MapRegHive(OverRideRegistry.HkeyUsers,this.RegLocation+"HKU");
            MapRegHive(OverRideRegistry.HkeyLocalMachine,this.RegLocation+"HKLM");
        }
        void MapRegHive(UIntPtr key, string location)
        {
            Console.WriteLine("Mapping " + key + "  to " + location);
            IntPtr createdKey=IntPtr.Zero;
            try
            {
                createdKey=OverRideRegistry.OpenRegistryKey(OverRideRegistry.HkeyLocalMachine,location);
                OverRideRegistry.OverrideRegistryKey(key,createdKey);
            }
            catch(Exception e)
            {
                Console.WriteLine("Caught exception: " + e.Message);
            }
            finally
            {
                //close key like responsible programmers
                OverRideRegistry.CloseRegistryKey(createdKey);
            }
        }
        //Functions to unmap registry hives.
        void UnMapRegHives()
        {
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyClassesRoot,IntPtr.Zero);
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyCurrentUser,IntPtr.Zero);
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyLocalMachine,IntPtr.Zero);
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyUsers,IntPtr.Zero);
        }

I then parse two command line arguments one for the DLL and another for the REG file to export. I then pass these as constructors to my class, which calls these functions.

        public RegSpyCOMInterop(string AssemblyPath, string RegFile)
        {
            try
            {
                Assembly a=Assembly.LoadFrom(AssemblyPath);
                RegistrationServices regServices=new RegistrationServices();
                //map hives to registry keys
                    MapRegHives();
                    //Register assembly for COM interop
                    regServices.RegisterAssembly(a,AssemblyRegistrationFlags.SetCodeBase);
                    //Unmap hives
                    UnMapRegHives();
                WriteToRegFile(RegFile);
            }
            catch(Exception e)
            {
                Console.WriteLine("Caught Exception : " + e.Message);
            }
        }

The WriteToRegFile() function launches regedit and exports the hive that contains our keys. It then uses simple text replacement to change the values of the exported file to make the REG file functional.

        public void WriteToRegFile(string regfile)
        {
            RegistryKey r=Registry.LocalMachine.OpenSubKey(RegLocation,false);
            PathToDelete=r.Name + "\\";
            Process pr=new Process();
            pr.StartInfo.FileName="regedit";
            pr.StartInfo.Arguments=" /e " + regfile + " " + r.Name;
            pr.Start();
            pr.WaitForExit();
            string line;
            StreamReader reader=new StreamReader(regfile);
            StreamWriter writer=new StreamWriter("TempFile.txt");
            while((line=reader.ReadLine())!=null)
            {
                writer.WriteLine(processRegistryName(line));
            }
            reader.Close();
            writer.Close();
            File.Copy("TempFile.txt",regfile,true);
            File.Delete("TempFile.txt");
            r.Flush();
            r.Close();
        }

        public String processRegistryName(string regname)
        {
            string newRegName="";
            newRegName=regname.Replace(PathToDelete,"");
            newRegName=newRegName.Replace("[" + PathToDelete.Substring(0,PathToDelete.Length-1) + "]","");
            newRegName=newRegName.Replace("HKCR","HKEY_CLASS_ROOT");
            newRegName=newRegName.Replace("HKCU","HKEY_CURRENT_USER");
            newRegName=newRegName.Replace("HKLM","HKEY_LOCAL_MACHINE");
            newRegName=newRegName.Replace("HKU","HKEY_USERS");
            return newRegName;
        }

There you have it. A command line utility to extract COM Interop Registry entries. Hope this answered some of your queries regarding registration without affecting the target system. You can find the entire source code at http://www.geekswithblogs.net/vagmi.mudumbai/articles/16581.aspx.

Microsoft's Whitepaper on Patch Sequencing

While browsing, I came across this excellent whitepaper on MSI 3.0 Patch Sequencing.

http://www.microsoft.com/downloads/details.aspx?FamilyId=AD7AC91E-2493-4549-AE6F-BF5E007C12A3&displaylang=en

Using WIX Toolset for Distributed Development

When I talk to my fellow developers and other installation developers about the WIX Toolset, they do not fully understand the implications of the WIX Toolset in its own right but map its features to their favorite installation development tools. The WIX Toolset mostly ensures that you follow the best practices while authoring packages. The biggest advantage of the WIX Toolset over other applications is the support for distributed development. I have raved and ranted about this in my previous blog entries but I would substantiate it with actual WXS code this time. For starters, WXS file is an XML file that has to follow the schema as specified in the wix.xsd file in the 'doc' folder of your distribution. Rob has an excellent demo on distributed development with the WIX toolset. But the quality of video is very poor and the sample files are not yet available for us to tweak around with. So I have come up with my own example.

Let us assume the following scenario. There are group of installation and application developers in the organization working on a setup project. Let us allocate roles (purely hypothetical. I am not a manager <grin/>) for our convenience. Let us assume that the application developers take care of maintaining a catalog of components and resources that go into the product. The second developer takes care of assigning the appropriate components into the respective merge modules or features in a product. Let another developer be responsible for creating UIs. Additionally, each application developer creates his own fragments of files by hand or by using a simple tool. Let us assume that we have to package three files.

1. Notepad.exe


2. ReadMe.txt


3. Calc.exe

Notepad.exe and Readme.txt are a logical unit and hence we would put them in a single component called Notepad.exe and set Notepad.exe as the key file of the component. We would also have Readme.txt as a companion file for Notepad.exe such that the versioning logic of Readme.txt is controlled by the versioning logic of Notepad.exe. Calc.exe is a standalone application that does not have anyother files. So here we go. The application developer codes his file named Dev1.wxs. Pretty simple isn't it?

<!--Dev1.wxs-->
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
  <Fragment>
    <DirectoryRef Id="INSTALLDIR">
      <Component Id="Notepad.exe" Guid="12CC008F-89A6-422d-868B-B066606FFD99" DiskId="1">
        <File Id="Notepad.exe" Name="Notepad.exe" LongName="Notepad.exe" KeyPath="yes" src="$(env.BUILDPATH)\Notepad.exe">
            <Shortcut Id="LaunchNotepad" Advertise="no" Description="Launches Notepad" Directory="DesktopFolder" Name="Notepad" LongName="Launch Notepad" WorkingDirectory="INSTALLDIR" Icon="Notepad.ico"/>
        </File>
        <File Id="Readme.txt" Name="Readme.txt" LongName="Readme.txt" CompanionFile="Notepad.exe" src="$(env.BUILDPATH)\Readme.txt"/>
      </Component>
    </DirectoryRef>
    <Icon Id="Notepad.ico" src="$(env.BUILDPATH)\Notepad.ico"/>
  </Fragment>
</Wix>

Now the second application developer creates his file dev2.wxs.

 

<!--Dev2.wxs-->
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
  <Fragment>
    <DirectoryRef Id="INSTALLDIR">
      <Component Id="Calc.exe" Guid="2C5ECB67-E585-4301-BAF4-5380FE6C26AB" DiskId="1">
        <File Id="Calc.exe" Name="Calc.exe" LongName="Calc.exe" KeyPath="yes" src="$(env.BUILDPATH)\Calc.exe"/>
      </Component>
    </DirectoryRef>
  </Fragment>
</Wix>

By now, the UI guy would have created a new WXS file for UI or stole the UI from a premade package. I normally steal UI from premade(I just coined this word) packages like Orca or some UI generated by freeware installers. Writing UI code for MSI using the WIX toolset is not exactly a pleasurable experience. <sigh/> So here goes the UI file. I am not going to include the WXS file here but if you need it, you can send me an email at vagmi.mudumbai@gmail.com. Now we have a file called dialogs.wxs which just has a bunch of dialogs, binaries, error texts, UITexts and properties defined for the UI.

Now we have to piece together each of these files and link them to our product. This is the MyProduct.wxs file which contains the feature-component mapping and the rest of the installation logic.

<!--MyProduct.wxs-->
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
  <Product Id="148E08FF-D7C4-46ed-8D4D-601C67FE0AFD" Language="1033" Name="MyProduct" Version="1.0.0" UpgradeCode="B7FE793A-800D-4c14-8CB4-B00AA84FF685" Manufacturer="Vagmi">
    <Package Id="D1192FCD-BA01-4d8f-BA7B-663CE9934BDE" Compressed="yes" Description="My WIX Installation" InstallerVersion="200" Languages="1033" Manufacturer="Vagmi"/>
    <Media Cabinet="Data.cab" EmbedCab="yes" Id="1"/>
    <Directory Id="TARGETDIR" Name="SourceDir">
        <Directory Id="ProgramFilesFolder" Name="PFiles">
            <Directory Id="INSTALLDIR" Name="Vagmi"/>
        </Directory>
        <Directory Id="DesktopFolder" Name="DTFOLDER"/>
    </Directory>
    <UI>
        <InstallUISequence>
            <Show Dialog="SetupCompleteError" OnExit="error" />
            <Show Dialog="SetupInterrupted" OnExit="cancel" />
            <Show Dialog="SetupCompleteSuccess" OnExit="success" />
            <Show Dialog="SetupInitialization" After="LaunchConditions" />
            <ResolveSource Before="CostFinalize"><![CDATA[Not Installed And Not PATCH]]></ResolveSource>
            <Show Dialog="InstallWelcome" After="MigrateFeatureStates"><![CDATA[Not Installed And (Not PATCH Or IS_MAJOR_UPGRADE)]]></Show>
            <Show Dialog="SetupResume" After="InstallWelcome"><![CDATA[Installed And (RESUME Or Preselected) And Not PATCH]]></Show>
            <Show Dialog="MaintenanceWelcome" After="SetupResume"><![CDATA[Installed And Not RESUME And Not Preselected And Not PATCH]]></Show>
            <Show Dialog="SetupProgress" After="MaintenanceWelcome" />
        </InstallUISequence>
    </UI>
    <Feature Id="TheOnlyFeature" ConfigurableDirectory="INSTALLDIR" Level="1" Title="TheOnlyFeature" Description="TheOnlyFeature">
        <ComponentRef Id="Notepad.exe"/>
        <ComponentRef Id="Calc.exe"/>
    </Feature>
  </Product>
</Wix>

As you can clearly see, the complex job of creating an installer package is split among the developers. This is a great feature as it does away with Merge Modules and all the associated hassles with modularization. Further changes or corrections can be made independently without affecting other parts of the installation. The only part that needs automation is the generation of WXS fragments for the developers. Although it is a simple piece of code to write, we cannot expect all developers to be conversant with XML and learn the WIX toolset. All we need is for them to specify the files and the destinations in a tool and a tool that should generate minimal WXS code for these fragments. I do not think that should be a problem considering the level of automation installation and build teams adopt for other tasks.

Building these is a straight forward task.

• Compile the files using the following command.
  candle dev1.wxs dev2.wxs dialogs.wxs myproduct.wxs
  
  This would produce dev1.wixobj, dev2.wixobj, dialogs.wixobj and myproduct.wixobj


• You can then link the generated object files using light.
  light -out build/product.msi dev1.wixobj dev2.wixobj dialogs.wixobj myproduct.wixobj

The myproduct.msi file would be generated in a folder named 'build' under the current directory.

WIX Tool

I have become quite lazy off late. I caught up with the WIX chat only after Rob posted the link on his blog. I was glad to find out that WIX was going to get integrated into Visual Studio.NET and there would be a MSBUILD task for WIX in Visual Studio .NET 2005. Very interesting indeed. Read the Windows Installer chat transcript for more information.

There has been a lot of commotion in the wix-users list about a good tool that generates WIX code for us. It was really encouraging to find out that there are a lot of developers just like me who feel that it is important to have a WIX code gen tool at least for the painful UI parts. Also it would help us get the overall view of the Setup project. I would visualize the tool to be some sort of a modeler than a full blown tool like InstallShield. I would still like to edit the XML code by hand and use the tool only as a layer to automate the redundant tasks. This would of course save a lot of time for more interesting things for more important stuff. (Yes. Gaming does count as one of the important stuff.) <smile/>

I am no C# guru and I really have started learning C# and .NET only for the past 6 months. Yeah! I have been a little slack in joining the .NET bandwagon. I have started coding in C# for fun and I have started to play around with the WIX code right about now. I have been trying to serialize and de-serialize WIX objects into XML contents using the Serialize namespace. But reality is just dawning on me. XML Serialization support in the System.XML namespace is not up to the mark. At least not up to my expectation. I spend most of my code type checking stuff and writing my own wrapper classes to expose the public members as properties. As I said, I am no tools guru but I was hoping to get each of the WIX object into the property grid. This would ease out most of the work and present a decent WXS reader if not a tool to write WXS file. Gabor has started working on a C# based WIX Dialog editor. I cant wait to get its sources. I am now looking for a good free refactoring plug in for Visual Studio .NET 2003 to convert public variables into public properties. I thought of writing the addin myself but I really don't want to re-invent the wheel. But if I end up writing it, I promise to expose the code to the community. <smile/>

Designing Good Setups

I have been mulling over the essentials of designing good setups and I found that most of it boils down to the component rules and the way you identify the data that goes to the setup. Microsoft enforces a set of rules that you should follow while authoring components. So, if the components are well managed, your product is well managed to a certain extent. Once you have identified the "Resources" in the setup, you would have to segregate them into components. Resources are bits installed by the setup, which do not change during the lifetime of the application, unless changed by a better version of the product by means of an MSI operation (small update/ minor upgrade/ major upgrade). Rob has blogged about what setup is and explains Resources, Application Data and User Data in good detail. I have my own opinions on handling application data. User Data is never installed, managed or removed and hence does not bother us.

Components are logical group of resources like Files, Registry entries, COM information, ODBC Data, Shortcuts and so on. Component is considered the atomic unit of installation. This statement is quite misleading and I have queued up a blog for Windows Installer's wierd behaviors to deal with that <grin/>.

• As a rule of the thumb, you should place PE files (EXE, DLL, OCX and so on) individually in a component. Ensure that the key file that you select for a component is versioned. If the developer has not versioned the file, let him know that versioning a PE is very important to have a reliable upgrade. There are certain situations in which the setup developer has to assert his/her position to ensure the quality of the software shipped. After all if the installation fails, the customers would have no chance to further evaluate or use the product. If you have non-versioned resources that need to be upgraded, ensure that they are made companions to the key file of the component. This would ensure that the versioning logic for the component is consistent.


• Never remove a resource or add a resouce to the component. If you have a file in the component, the only thing you can do is to bump up the file version or change the contents of the file itself. You can never add a file or remove a file from a component. This would result in incorrect reference counting and might prematurely remove resouces from the system or leave resource behind during uninstallation.


• You cannot include resources going to multiple locations in the same component. Also you cannot change the destination of the component once declared. You would have to ensure that once the component design has been finialized, its destination's position in the directory product tree structure does not change.


• If you have same file, especially a key file included in two different components that go to the same destination directory (e.g., 32 bit and 64 bit files or different language versions) ensure that these components are mutually exclusive by setting appropriate conditions on the components.


• Component code and name of the component must be the same.


• Make sure that there is only one COM server per component. Also ensure that the component is not linked to multiple features and is linked only to a single feature.


• Ensure that shortcuts are included in the same component that has the file. Thus, the shortcuts are installed only when the file is installed. This might seem very elementary but people from other installation worlds like InstallScript, find this particularly hard to digest. 

Microsoft has also included a nice little article on things that could happen when component rules are broken in the Platform SDK.

WIX features

After my recent cribbing about my wish-list for WIX, it is only fair that I talk about the advantages of WIX over other tools like InstallShield. I am primarily comparing it with InstallShield, as it is the only tool that I am aware of to an extent and is also the current market leader (in my opinion) in the installation tools market place.

1) XML Format: WIX source files are in a programmer friendly format. You are allowed to comment the code at any place, that makes the installation project source files as readable as the application's source files. Tools like InstallShield X allow you to save your file in the XML format but editing the XML and adding comments to it is not really easy. Furthermore, the changes in the source code across versions can be easily tracked by diffing from the source control, which would be very inconvenient with binary files.

2) Distributed Development: Each setup developer or even the application developer can edit a small fragment of the project to produce multiple WXS files which can be compiled independently and linked in the final process to produce MSI/MSM files. To my knowledge, this feature is currently available only with WIX. Of course, you could use other tools to create Merge Modules, but they would only add to the confusion with the modularization of the columns than to aid the process of distributed development. You can use WIX to maintain the readability of code. You could include comments within the code to actually point to the WXS file from which the element is referenced.

3) Clean Installation: You do not have to go through a whole big deal of the installation process just to get the package authoring system available on the build and the design environments. The WIX package decompressed is only 3.56 MB including the documentation and comes completely with custom actions to work with IIS, SQL Server, hypercharged LockPermissions functionality and much more. There are no DLLs to be registered, no registry entries to be created. Just unzip it into a machine with .NET Framework 1.1 installed and WIX would run without any problems. You would need Platform SDK on the machine to work with merge modules. Or at least the mergemod.dll registered on the build machine.

4) Custom Action Library: WIX comes with a custom action library to install/configure IIS websites and virtual directories. It is also present in InstallShield. WIX has support for SQL Server 2000 which is also present in InstallShield X. So I guess that levels the playing field. What WIX currently does not have is the support for installing COM+ Applications. But people following the Wix-Users list would know that these are already been worked upon and there have been a couple of users who have implemented the code and are currently contributing it back to the community.

What does WIX need?

After a long time, I have got some time to get to my blog. After a long day of presentations, I guess it is only fair enough that I relax by concentrating sometime on my long forgotten blog. We in our organization primarily use InstallShield for most of our setup development needs. I and one of my colleagues were discussing the topic of distributed development. InstallShield is good in a lot of things but distributed development is not one of them.

As a common sense approach, distributed development is done using merge modules. Although these may be discrete chunk of non reusable installation units, merge modules are the only way to enable Distributed development with InstallShield. I then truly realized the power of WIX which inherently supports distributed development. I then mentally started preparing a list of features that WIX ought to have to replace tools like InstallShield.

1) GUI - I guess WIXStudio need to be pushed a little harder. I personally do not believe that a development tool such as WIX really needs a GUI. But if the world were to run only on my beliefs, it would be a far better place than it is right now . WIX language is pretty straight forward and the tool nudges the developers on to the right track. Where the tool fails to point to the right direction, wix-users list does the job.

2) Bells and Whistles - Tallow is a cool tool but is unfortunately very buggy. As Rob suggested it is a fertile field for development and is constantly improving. I have seen some pretty encouraging messages in the Users list. I still have not become a part of the dev-list as yet, as I really cannot contribute at this point in time. I would however like to do so as soon as I have access to my computer at home.

3) Dialogs - It would be great to have a dialog editor. If not a full blown IDE for WIX, the least setup developers expect would be to have a dialog editor. Rob would argue that setup need not have a user interface, but in reality, dialogs are required for a enterprise setup of a decent quality. I have posted a message on WIX-Users-list for a list of free/open source MSI editors with visual Dialog editors. We could then dark the setup and create a WXS fragment.

4) Documentation – Last but not the least, WIX really needs a lot more documentation. I have been really busy off-late and have not been able to contribute as much to WIX as before. I hope the WIX Wiki would grow to fill this need.

IMHO, WIX is still the best tool to create installations for large projects.

Good Bye!

Imagine a blog entry where I talk about my last day in my organization. Incidentally Rob is moving to the Longhorn Windows Setup team.

Riko's Blog Entry on "Resources and Data"

Imagine a blog entry where Riko speaks on the Resources, User Data and Application Data. As noted in his blog, he is yet to make his point. It is an excellent blog entry. Also read Rob's entry on what setup is. There are a couple of links to the same entry in Riko's article too.

Handling Application Data

Recently, I had initiated a thread in the WIX users list regarding the inclusion of application data in the MSI package. My thanks to Riko, who shared his views on that. It has always been tricky and will continue to be a tricky situation to deal with application data. Most applications include their application runtime and configuration data as registry entries, INI or XML files. An unassuming developer might package these registry entries with the Main application's component, as they are dependent on the main application. This would work fine during the first install, but would cause significant issues during an upgrade. Geoff has blogged one such scenario. This blog tries to capture some of my thoughts regarding such Application Data.

For starters, lets look at what happens when you include the application data in the MSI package. The user, to configure the software the way they want, usually modifies these entries. Many times, it is the application itself that changes or writes data to the registry, INI or XML files to persist the configuration state of the application. Assuming we have included such files and registry entries grouped in the same component as the main executable. We can argue that it would be a good design as these entries are logically bound to the application and have no sense existing as a standalone unit. The application would roll out pretty smooth and will have no problems until you hit a point when you have to upgrade. Let us consider a minor upgrade, no major changes, a few DLLs modified and couple of executables added. Let main executable file have its version bumped. As the executable is the key file of the component, it qualifies for an upgrade. Let us also consider changes made to the XML files and a few Registry entries to accommodate the change. During an upgrade, all hell would break loose. The registry would be overwritten and the registry might be left with conflicting entries and some XML files would updated, the others may be left untouched as it would have a greater modified date. If you are really unlucky, the testing team might not quite get the tests done right. The Result: Total Chaos.

So how should these things be handled? The answer to this question is not straightforward and seems to be pretty tricky. The most ideal scenario would be to let the application handle its own configuration. The application should be able to create these registry entries and XML files during the first run. The setup developer then has to merely author the RemoveFile and RemoveReigstry tables to get the files out during uninstallation. As far as XML handling is concerned, it is best left to the application to handle it. MSI does not support XML handling and we need robust custom actions to do it for us. Handling XML data/SQL Data during upgrades can only be done using custom code. It would be desirable if this code is present on the application's end and not on the installer's end. If you do have to include the application data in the MSI package, the following guidelines may be useful.

1) Identify and Isolate the Application Data:
Clearly identify the resources like registry entries, INI Files and other configuration files that are to be used by the application. Use the INIFile table to author INI Files. Avoid including the INI file itself in the File table. Once identified, include the application data in a separate component. Use multiple components, if necessary.

2) Minor Upgrade and Uninstall Scenario:
If the application creates extra files or registry entries, ensure that the RemoveFile and the RemoveRegistry tables are populated. The problem is that these tables are referred to even during installation of the component. So you would have to set the Never Overwrite bit of the component to make sure that this component is never marked for an upgrade. Code the upgrade logic for the application data in to the application itself. In the unlikely event of it not being feasible, write custom actions to achieve the same. Avoid writing custom actions for tasks that can be handled by the application.

3) Major Upgrade Scenario:
Certain applications like InstallShield X, schedule the RemoveExistingProducts action right before the InstallInitialize action. This is not the most efficient placement for this action. The most efficient placement for this action would be after the InstallFinalize action. I will talk about this in detail another time, if you haven't figured it out as yet. If the authoring tool places the RemoveExistingProducts action before InstallInitialize, manually schedule it after InstallFinalize. There are certain custom actions in InstallShield X (I guess Component Services) that are not very happy with this placement, although I am not very sure about this. In these cases, you would have to use related locator (AppSearch, RegLocator, IniLocator, etc.,) and signature tables to get save the information into properties and reuse them in the install. This is a very elaborate and time-consuming exercise. The shorter and a devilish way to get around this is to write custom actions to read the values in the registry and store them in a text file, back up your application data files and restore them back with another set of custom actions. As unscientific as it might sound, a lot of people have used it to get out of this mess.

Rob is currently working on a similar blog entry. I can't wait to read on what he has to say about this.