To Nest or To Chain or To Merge

This blog entry is for relatively novice setup developers and not for the die-hard Windows Installer gurus. Normally different groups in an organization develop components of software, which can be used standalone or as a part of a suite of products. This is a very normal practice. So they go ahead and create a software package with a complete setup.exe and a separate MSI for it. But when they plan to integrate it with the suite. That is when the setup developer's headache starts. The setup developer might be hard pressed for time and would choose nested install of some sort to cut down time. This article discusses a few techniques to handle it in a better fashion.

Firstly, nested installs are evil. Let me tell you why. They can never be patched and cannot be cleanly removed from the target system. That alone should drive you away from using nested installs. You perform a nested installation. And Boom! The next thing you know is that you have lost control over the life cycle of your application, your house has been burgled and your wife has run away with your neighbor. Well, it was a slight exaggeration but you get the picture.

A smarter way to handle this, as many argue would be to call the child install by launching the MSI package's setup.exe or launch msiexec and pass appropriate command line parameters to install the package. This is equally bad. Such installations do not have their rollback logic integrated with the parent installer. Thus if the nested setup fails, you do not have any way to roll back the parent setup reliably or the vice-versa. Furthermore, Windows Installer's architecture allows only one installation at a point to make changes to the system. Having two installations modify system resources might not be the smartest thing to do.

The only way out of this mess is to use something called as the Merge Module. Merge Module is an atomic unit consisting of components, custom actions and various resources like files and registry entries. These merge modules can be integrated into the main installer at design time by the merge tool. You can either use tools like InstallShield to include merge modules in the application or use WIX's tag to include the merge module during build. The advantage of using the merge module is that the components in the merge module remain immutable. Thus it ensures that setup developers follow some of the many important component design rules. A resource going to a location will always have the same component GUID regardless of the product its being installed with. Windows Installer will then be able to do clean refcounting of the components and will give you a solid setup.

Beyond all this, if the setup were a third party setup, you would have to use the MSI package unless the vendor agrees that you can repackage his setup. Buts lets just assume that he does not. Even in that case, it is recommended that you keep the logic of the third-party setup miles away from your application. You can handle the installation of the third party application from the bootstrapper. There are a couple of applications that do that. InstallShield X Premier and Professional editions have a neat feature called as Setup Prerequisites, which let you call other MSI or non-MSI based installations from the bootstrapper. If you would like to stay open source, then you can use DevAge’s DotNetInstaller to achieve the same. Both of them are very easy to use. InstallShield’s solution comes with a price tag, sporting a snazzy killer interface and excellent documentation. While the DotNetInstaller is not a setup creation program but is just a bootstrapper and it is free.

All About WIX

Of late, I was very busy amidst many meetings. My friends would know the reasons for that. As soon as I got away from my day job (actually afternoon, I work from 1PM to 10PM), I got really involved with WIX. I have found this tool very versatile, stable and extremely lightweight. I believe that this is used internally at Microsoft by MS Office team. The MS Office team actually created Windows Installer (Codenamed Darwin) and hence it is only fair to assume that they use every feature of Windows Installer. This should give you some idea of the versatility of this tool. For folks who don’t know what WIX is, Rob has included a link to the Introduction to Windows Installer XML video in his blog. He gives a broad overview of the tool and demonstrates the usability of this tool in real-time. Although this tool is versatile, learning it would be difficult if you do not know Windows Installer. Unfortunately, there are not enough good resources to learn Windows Installer. I did my learning with Bob Baker's books published by InstallShield Press. I supplemented my knowledge of Windows Installer with the Mike Gunderloy's book VB/VBA's developers Guide for Windows Installer. I have to yet read Phil's The Definitive Guide to Windows Installer. I am not sure if I could get hold of it in the near future, as it is not available in the local bookshops. But the bible or should I say the Gita of Windows Installer (MSI.chm) is my lifesaver any day.

It was pretty fun working with WIX and I believe that all setup developers must seriously consider this tool as an alternative to other commercial tools. There are a number of high-level utilities which help you automate most of the rudimentary tasks with the XML file but are still a fertile field for more development. There is currently no CodeDOM available for WIX so generating the WIX source is not as easy. But I believe that there are a couple of initiatives for the same. So for people who have a phobia for editing text files, WIX is NOT for you, atleast until the higher-level apps come in. The documentation is still skimpy but I believe that we should be able to see that changing shortly. WIX, however has a very active users community to extend help when you get stuck. As the tool is open-source, you can go ahead and fix a bug yourself depending on the criticality. There is a lot of scope for development for the tools like Tallow and sca.dll. Tallow.exe is a all purpose utility which does some rudimentary code gen, extracts self-reg entries, extracts registration information for assembles, process .rc files to create WIX UI fragments and the like. Sca.dll provides several custom actions like creating WebSites/Virtual directories, Users, run SQL Scripts, etc. Since the .wxs files are plain XML files, they are much easier to check in and check out than binary formats used by commercial tools like InstallShield. BTW, InstallShield does support XML format to store its project file but is nowhere close to the level of distributed application development functionality supported by WIX. Watch the video for more information about this. So if you are all set to download WIX and get running with it, jump here and click on the download link. You might also want to read an article about WIX on O'Reilly.

I just finished stealing UI from one of my InstallShield Basic MSI projects by "dark"ing (decompiling) the built MSI and cleaning up the WXS file and editing it down to size. I had the custom actions and InstallShield specific properties cleaned out and removed the branding. Thanks to the folks at Wix-Users mailing list, I have successfully separated the UI from my main product's installation and have documented the instructions to include the exact and to be used. I am still a developer so don’t expect me to write many lines of verbose comments. It is just a commented out block of code that you can cut and paste in the main WXS file. If you would like to have a copy of this .wxs file, please email me at vagmi.mudumbai@gmail.com.

Self-Registration vs. Registry tables group

COM Servers are every Setup developer's nightmare. One of my friends, a release engineer, expressed her extreme frustration over the technique that they had used to register COM Servers. The setup developer had written 100s of lines of code to launch RegSvr32 using the '/s' switch to register these COM Servers. In theory, the process of registration of a COM Server is perfectly fine when done via the regsvr32 utility. But there are certain disadvantages while doing that from the MSI Engineer's perspective. Firstly, all the dependencies of the server need to be registered in a specific order. And secondly, the user performing the installation should have administrative rights. The latter aspect makes setup developers and administrators loathe the registration process.

Further, system administrators do not know the registry settings and COM interfaces that are exposed by the COM object. COM objects do not support reflection and hence are essentially black boxes. Windows Installer has a 'SelfReg' table which can be used to self-register COM servers. There are several disadvantages while using this table.

• Order of registration cannot be specified. Thus if the DLL has dependencies and the dependencies are not registered already, the self-registration would fail.


• Rollback cannot be done reliably


• Self-Registration requires administrative privileges


• You cannot exploit the install-on-demand features of Windows Installer


• You cannot register EXE (Out of process) servers.
  

Tools like InstallShield X provide alternative methods to perform self-registration. This works around certain limitations like specifying the order of registration and registering EXE files. But largely there are limitations that are not in sync with Windows Installer's philosophy of a setup. Speaking of Setup Philosophy, RobMen's blog on his philosophical musings is really worth a read. I personally consider performing self-registration a fiendish act. Microsoft does not absolutely recommend it. Instead it recommends the use of Registry Table group. These are a set of tables like Class, ProgId, Typelib, MIME, Extension and so forth that can be used to store COM information. Most of this information should be available in the generated IDL file for the COM component. If you have any trouble figuring it out, you can peep into the registry during the registration process using various registry-spying tools and extract the appropriate information. Thankfully, tools like InstallShield X abstract this process. For example, using InstallShield X you can extract COM information from the key file of the component during the build time by setting the "COM Extract at Build" attribute of the component in the InstallShield X IDE. Although this makes life easy for a lot of setup developers, this technique has its share of pains. Although not perfect, this technique would solve most of the issues related with COM Registration.

InstallShield also provides a simple utility call RegSpyUI.exe that helps users look at COM information for a specific DLL or exe. This tool is undocumented, unsupported and hence I do not know all the bad things that it might do. But it is a really cool utility, which helps us work around such nasty limitations. Search the InstallShield X or DevStudio installation directory for this tool. There are a few references to this tool in the InstallShield Communities. So if something bad happens to your computer, don’t blame me. You have been warned.

Some of the motivating factors for authoring these tables as opposed to self-registration are:

1. You can forget about dependencies. Since all the information is already authored in the table, you do not need the COM servers registered in order.


2. You can register all kinds of COM Components (DLL, EXE, OCX and the like.)


3. You do not need administrative privileges. Since the registration is carried out by the standard actions, you can exploit MSI's support for elevated privileges. This is a boon for people handling locked down environments.


4. You can now support Installation-on-Demand for these COM Components. I would cover this in detail some other day.

With all these facts by your side, I am now sure that you can convince any setup developer to use the Registry Table Group instead of performing self-registration.

Reading Windows Installer Logs

As we all know Windows Installer is a complicated and a elegant technology. Its beastly complexities are beautifully abstracted by the authoring packages like InstallShield X, DevStudio and Developer. But we live in a less than perfect world and Murphy’s Law still holds good for the Windows Installer world too. There might be several reasons why an Installation might fail. Some errors might be meaningful while others might just spit out a cryptic error message with an error number during installation. The only way to make sense of these would be to look at the log files generated during installation. You can generate a log file for the installation using the following command line.

msiexec /i productname.msi /l*v package.log

This command generates a verbose log file logging all errors, warnings and debug messages and produces a huge log file in the text format. The first look at the log file might be slightly intimidating. This article tries to break the ice with the basics of reading the log file to troubleshoot problems with installation. Discussing all the error messages and its aspects is out of scope of this article. If you have specific questions, feel free to drop me a word. (at installneo@yahoo.com)

The Bottom Up Approach:

Whenever I read a log file, I start reading it from the bottom of the file. You might dismiss this technique as one of my eccentricities but I believe that this is the quickest, if not the most efficient, way of reading a log file in case of errors like 1603 or 1721. As a convention, the letter 'C' within brackets denotes the client context and the letter 'S' denotes the server context. So all messages in the log, which start as below, are messages in the client's context.

MSI (c) (A0:4C): Doing action: INSTALL
Action start 0:12:20: INSTALL.
MSI (c) (A0:4C): UI Sequence table 'InstallUISequence' is present and populated.
MSI (c) (A0:4C): Running UISequence

All messages in the log which start as below are messages from the MSI running in the server's context.

MSI (s) (1C:7C): Doing action: INSTALL
Action start 0:12:54: INSTALL.
MSI (s) (1C:7C): Running ExecuteSequence

Let us first analyze a successful log file and get ready to compare the anomalies with a not so perfect log file. I have used a simple Basic MSI project created with InstallShield X that just installs notepad.exe and I have disabled Update Ser. What we should be looking at is actions that do not have a return code of 1. You would find lines similar to the ones below just above the Server context property dump.

MSI (c) (04:B0): Doing action: SetupCompleteSuccess
Action start 18:03:47: SetupCompleteSuccess.
Action 18:03:47: SetupCompleteSuccess. Dialog created
Action ended 18:03:48: SetupCompleteSuccess. Return value 2.
Action ended 18:03:48: INSTALL. Return value 1.


Hmm... What does this mean? The "Action ended 18:03:48: SetupCompleteSuccess. Return value 2." message indicates that user cancelled the action or the operation was aborted due to a user instruction. But god knows that I did not instruct it stop. There must be something else that is wrong. The most obvious place to look for it would be the "Finish" button in the SetupCompleteSuccess dialog. You would find a line like the one below once you click on the Behavior tab of the dialog and select the OK control. If you insist on doing it via Orca, you can look at the ControlEvent table for the OK control in the SetupCompleteSuccess dialog.

EndDialog - Exit - 1

BINGO! We have found the culprit. The argument for the control event must be 'Return' instead of 'Exit'. Refer Windows Installer SDK help library article titled "EndDialog ControlEvent" for more information. Rebuild the project after changing it to 'Return' and you would find that none of the actions have return values other than '1'.

We have thus finally solved a trivial problem with the help of the log file, which would have gone unnoticed otherwise. Ah! Sweet Perfection.

:-)

Deferred Custom Actions:

One of the common places where users get confused is with deferred custom actions. Deferred Execution Custom Actions are not executed the first time the installer processes the custom actions in the Execute Sequence table. They are instead written to the MSI script. All these actions are executed within the InstallFinalize action. In these cases, you would have to look at the return value of the InstallFinalize action. A few lines above the line that indicates the return value of InstallFinalize would hint the action that failed. Let us verify this with a simple VBScript Custom Action that does nothing else but cause the install to fail </wink>. This is the code for the VBScript.

function MyFunction
MyFunction=1603
end function

I then create a Type 1030 custom action (VBScript Stored in Binary Table + Deferred Execution). This custom action is scheduled after InstallFiles action in the Install Execute Sequence. Run and log the installation. The installation proceeds and comes up with a dialog, which states that the Installation operation has failed. Let us look into the log for more information.

The final few lines in the log file complain that installation failed. But it still does have not enough information to identify an error.

MSI (c) (34:30): Product: SimpleInstaller -- Installation operation failed.

MSI (c) (34:30): Grabbed execution mutex.
MSI (c) (34:30): Cleaning up uninstalled install packages, if any exist
MSI (c) (34:30): MainEngineThread is returning 1603

Above the client contexts' property dump, the following lines of code indicate that INSTALL (Top level action) returns a value of 3 and the main thread is returning 1603. The SetupCompleteError is the same as we discussed earlier. This does not give much information either.

MSI (s) (DC:D8): MainEngineThread is returning 1603
MSI (c) (34:30): Back from server. Return value: 1603
MSI (c) (34:30): Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied. Counter after decrement: -1
Action ended 19:24:28: ExecuteAction. Return value 3.
MSI (c) (34:30): Doing action: SetupCompleteError
Action start 19:24:28: SetupCompleteError.
Action 19:24:29: SetupCompleteError. Dialog created
Action ended 19:24:30: SetupCompleteError. Return value 2.
Action ended 19:24:30: INSTALL. Return value 3.

Assuming that you have opened the log in notepad, search quickly for the word 'InstallFinalize' and specify the direction of search as up. You will now be able to see the following lines.

...
Our custom action is being called
MSI (s) (DC:D8): Executing op: ActionStart(Name=NewCustomAction1,,)
Action 19:24:28: NewCustomAction1.
MSI (s) (DC:D8): Executing op: CustomActionSchedule(Action=NewCustomAction1,ActionType=1030,Source=function MyFunction
MyFunction=1603
end function,Target=MyFunction,)
MSI (s) (DC:D8): Creating MSIHANDLE (3) of type 790536 for thread 2008
MSI (s) (DC:D8): Creating MSIHANDLE (4) of type 0 for thread 2008
MSI (s) (DC:D8): Closing MSIHANDLE (4) of type 0 for thread 2008
MSI (s) (DC:D8): Closing MSIHANDLE (3) of type 790536 for thread 2008
Our custom action fails causing the InstallFinalize action to fail
Action ended 19:24:28: InstallFinalize. Return value 3.
Initiate Rollback
MSI (s) (DC:D8): User policy value 'DisableRollback' is 0
MSI (s) (DC:D8): Machine policy value 'DisableRollback' is 0
MSI (s) (DC:D8): Executing op: Header
...
Rolls back changes

As you would have aptly guessed, the lines in bold are my comments. So you can nail most of troublesome deferred custom action using this technique.

I am really sleepy and have got some real work to do. I am sure I would touch on the log files again covering more aspects. Love it or Hate it, feel free to post your comments.

This is the beginning of....

I have just started this blog to document some of the interesting and difficult...ahem..challenging scenarios I faced while supporting people who package applications. I always wanted to write technical articles. I guess this would be a start.