Lisp Tutorial

Following Peter Seibel's request, anybody looking for a Lisp Tutorial should read Practical Common Lisp. It is the best Common Lisp resource for beginners currently available online.

Update: Made the post title a link and point it to the book.

I nuked my windows partition

I accidentally nuked my Windows partition. I am perform a clean wipe of my hard disk and installing Feisty as the only operating system on my laptop. Woohoo. :-)

Building Websites

One of my co-workers asked me today that he would like to build a website. A freewheeling discussion started from Google Calendar, software as a service, dynamic languages, web frameworks, open source and so on. But finally he asked me to send some material on building a dynamic website which is also cost effective. I hunted on Google and could not find a single resource that kind of captured the essence of our discussion and that results in this blog entry.

His idea was that he had a dance studio that wanted to get on the web. Basic stuff like events management, feedback, news and so on. A bit of e-commerce involving selling some salsa dance related goodies like dancing shoes and so on. So there it goes.

Domain Registration and Basic Hosting

If your site is a bunch of blog posts and set of static content, then you can get away with paying only for the domain. It would cost you about $10/year. Choose your domain registrar carefully and make sure that the registrar supports a good control panel to change your DNS entries as you wish. Some domain registrars require you to send a mail to them to change your CNAME records. And frankly, that sucks. NearlyFreeSpeech.Net supports a nice and elegant control panel to manage your domain entries.

Most blogging sites like Blogger allow you to host a blog on your own domain. One other thing that you have to do is to get Google Apps for your domain. That makes setting up email and calendar on custom domains really easy. Google Apps for your domain comes with Google Pages which are static HTML pages. Their page construction tools are not very helpful but some space is better than no space at all.

Cranking out your Web App

The next thing to do would be to crank out your web application. There are several choices you have. In fact, down right daunting. Since my good friend is a enterprisey kind of guy, let me place this advice right at the beginning.

Don't even try building a serious website with ASP.NET or J2EE. It would take forever and you are tying yourself down to a track facing an oncoming train. Chances of your survival are nearly zero. One of the hassle free solutions which would quickly get messy is to use some Content Management System like PHP Nuke or Post Nuke. This would help you get started very quickly but you cannot do much one you get started.

Alternatively you can crank your own PHP app with the Ruby on Rails wanna be frameworks like Cake or Symfony. I only suggest this because PHP hosting is well supported and easy to setup with a shared hosting partner. NSFN has some pretty cool plans. Check them out.

Alternatively, if you are ready to do some hard work, you can create a Ruby on Rails or a Django based web application. Both these frameworks are pretty good. I am not attempting to start a Ruby or Python war here. I have not hosted Ruby on Rails applications. But the word on the street is that Ruby on Rails deployment is a little more difficult than Python with mod_python. There are camps on either side throwing spit balls at each other. But I would like to refrain from commenting either ways. Look at it this way. Frameworks like RoR, Django, TurboGears (or is it Pylons??) help you achieve tremendous productivity. You will just have to swallow the pill and deal with deployment. Even though I love python and Django, I would have to commend the Ruby on Rails community for keeping change management in mind while designing the framework. You have stuff like Migrations and Capastrino to save several hours of upgrade headaches. I give two thumbs up for Ruby on Rails. :-)

If you are impressed by Django's admin interface capabilities. I would strongly suggest you to look at Streamlined. The amount of code you write to create a fully functional application would be considerably reduced by using these tools with prudence.

For static content, you must seriously consider Amazon's Simple Storage Service. It lets you park your static content for a very low fee. You would save tonnes of hassles if your hosting provider pulls you down for exceeding your bandwidth capacity.

If your target audience is small, I reckon that you might get away with paying about $10-$15 monthly and about $7-$10 yearly for the domain registration.

Suggestions for the Dance School

Now that I have scared you with the options and have left you dazed and confused, let me tell you how I would go about getting the dance studio on the web.

• Calendar - Google Calendar (no doubts about it). It has simple and clear API to integrate it with your application.


• Feedback - If the feedback is on particular articles, then you can simply use feedback articles on your blog. Google Calendar also supports comments.


• Static Content - Use google pages or Nearly FreeSpeech.NET's hosting. Its pretty cheap and is pay as you use.


• ECommerce - Now you are talking web-apps. I would strongly recommend Ruby on Rails. Throw the hosting question to the community. You will get tonnes of replies. Evaluate them and choose the best. Make sure that you ask the hosting provider about a reference customer so that you could validate your evaluations.

All the best to you all.

Getting f&#ked by OO write

Excuse my french. I am really irritated with Open Office and at the point of bashing my laptop screen in frustration. Here I am working my ass off on a business case at 1.45 AM and Ivor brings a PPT that he just found on the net which had some relevant information. My OpenOffice write crashed along with OO Present. I accidently said 'no' to document recovery (it was 1.45 am) and all my overnight work has gone down the drain. I have just downloaded Abiword. I need to get up early and work on it again. The good part is I am getting back to Bangalore tomorrow night. Cant wait to get back home. Can somebody write a decent word processor for Linux. This whole open office thing is a stupid farce. I already use Gnumeric for my excel related activities. I have very basic expectations from a word processor. I guess Abiword should suffice. I am sure it would be a lot faster than OO write.

Do you guys know a PowerPoint equivalent on Linux? I don't want to use stupid flaky open office and nor do I want to run Wine. If you guys have any other suggestions it would be great. BTW, Google! When are you building the offline stuff into your docs and spreadsheets. Are you going to have a presentation software?

Google Gears - Woohoo!

If you have not heard about Google Gears yet, check their homepage first. And in the very web way of getting excited, you can view a video presentation of Google Gears here.



There are other projects that are coming up on Google Code hosting related to Google gears. One of the more interesting ones is gears-dblib. You have other pointless projects like Tuiki. Nevertheless, it is good to get you up and running with Gears. Have fun coding. :-)

Not so angry response to "Eight Years Later...and Linux Still Sucks"

I read this (http://www.tipsoftheweekonline.com/linuxsucks.html) and this (http://www.tipsoftheweekonline.com/angry.html). First off, I don't believe people like rmerrick would have contributed anything of importance to the community. And they are definitely not the kind of geeks using Linux. They are the kind of people who intend to behave like geeks or nerds but end up behaving like jerks.

I understand the frustration that you experienced. And I can totally relate to that. But it is really not the fault of Linux. You faced exactly 3 problems

1) Monitor detection and configuration
2) Unable to play proprietary file formats
3) And problems with uninstallation which was due to the proprietary file formats issue. A problem nevertheless.

I agree that 1 and 3 are problems. Although the 3rd one does not qualify completely. Have you used synaptic? I always found the Ubuntu's Add/Remove should be replaced with Synaptic Package Manager. But telling that "Linux" totally sucks is very misleading. Your comment should instead have been "Ubuntu sucks for me.". I have been using Linux for the past few years and it has its fair share of annoyances. But guess what nothing beats the annoyance of a virus/worm/spyware on a Windows machine.

Let me share the incident which prompted me to switch to Linux. Note that I was always an open source fan for its economic reasons and I was okay doing open source on Windows. I am not one of the raving fanboy type of guys. I am not an illiterate computer user either. I've been working on these things since my school days. I always used to log on as restricted user in Windows for the fear that Windows might catch a virus. I diligently updated my antivirus periodically. I keep changing my passwords. I make sure that my Run/Runonce/Startup menu items are clean. I was kind of paranoid about it. But this one day, I got a media player with my korean DVD drive which required administrative access to run. Not to install... just run. So I used to run that in the adminstrative mode using the RunAs command. Safe - I would presume as this is the only application that runs with the elevated privileges. The player software when closed used to open an instance of IE to check for some stuff.. (for updates I guess) and close it in the background. Unknown to me, IE had somehow got infected and through that my whole system had been compromised. It affected all my executables on my drive. Some of my programs started behaving in a funny manner. I did not notice this until I ran python. The infected executable when run would actually run the virus and would create the actual executable image with a file name of extension .hwd and create a new process. I would have two processes running for each infected executable. The module names for the processes were executable.exe and executable.hwd. Before I knew it, the virus had completely blown away my windows installation. Not one of my executables were sane now. I tried cleaning them but PandaAV was not much help and it got infected. I forgot the name of the virus. :-( Thats when I decided that I would switch to Linux.

My initial experience with Fedora Core 4 was horrible. I had to do everything myself. RPM this. RPM that. Broken dependencies and what not. But I had been through hell with Windows which made this a cakewalk. But I ultimately got frustrated with Fedora and decided to try Ubuntu 5.04. I guess it was called Hoary Hedgehog. The installation was very pleasant and synaptic and apt-get was really cool. I upgraded to breezy after that and haven't looked back. So my statement would be "Fedora sucked for me" but "Ubuntu is great for me." I had no problem connecting to the universe and multiverse repositories to get the ugly plugins. The folks in the community are quite helpful unlike other communities that I have interacted with. You should probably try a linux distribution like Linspire. It is lot more polished than Ubuntu and would definitely solve your virus and worms problem. Linspire has legal codecs to play proprietary formats. You even get the benefit of real customer support. Before dismissing Linux of completely, I would suggest that you have a look at the existing distros, give it a spin and give a fair assessment.

I guess, I am really in the mood of ranting this evening.

Is piracy good for Microsoft?

I was watching this video on Cranky Geeks and they were talking about Romania and piracy. The story is covered here.

"Software piracy has many negative economic consequences, such as local software industries being crippled by competition with pirated software from abroad, and lost tax revenues and jobs from the lack of a legitimate market," said the BSA report.

"These costs reverberate up and down the supply and distribution chains."

I was not surprised. I find this it was extremely immature and stupid on the part of the government. I am from India and I can relate to this. The government and people here are equally immature and stupid. I believe that we are painting ourselves into a corner. What appalls me is the ease in which we can get pirated software vis-a-vis their open source counterparts. We can get pirated software from friends, or a walk on SP Road or any of the PC assemblers. But to get the latest versions of open source software and its documentation in terms of books is an extremely costly affair. If you don't believe me visit Gangarams or Sapna Book house and look at the price of linux distros there. I do not believe that technical superiority is responsible for the liberal presence of Windows operating system in Indian homes. It is due to its easy availability of pirated software. System assemblers know Windows better than Linux. The people who get it buy computers for browsing, surfing, watching movies and playing games. And not to mention the uncles and aunties fiddling with worthless astrology software that runs only on DOS (yes! you heard me. DOS!!!) and Windows. Probably we must make such astrology software for Linux and make it available on Live CDs for all these maamas and maamis to try it out.

I have some friends who are Indians living in India and yet buy Windows XP Home licenses. Can you believe that? They definitely are the minority. They are statistically insignificant. And some of them do have a Ubuntu installation safely tucked into one of their partitions. They were actually sold on the fact that Microsoft can actually track you down and sue you for using pirated versions of their software. But when that does happen, then the whole country has to be worried. IRCTC's website does not work well with Firefox. I book my tickets most of the time with IRCTC. Indian airlines' website does not work well with firefox. ICICI's website has just started working with Firefox. We are actually glued on to windows. I cannot find a single indian music site which works on flash audio like www.odeo.com or www.pandora.com. My favorite is www.musicindiaonline.com, which hosts an ActiveX control. I am not against Microsoft but I believe that it is not an economically viable alternative for us as a nation. I strongly believe that India is better off with open source software. Wine is not the solution. It will only worsen the situation. What we need is cross platform applications.

Microsoft will not bother about piracy in India as it is generating a huge talent pool and establishing a base of future customers. These customers are already trained on Microsoft software. So the organizations that hire them run Windows. Even the Java schools have a starter program for the computer illiterates and they start of with Microsoft Office, Internet Explorer and Outlook Express. Why don't they start of with Ubuntu or Linspire? Why dont they teach them the Google life. I am not against Microsoft or Windows. I am against the fact that we think it is smart on our part to pirate software and use it while all we are doing is increasing the liability on our employers and India Inc., as a whole. Open source software can solve most of these problems. With increased broadband penetration and web increasingly becoming a platform, we must get away from IE and ActiveX as far as possible have cross platform technology solutions.

Both Windows and Linux have their advantages and disadvantages depending on the kind of applications that you try to run. I feel Linux is better for me. It has been over 2 years since I switched to Linux as my primary operating system and I have never looked back. I wish others could see this in the same light as I do.

Single no more

There have been a lot of changes with my life lately. The most prominent one of all is that I am single no more. I am now happily wedded to my long time love Shanthi. I stole a few minutes off my busy schedule to post this .

I am going drop off the web for a couple of weeks now. Bye Bye web. Cya later.

6006l3 4 h4x0r

I just came across this. 1337 5p34k is a form of hacker language that was very popular on BBSes in the good old days. I did this in my college days. I was pleasantly surprised to find this page on Google. :-)

The Observer Pattern using Ruby Meta- Voodoo

The ruby standard library already has an implementation for the observer pattern. The implementation of the pattern was not very rubyish. It seemed a direct implementation of the Java equivalent of it. Ruby is far more powerful. This is the more rubyish version of the of the code goes something like this.

require 'observerext'

class Person
include Observer #include the observer related methods
observable :name,:address # the metavoodoo to create the accessor methods and fire events
end


admin=Person.new
admin.name='vagmi'
admin.address='admin@somesite.com'

# first observer
admin.observe :name do |old,newvalue|
puts %{The administrator changed from #{old} to #{newvalue}}
end

# the second observer
admin.observe :name do |old,newvalue|
puts 'there can be multiple observers'
end

admin.name='root'
#this prints the following statements
# > The administrator has changed from vagmi to root
# > there can be multiple observers

You can find the code for observerext.rb here. What I have basically done is to extend the Module class, which defines the observable method. The observable method injects the attributes' getters and setters. The setter method notifies the observers if there are any. The Observer module does the leg work of registering and unregistering the blocks and the notifyobservers method.

Update: Forgot to credit Greg Houston for explaining some of the ruby voodoo.

Moving back to blogger

I have moved back to blogger from my previous blog on geekswithblogs.net. I know that frequently changing the URL of a blog will hurt the traffic. But I have no other choice. Geeks with blogs admin application does not support unicode content and that really sucks. Apart from that fact, I have been working on phonetic layouts for both हिन्दी and தமிழ் on Ubuntu. I have just completed them . The first version can be downloaded here.

Most people like me are living the g-life. I use almost all the services that google exposes. Search, Calendar, Blogger, CodeHosting, Adsense, Docs and Spreadsheets, Picasa web, Youtube (although not technically a google service yet) and maps to name a few. Google has long since become the synonym for the word search. It is now pioneering in the applications space. Google Apps for your domain will give products like M$ Exchange and Lotus Domino a run for their money. It makes perfect business sense for SMEs to base their groupware on Google's infrastructure than maintain their own. This is of course given that you must have the kind of bandwidth and the infrastructure to remain connected and ensure that the business is not affected. But I do not see this as a huge problem. GMail exposes your inbox via POP3 and hence it is possible to have your mails both offline and online. Google Calendar can be used to sync up appointments. What is missing from Google right now is a Wiki. That would make it a complete groupware solution. Google would probably integrate its docs and spreadsheets application when they are mature with your domain. Google is taking web to the next level. And it is way more than Web 2.0. But let us refrain from naming it Web 3.0. Just imagine if we have enough bandwidth and google lets us host our databases on its infrastructure. I am sure google would like it. :-)

We are now using the Web as its creator Tim Berners Lee envisioned it to be. A readable and a writable medium. Web is no more a place for companies to advertise their business. It is now a formidable platform on which they conduct business.

Its midnight and I am very tired. Its about time I hit the sack. Good night world.

Rolling with RubyCLR

I downloaded RubyCLR today to play around with it and found that it was much more productive than IronPython. The reason is quite the same as what I mentioned in my previous blog entry. The Ruby-CLR bridge, which is essentially a library, is implemented as a native Ruby extension. So, this gives the developers the choice of using both C based modules and .NET based assemblies from the same script. And thus, in my opinion, RubyCLR is better than JRuby. We do not have to rewrite most of the native wrappers that are already written. In fact, some of them can be brought to .NET. I am not sure on how to embed Ruby as a scripting language for .NET applications. The other way is pretty clear from the samples that are distributed with the download.

Go ahead. Try it out. You will be surprised on how productive you become.

Madai Thiranthu - A really cool Tamil Hip Hop Remix

While I was browsing the net, I came across this amazing remix. As Yuvan is busy butchering his father's masterpieces in the name of remixes, this group of Malaysian hip hop artists have produced another masterpiece remixing the older master piece by Ilayaraja.

http://www.youtube.com/watch?v=uQSDr4609BU

When Cross Platform is not good enough...

I have been an avid fan of programming languages like C#, Python and Java as they take away most of the clunkiness of C/C++ code. Java, .NET/Mono, Python, Ruby and all other similar languages make claims that they are cross platform compliant. But the truth is, the library developer is bound to a specific technology.

For instance, the bzr version control system is written entirely in python. It has quite some promising features but it can never measure up to Subversion. Just imagine integrating bzr with Eclipse or using it with Visual Studio 2005. I am sure that I am not the first person to raise this as a concern. It is a part of the FAQ here. But my concern is not about the performance but the portability. It might, at the outset, seem as a funny argument as Python is supposed to be an extremely portable language so why should anyone have such a concern? The problem is not with the platform portability but with the technology portability. You would have to jump through several hoops before you can get bzr working with Java or .NET/Mono. Another argument might be that people could simply port bzr to Jython or IronPython. This might be possible with open source software but might not be the case with the closed source counterparts. There are a lot of good number of libraries written in Java, which unfortunately have to be rewritten in .NET. A testimony to this is the J* open source projects being ported to .NET and called as N*, like xUnit, xAnt, xVelocity, etc.

I understand that Java, C#, Python or Ruby make developers insanely productive. But library developers have to resist the urge to develop libraries in these languages and try to write their libraries in C/C++. By libraries, I also mean the tools and toolkits that are associated with them. Cross platform portability is definitely a concern. But it can be addressed but portable libraries like Boost or APR. You can later provide bindings to higher languages in the .NET stack, Java, Python or Ruby using interface generators like SWIG. There are numerous projects that have followed this approach and are hugely succesful. For instance, Subversion, wxWidgets, GTK , SQLite, MySQL are all glorious examples of toolkits and libraries that have core C/C++ components and have language bindings to higher level languages.

I am not saying that programming in Python or Ruby is bad. I absolutely love the way they make you productive. But high level languages are not the panacea for all programming problems. There are certain categories of software that ought to be written in C/C++ and not with higher languages. You can then write wrapper/helper libraries to make the access even simpler. But if the library developer ensures compliance across technologies, higher the chances of the library being usable and consequently popular.

Just my 2 cents on library development.

Configuration Data is such a Pain

I have finally got my computer to my new house in Bangalore. I dont have access to the internet, so I will essentially by offline. I am yet to install SharpDevelop and .NET Framework SDK v1.1. Hopefully, I should be able to get it done today. I had some time today morning but I was too busy playing NFS Hot Pursuit 2 and Quake 3. (No real souls to frag. Only bots. )

Currently one of our installation packages does something that it should not be doing. It is handling configuration data for several applications. Clearly this created several issues during upgrades. Now, for the next schedule major release, we (the packaging team) decided that we would let the applications handle the configuration information. This was duly presented as a proposal to other "application development" teams in a meeting. The people were a little hot under the collar during the meeting. Other development groups suspected that we were doing this to brush off our responsibility. I am sure that other setup developers would also have faced such similar predicament.

I was recently browsing through the InstallShield Community forums and found that mosth setup developers were invariably including configuration data as a part of their installation. I even saw one post where a setup developer had used a VBScript custom action with the FileSystemObject and tried to replace certain place holder texts in the installed configuration file. I am sure that many of us, who have burnt their fingers with VBScript and FSO, would agree with me. Logically speaking, there is nothing wrong with this approach. But we have to realize that we are introducing a element that can go wrong. Having deferred VBScript custom actions with FSO is a lot of fun while debugging.

My recommended approach would be to have a small configurator utility, which would accept command line parameters and perform the same tasks. This serves two purposes.

1. The configuration data is effectively taken out of the MSI. You can avoid slicing and dicing the CustomActionData property to get the desired values. There is one element less in the project that can go wrong.
   
2. The user need not run the installation, if he has specified incorrect configuration. He has the option of configuring the software after installing it. While installing on locked down environments, the setup author has to ensure that the permissions table is authored such that the current user has no problems writing on the the specified resources. This should'nt be a problem for registry keys or folders as their permission tokens can always be altered by the installer.
   

Finally, Riko is back. It seems that he is going to submit his tallow source to the community. I can't wait to get my hands on it.

My 2 cents

If people were following the the WIX-Users Mailing list, they would find that it has been a little hot recently. The first instance (as far as my knowledge goes) was with Michael creating a validation tool for the WIX Toolset called 'smoke'. I personally found it very useful as it could easily be included as a part of the build process. The output is in a structured XML format that makes it easier for us to crunch and build reports. But, some of the members in the group did not share the same level of enthusiasm. Michael seemed to be put off by that and included a disturbing update to his blog entry. Rob immediately acknowledged it and posted a blog entry about being thick skinned.

Recently, history repeated with Marc Bogers starting to ask about the ICE 33 warnings as Tallow populates the Registry table instead of the Registry Tables group. Firstly, Tallow is only a helper utility and is anything but perfect. It is useful but does not exactly produce usable WIX code. Heck, it doesn't even write to a file. Rob had said that ICE33 warnings can be ignored as it created some "strange behavior". More precisely, the resiliency associated with advertisement would prompt the end user for the media when Windows Installer detects some inconsistencies with the CLSID registration. Michael dismissed the argument that it arose from sloppy authoring and had nothing to do with MSI itself. Follow Michael's blog entry on COM registration for more information.

As we all know, Rob does not agree on a lot of things like inserting GUIDs while generating output with Tallow, using the Registry Tables Group and so on. I am sure that he has good reasons behind them. I am also sure that many people (including me), have modified Tallow to suit their requirements. Some people are using Riko's Tallow that does write to the Regsitry table's group using the Class and TypeLib elements. (On a totally different note, Riko, when are you going to release that code of yours?) I have a version of Tallow that creates components with GUIDs and names them appropriately. I am pretty sure that I am aware of the consequences and I have written code to persist the component information for future reference.

This freedom is the inherent beauty of open-source software. People can use their own ingenious imagination to tailor the tool to best suit their requirements. Of course, some of our opinions differ and we need a healthy debate to analyze these opinions and make the best of it. We should refrain from flaming and should try to embrace new ideas. Rob may be the BDFL for the WIX Toolset but I am sure that he would like the WIX Toolset to be successful and that helps developers create better installers without breaking much sweat. I tried hard not to write about this topic but after reading Michael's blog and comments with certain conspiracy theories, I wanted to voice my opinion too. I am sorry if I have hurt anybody's feelings.

Using an InstallScript application as the setup launcher

This, I guess, is my first InstallShield specific post. MSI is good in a lot of things but not when dealing with more than one package in a single transaction. MSI engine is designed to be run as a single instance and cannot be used effectively to install more than one package. Nested installations are possible but often lead to undesired consequences. Having a setup chainer is good but it involves writing code and handling reboots. So, for all who are using InstallShield DevStudio and up can benefit by using the InstallScript project as their setup launcher and monitor. We can effectively "script" a launcher application and have complete control over the installation process using the available libraries.

Why InstallScript....? I would not usually recommend InstallScript for any of the custom actions in the project, as the MSI-Scripting bridge is a little weak and has certain limitations. However, InstallScript libraries would cut down most of the code that you would have to write with a brand new C++ application. Also the scripting engine supports the silent mode. So you do not have to code separately for that either. But on the flip side, you would have an overhead of 1 Meg for the InstallScript engine. But this is okay considering the fresh set of bugs that you would introduce with the first version of the launcher application. For those not concerned about silent mode, you can check out the free open source setup launcher (dotNetInstaller).

The trick here is to use the InstallScript's program....endprogram syntax instead of the event based model. This makes sure that you do not register the application and does not call unknown events in between. Good old block scripting. <smile/> This also means that you need to take care of exceptions. If you are a setup developer from the good old InstallShield Professional 5.x days, you would immediately be at home with this kind of approach. So a simple script to just display a message box would be as below.

#include "ifx.h" //The standard header
//Our program block starts here
program
 MessageBox("Hello World!",INFORMATION);
endprogram

InstallScript engine just executes the statments present between the program and endprogram statements. As you can see, this approach gives you the flexibility of a complete scripting language with libraries custom made for creating installations. InstallScript projects are known for their flexibility but miss on the other aspects of MSI such as Elevated Privileges, Install on Demand, Source Resiliency and Self-healing.

Spying on Registry Entries

I am probably letting out the best kept secret of installation industry. I always used to wonder and have now discovered how most of the installation tools spy on registry entries that are created during COM registration or similar processes without actually affecting the build system. The spying program creates temporary registry keys for each of the registry hives HKCR, HKLM, HKCU, HKU and it maps the registry hives to these temporary registry keys. It then triggers the registration function which creates registry entries withing the registry keys specified instead of including it in the hives. I came across this revelation while I was wading through the source code for Tallow in the WIX toolset.

The core of this spying exercise relies on functions exposed by the advapi32.dll. The actual hive to key mapping is performed by the RegOverridePredefKey() function. The handle to the registry key is passed by using the RegCreateKeyEx() or the RegOpenKeyEx() function. After the mapping is done, you can invoke the DllRegisterServer() function after loading the library using the LoadLibrary() function. This mapping would be active for all the registry entries created by that particular process. So out of process registration for exe files may not directly work with this method. For the sake of simplicity, I am going to extract COM Interop settings from a given assembly. So let us write a simple console app in C# to do this. This app would perform the mapping and write the registry entries to a REG file and wipe out the key after the file is written.

Extracting registration information from a DLL file is similar but involve the importing other functions from Kernel32.dll. So I am giving that a raincheck now. You can download the Wix toolset's source package if you are interested. To start off with, let us write a class with static members to import the functions from advapi32.dll and create wrappers for them. Ensure that you have System.Runtime.InteropServices namespace included.

    class OverRideRegistry
    {
        //We first declare some stuff required. The are defined in winreg.h and windows.h
        public static readonly UIntPtr HkeyClassesRoot = (UIntPtr)0x80000000;
        public static readonly UIntPtr HkeyCurrentUser = (UIntPtr)0x80000001;
        public static readonly UIntPtr HkeyLocalMachine = (UIntPtr)0x80000002;
        public static readonly UIntPtr HkeyUsers = (UIntPtr)0x80000003;

        public const uint Delete = 0x00010000;
        public const uint ReadOnly = 0x00020000;
        public const uint WriteDac = 0x00040000;
        public const uint WriteOwner = 0x00080000;
        public const uint Synchronize = 0x00100000;
        public const uint StandardRightsRequired = 0x000F0000;
        public const uint StandardRightsAll = 0x001F0000;

        public const uint GenericRead = 0x80000000;
        public const uint GenericWrite = 0x40000000;
        public const uint GenericExecute = 0x20000000;
        public const uint GenericAll = 0x10000000;

        #region The Interop Import Stuff
        //we now import the functions exposed by advapi32.dll
        //Use RegCreateKeyEx to get handle to the openedKey
        [DllImport("advapi32.dll", EntryPoint="RegCreateKeyExW", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
        internal static extern int RegCreateKey(UIntPtr key, string subkey, uint reserved, string className, uint options, uint desiredSam, uint securityAttributes, out IntPtr openedKey, out uint disposition);

        //This does the actual hive to key mapping
        [DllImport("advapi32.dll", EntryPoint="RegOverridePredefKey", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
        internal static extern int RegOverridePredefKey(UIntPtr key, IntPtr newKey);

        //Like good programmers, we release our handles.
        [DllImport("advapi32.dll", EntryPoint="RegCloseKey", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
        internal static extern int RegCloseKey(IntPtr key);
        //Our interops are done :-)
        #endregion

        //Now we actually write wrapper functions to use the imported functions
        //Wrapper for creating Registry Keys
        internal static IntPtr OpenRegistryKey(UIntPtr key, string path)
        {
            IntPtr newKey = IntPtr.Zero;
            uint disposition = 0;
            uint security = StandardRightsAll | GenericRead | GenericWrite | GenericExecute | GenericAll;
            int error = RegCreateKey(key, path, 0, null, 0, security, 0, out newKey, out disposition);
            return newKey;
        }
        //Wrapper for the mapping
        internal static void OverrideRegistryKey(UIntPtr key, IntPtr newKey)
        {
            int error = RegOverridePredefKey(key, newKey);
        }
        //Wrapper for freeing the handle
        internal static void CloseRegistryKey(IntPtr key)
        {
            int error = RegCloseKey(key);
        }
    }

Now that we have the helper class with the static methods, we can define our main Console App class, which would use the functions in the helper class to register the assembly and steal the registry entries. Let us define the functions that Map and Map the registry hives. I had initialized the RegLocation to a string "Software\\Vagmi\\RegInterop\\". So the extracted registry entries would be put in the HKLM hive within the specified path. Also not that I have mapped HKLM as the final mapping. Else all keys that are subsequently created would be created within our registry key.

        void MapRegHives()
        {
            MapRegHive(OverRideRegistry.HkeyClassesRoot,this.RegLocation+"HKCR");
            MapRegHive(OverRideRegistry.HkeyCurrentUser,this.RegLocation+"HKCU");
            MapRegHive(OverRideRegistry.HkeyUsers,this.RegLocation+"HKU");
            MapRegHive(OverRideRegistry.HkeyLocalMachine,this.RegLocation+"HKLM");
        }
        void MapRegHive(UIntPtr key, string location)
        {
            Console.WriteLine("Mapping " + key + "  to " + location);
            IntPtr createdKey=IntPtr.Zero;
            try
            {
                createdKey=OverRideRegistry.OpenRegistryKey(OverRideRegistry.HkeyLocalMachine,location);
                OverRideRegistry.OverrideRegistryKey(key,createdKey);
            }
            catch(Exception e)
            {
                Console.WriteLine("Caught exception: " + e.Message);
            }
            finally
            {
                //close key like responsible programmers
                OverRideRegistry.CloseRegistryKey(createdKey);
            }
        }
        //Functions to unmap registry hives.
        void UnMapRegHives()
        {
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyClassesRoot,IntPtr.Zero);
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyCurrentUser,IntPtr.Zero);
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyLocalMachine,IntPtr.Zero);
            OverRideRegistry.OverrideRegistryKey(OverRideRegistry.HkeyUsers,IntPtr.Zero);
        }

I then parse two command line arguments one for the DLL and another for the REG file to export. I then pass these as constructors to my class, which calls these functions.

        public RegSpyCOMInterop(string AssemblyPath, string RegFile)
        {
            try
            {
                Assembly a=Assembly.LoadFrom(AssemblyPath);
                RegistrationServices regServices=new RegistrationServices();
                //map hives to registry keys
                    MapRegHives();
                    //Register assembly for COM interop
                    regServices.RegisterAssembly(a,AssemblyRegistrationFlags.SetCodeBase);
                    //Unmap hives
                    UnMapRegHives();
                WriteToRegFile(RegFile);
            }
            catch(Exception e)
            {
                Console.WriteLine("Caught Exception : " + e.Message);
            }
        }

The WriteToRegFile() function launches regedit and exports the hive that contains our keys. It then uses simple text replacement to change the values of the exported file to make the REG file functional.

        public void WriteToRegFile(string regfile)
        {
            RegistryKey r=Registry.LocalMachine.OpenSubKey(RegLocation,false);
            PathToDelete=r.Name + "\\";
            Process pr=new Process();
            pr.StartInfo.FileName="regedit";
            pr.StartInfo.Arguments=" /e " + regfile + " " + r.Name;
            pr.Start();
            pr.WaitForExit();
            string line;
            StreamReader reader=new StreamReader(regfile);
            StreamWriter writer=new StreamWriter("TempFile.txt");
            while((line=reader.ReadLine())!=null)
            {
                writer.WriteLine(processRegistryName(line));
            }
            reader.Close();
            writer.Close();
            File.Copy("TempFile.txt",regfile,true);
            File.Delete("TempFile.txt");
            r.Flush();
            r.Close();
        }

        public String processRegistryName(string regname)
        {
            string newRegName="";
            newRegName=regname.Replace(PathToDelete,"");
            newRegName=newRegName.Replace("[" + PathToDelete.Substring(0,PathToDelete.Length-1) + "]","");
            newRegName=newRegName.Replace("HKCR","HKEY_CLASS_ROOT");
            newRegName=newRegName.Replace("HKCU","HKEY_CURRENT_USER");
            newRegName=newRegName.Replace("HKLM","HKEY_LOCAL_MACHINE");
            newRegName=newRegName.Replace("HKU","HKEY_USERS");
            return newRegName;
        }

There you have it. A command line utility to extract COM Interop Registry entries. Hope this answered some of your queries regarding registration without affecting the target system. You can find the entire source code at http://www.geekswithblogs.net/vagmi.mudumbai/articles/16581.aspx.

Microsoft's Whitepaper on Patch Sequencing

While browsing, I came across this excellent whitepaper on MSI 3.0 Patch Sequencing.

http://www.microsoft.com/downloads/details.aspx?FamilyId=AD7AC91E-2493-4549-AE6F-BF5E007C12A3&displaylang=en

Using WIX Toolset for Distributed Development

When I talk to my fellow developers and other installation developers about the WIX Toolset, they do not fully understand the implications of the WIX Toolset in its own right but map its features to their favorite installation development tools. The WIX Toolset mostly ensures that you follow the best practices while authoring packages. The biggest advantage of the WIX Toolset over other applications is the support for distributed development. I have raved and ranted about this in my previous blog entries but I would substantiate it with actual WXS code this time. For starters, WXS file is an XML file that has to follow the schema as specified in the wix.xsd file in the 'doc' folder of your distribution. Rob has an excellent demo on distributed development with the WIX toolset. But the quality of video is very poor and the sample files are not yet available for us to tweak around with. So I have come up with my own example.

Let us assume the following scenario. There are group of installation and application developers in the organization working on a setup project. Let us allocate roles (purely hypothetical. I am not a manager <grin/>) for our convenience. Let us assume that the application developers take care of maintaining a catalog of components and resources that go into the product. The second developer takes care of assigning the appropriate components into the respective merge modules or features in a product. Let another developer be responsible for creating UIs. Additionally, each application developer creates his own fragments of files by hand or by using a simple tool. Let us assume that we have to package three files.

1. Notepad.exe


2. ReadMe.txt


3. Calc.exe

Notepad.exe and Readme.txt are a logical unit and hence we would put them in a single component called Notepad.exe and set Notepad.exe as the key file of the component. We would also have Readme.txt as a companion file for Notepad.exe such that the versioning logic of Readme.txt is controlled by the versioning logic of Notepad.exe. Calc.exe is a standalone application that does not have anyother files. So here we go. The application developer codes his file named Dev1.wxs. Pretty simple isn't it?

<!--Dev1.wxs-->
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
  <Fragment>
    <DirectoryRef Id="INSTALLDIR">
      <Component Id="Notepad.exe" Guid="12CC008F-89A6-422d-868B-B066606FFD99" DiskId="1">
        <File Id="Notepad.exe" Name="Notepad.exe" LongName="Notepad.exe" KeyPath="yes" src="$(env.BUILDPATH)\Notepad.exe">
            <Shortcut Id="LaunchNotepad" Advertise="no" Description="Launches Notepad" Directory="DesktopFolder" Name="Notepad" LongName="Launch Notepad" WorkingDirectory="INSTALLDIR" Icon="Notepad.ico"/>
        </File>
        <File Id="Readme.txt" Name="Readme.txt" LongName="Readme.txt" CompanionFile="Notepad.exe" src="$(env.BUILDPATH)\Readme.txt"/>
      </Component>
    </DirectoryRef>
    <Icon Id="Notepad.ico" src="$(env.BUILDPATH)\Notepad.ico"/>
  </Fragment>
</Wix>

Now the second application developer creates his file dev2.wxs.

 

<!--Dev2.wxs-->
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
  <Fragment>
    <DirectoryRef Id="INSTALLDIR">
      <Component Id="Calc.exe" Guid="2C5ECB67-E585-4301-BAF4-5380FE6C26AB" DiskId="1">
        <File Id="Calc.exe" Name="Calc.exe" LongName="Calc.exe" KeyPath="yes" src="$(env.BUILDPATH)\Calc.exe"/>
      </Component>
    </DirectoryRef>
  </Fragment>
</Wix>

By now, the UI guy would have created a new WXS file for UI or stole the UI from a premade package. I normally steal UI from premade(I just coined this word) packages like Orca or some UI generated by freeware installers. Writing UI code for MSI using the WIX toolset is not exactly a pleasurable experience. <sigh/> So here goes the UI file. I am not going to include the WXS file here but if you need it, you can send me an email at vagmi.mudumbai@gmail.com. Now we have a file called dialogs.wxs which just has a bunch of dialogs, binaries, error texts, UITexts and properties defined for the UI.

Now we have to piece together each of these files and link them to our product. This is the MyProduct.wxs file which contains the feature-component mapping and the rest of the installation logic.

<!--MyProduct.wxs-->
<Wix xmlns="http://schemas.microsoft.com/wix/2003/01/wi">
  <Product Id="148E08FF-D7C4-46ed-8D4D-601C67FE0AFD" Language="1033" Name="MyProduct" Version="1.0.0" UpgradeCode="B7FE793A-800D-4c14-8CB4-B00AA84FF685" Manufacturer="Vagmi">
    <Package Id="D1192FCD-BA01-4d8f-BA7B-663CE9934BDE" Compressed="yes" Description="My WIX Installation" InstallerVersion="200" Languages="1033" Manufacturer="Vagmi"/>
    <Media Cabinet="Data.cab" EmbedCab="yes" Id="1"/>
    <Directory Id="TARGETDIR" Name="SourceDir">
        <Directory Id="ProgramFilesFolder" Name="PFiles">
            <Directory Id="INSTALLDIR" Name="Vagmi"/>
        </Directory>
        <Directory Id="DesktopFolder" Name="DTFOLDER"/>
    </Directory>
    <UI>
        <InstallUISequence>
            <Show Dialog="SetupCompleteError" OnExit="error" />
            <Show Dialog="SetupInterrupted" OnExit="cancel" />
            <Show Dialog="SetupCompleteSuccess" OnExit="success" />
            <Show Dialog="SetupInitialization" After="LaunchConditions" />
            <ResolveSource Before="CostFinalize"><![CDATA[Not Installed And Not PATCH]]></ResolveSource>
            <Show Dialog="InstallWelcome" After="MigrateFeatureStates"><![CDATA[Not Installed And (Not PATCH Or IS_MAJOR_UPGRADE)]]></Show>
            <Show Dialog="SetupResume" After="InstallWelcome"><![CDATA[Installed And (RESUME Or Preselected) And Not PATCH]]></Show>
            <Show Dialog="MaintenanceWelcome" After="SetupResume"><![CDATA[Installed And Not RESUME And Not Preselected And Not PATCH]]></Show>
            <Show Dialog="SetupProgress" After="MaintenanceWelcome" />
        </InstallUISequence>
    </UI>
    <Feature Id="TheOnlyFeature" ConfigurableDirectory="INSTALLDIR" Level="1" Title="TheOnlyFeature" Description="TheOnlyFeature">
        <ComponentRef Id="Notepad.exe"/>
        <ComponentRef Id="Calc.exe"/>
    </Feature>
  </Product>
</Wix>

As you can clearly see, the complex job of creating an installer package is split among the developers. This is a great feature as it does away with Merge Modules and all the associated hassles with modularization. Further changes or corrections can be made independently without affecting other parts of the installation. The only part that needs automation is the generation of WXS fragments for the developers. Although it is a simple piece of code to write, we cannot expect all developers to be conversant with XML and learn the WIX toolset. All we need is for them to specify the files and the destinations in a tool and a tool that should generate minimal WXS code for these fragments. I do not think that should be a problem considering the level of automation installation and build teams adopt for other tasks.

Building these is a straight forward task.

• Compile the files using the following command.
  candle dev1.wxs dev2.wxs dialogs.wxs myproduct.wxs
  
  This would produce dev1.wixobj, dev2.wixobj, dialogs.wixobj and myproduct.wixobj


• You can then link the generated object files using light.
  light -out build/product.msi dev1.wixobj dev2.wixobj dialogs.wixobj myproduct.wixobj

The myproduct.msi file would be generated in a folder named 'build' under the current directory.