Markov Chains

You know developers have an itch to implement something cool that they learned about. No can say it better than XKCD.



I learnt about Markov Chains today.

In mathematics, a Markov chain, named after Andrey Markov, is a discrete random process with the Markov property. A discrete random process means a system which can be in various states, and which changes randomly in discrete steps. It can be helpful to think of the system as evolving once a minute, although strictly speaking the "step" may have nothing to do with time. The Markov property states that the probability distribution for the system at the next step (and in fact at all future steps) only depends on the current state of the system, and not additionally on the state of the system at previous steps. Since the system changes randomly, it is generally impossible to predict the exact state of the system in the future. However, the statistical properties of the system at a great many steps in the future can often be described. In many applications it is these statistical properties that are important. - From the wikipedia entry on Markov Chains

It was so cool that I decided to do something with it. So I wrote a simple helper that generates articles for the app that I was testing. Here is the code in all its glory. It is not the fastest piece of code and is a memory hog, but who cares?

	module Markov
	class TextGenerator
	attr_accessor :markov
	attr_accessor :input_file
	def initialize(input_file)
	@input_file=input_file
	end
	def get_words
	content=File.read(@input_file).split(/[\.\?]/)
	content.each do |sentence|
	sentence.strip!
	before = ""
	sentence.split.each do |word|
	if(word.strip != "")
	yield before, word
	before = word
	end
	end
	end
	end
	def generate_markov
	@markov = {}
	if(File.exist?(@input_file))
	get_words do |before, word|
	markov[before] ||= []
	markov[before] << word
	end
	end
	@markov
	end
	def generate_paragraph(len=20)
	sentences=[]
	len.times do
	sentences << generate_sentence
	end
	sentences.join " "
	end
	def generate_article(len=10)
	paragraphs=[]
	len.times do
	paragraphs << generate_paragraph
	end
	paragraphs.join("\n\n")
	end
	def generate_sentence(len=20)
	if(not @markov)
	generate_markov
	end
	words = []
	before = ""
	loop do
	before = @markov[before][Kernel.rand(@markov[before].length)] rescue nil
	break if before.nil?
	break if (words + [before]).length >= len
	words << before
	end
	return words.join(" ")+"."
	end
	end
	end

view raw markov.rb hosted with ❤ by GitHub

This is how you use it.

	require 'markov'
	puts Markov::TextGenerator.new('alice.txt').generate_article

view raw usage.rb hosted with ❤ by GitHub

And this produced this fine article. Isn't this cool?

The problem with certifications

There was a question on ILUGC today to which I responded. I am reposting it as a blog entry.

You should first understand how certifications work from the perspective of a vendor who offers such certification say Microsoft, Redhat or SAP. The vendor has an economic interest in selling his product to a company. The company which buys a product (or a service in case the of Redhat) wants to know if the product/service they buy can be used by people they are going to employ. This is where the brilliance of marketing sets in.

They con the students and the candidates in the job market that there are several jobs awaiting them if only they clear the certification in their technology. They then go and talk to all the colleges and institutes like NIIT and convince them to market the certification and give them the necessary materials to make it a course. They then contact book authors who fill in pages and pages with mind numbing facts about the technology. The institute in the meanwhile has started tailoring a course and claims that it can get you from a dud to a certified professional in 3 months. As with any statistical distribution in a country with a billion people, a sizable few take up the certifications. The first few batches are always tough. Only the good guys clear the first 3 or 4 batches. But good guys like to share knowledge. So they share the dumps with the world.

The vendor does nothing to oppose this as they need that pool of skills in the market. After they have got enough number of people with the piece of paper with the name of the vendor and a printed signature of the CEO/founder of the company, the marketing department gets busy making powerpoint slides. The slides have graph of stating the number of people who have finished the certification and how easy it would be for their customers to find the right skills. And since the numbers (supply) are aplenty and the licensed customers are a few (demand), by the laws of economics, they should have no problem in finding people with the right skills (and a cheap pay).

The mindless HR then goes and "recruits" people with certifications. So if you notice, if you are a good candidate, you can lower your value just by getting certified. There are alternative means to improve your employability. You can start contributing to open source for a start. Do not pick difficult projects. If you cannot find a project, try building something. Try creating a simple webapp or a game. I am seeing too many freshers with "Biometric AI-Driven Neural Network Enabled Grid Based Path Finding World Domination Engine" on their resumes. And they want me to believe that they did it in 3 months without being able to write fizz buzz in their favorite language.

As an employer myself, I would recommend you to do the following. Take up a small project. A small pong game or a simple webapp that solves your problem and document all your learnings in a blog. This will make you a better writer and a better programmer. Use open source technologies. Apart from being the best choice in terms of freedoms that you get, you also get to interact directly with better programmers in mailing lists like these.

To pioneer a change, I, on behalf of Artha42, am ready mentor a select group of students every year. Your work will be entirely open sourced. I promise that you will not write a single line of proprietary code. That said, we are not Google. We are a small startup with limited resources. We cannot hold something as large as the Google Summer of Code. But we can do something smaller. If you are interested in such an internship opportunity, do drop in a line to careers artha42 com with "internship" in the subject line or visit our careers page.

IE is being mean to me again

Exactly my sentiments.



BTW, Artha42 is also hiring talented and passionate developers. Drop in a like to careers artha42 dot com, if you are interested.

Artha42 is hiring

Artha42 is hiring. I have been conducting a lot of interviews lately and am completely dissappointed at the state of freshers coming out of engineering colleges. They spend 4 years supposedly studying IT and Computer Science and cannot answer a simple question like fizzbuzz. Come on. I completely agree with krishashok on this one. Chennai does have concentration camps and they are the bane for a creative society.



If there are guys wanting to work in a fun, passionate and challenging environment, please send your profile to careers artha42.com.

Modifying Radiant's Database Mailer

Radiant's Database Mailer is an wonderful extension that can quickly set up mailer forms and add stuff to the database too. But they have tried to do some serious magic to ensure that arbitrary structures can be imported by defining them in the config/environment.rb. That is a bit too hacky for Rails to handle.

If you decide to change the structure any time later, you cannot migrate those changes. That is quite a bummer. The crazy part is you cannot uninstall Database Mailer after that either. It keeps reporting an "undefined method set_schema_version" error. To uninstall it you have to manually nuke the form_datas (isn't data already plural?) table and related attachment tables. You also have to remove the migrations in the schema_migrations table manually. Can someone suggest a better alternative?

Getting to tolerate Radiant

I have been working with Radiant for the past couple of days. It is definitely impressive. But I simply hated the fact that you need to work within the confines of the browser's textarea to fill in content. This is just plain lame. I am so used to Vim that typing within that small window is just plain irritating. You can of course copy paste stuff from and to vim but that is equally lame. Luckily there are a couple of ways you can work around this.

The first being the "Its all Text" extension. Its all text enables me to type anything that I type in a text area in vim. It is actually pretty cool.

The second being radiant-file-system-extension. It allows me to use the file system and version content with git. Happy hacking with radiant.

Git Branch on the command prompt

It is sometimes very useful to know the git branch you are working on right from the command prompt. You can do this by editing the ~/.bashrc file.

	# snip .bashrc
	function bash_git_branch
	{
	git branch 2> /dev/null | grep \* | python -c "print '['+raw_input()[2:]+']'" 2> /dev/null
	}
	#snip .bashrc
	PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w$(bash_git_branch)\$ '
	#snip the rest

view raw gistfile1.txt hosted with ❤ by GitHub

This will show up a prompt which looks like this and fails gracefully when you are not on a git repo.

vagmi@deepthought:~/work/testproj[master]$ git checkout new_feature
Switched to branch "new_feature"
vagmi@deepthought:~/work/testproj[new_feature]$ cd ..
vagmi@deepthought:~/work$

This saves a lot of "git branch" when you are coding.

Liquid and Sinatra

Liquid is a templating language used when you need safety for your templates. I was planning to build a CMS and Liquid fit the needs of a templating library really well. I also like Sinatra for its simplicity. Unfortunately, Sinatra did not have support for liquid yet so I went ahead and built one. You can get that from my liquid topic branch on github.

There are few significant changes that you need to care about when using liquid instead of ERB or HAML.

• None of the local members or class members declared will be available in your template


• The locals hash is respected even for the layout.


• You have to use {{ content }} in your layouts, instead of yielding.

And finally a "hello world" sinatra app with liquid.

	$:.unshift File.join(File.dirname(__FILE__),'lib/sinatra/lib')
	require 'rubygems'
	require 'sinatra'
	use_in_file_templates!
	get '/:name' do
	params['title'] = 'Sinatra and Liquid rocks'
	liquid :index, :locals => params
	end
	__END__
	@@layout
	<html>
	<head><title>{{title}}<title></head>
	<body>
	{{content}}
	</body>
	</html>
	@@ index
	<h1>{{ name }}</h1>

view raw sinatra_liquid.rb hosted with ❤ by GitHub

Do let me know if you like it.

Star wars a cappella video

I just have two words for this. Bloody Brilliant.

Lets build something already

Have ever had the feeling that when you wanted to get started with something, something else had to be done first. If you are a lone hacker banging out code, you are probably fine as you know what you are building. You do not ever have to write it down. But in teams it is vitally important that everyone knows what they are actually building.

This is where product managers and project managers come in. They are supposed to have a holistic view of where the project is and where it is heading to. They are the communication hubs who deal with all sorts of information about a project and protect the team as virtual filters. The team then gets information from a single source. But you might ask what happens when there are multiple project managers and product owners but no developers in a planning meeting. The meetings tend to be something like this.



Fortunately, there is an easy way to deal with this. Get building something already. It is far easier to figure out if the presented solution fits your needs than to speculate on how it would look like. So to all the teams that are suffering from this analysis paralysis, this is my advice. Stop discussing and start building.

An interview with Ayn Rand

Objectivism is very relevant, especially in today's context.

Part 1

Part 2

Part 3

A video on the monetary system

Apart from the conspiracy theory part in the end, this is by far the best explanation of the current monetary system.

India, say hello to Corporate Greed

I was browsing the net about the Satyam fiasco when I saw this. What infuriated me were the comments on the blog. They are asking people to stick to the company. This is the time the talented people should extend their middle finger and say FUCK YOU to the management. Make sure you enunciate that properly. It works best if told in groups. Tell it with enough intensity to make them die in shame.

Stay with Satyam... Are you crazy?? Jump the ship now. Don't commit a career suicide. If the man had an ounce of dignity he would not have jeopardized the life of so many people. Indians do not have social security. If a guy in India does not have a job, he is literally on the streets.

Ramalinga Raju was sure as hell not maintaining family accounts. The CFO, other senior managment and the financial/controlling staff would have known this. I am sure PWC would have spotted this and would have been duly silenced. Corporate Greed has entered India.

If you are a good programmer, you would not have joined Satyam in the first place. But if you are there, jump now. You may not get a raise, but what the hell. You can get out of the depressing situation and have a chance a save your soul.

The following days to come in an organization like Satyam would be extremely depressing and would be enough to demotivate a motivated soul. It would be full of bureaucracy where mediocrity will reign supreme. Sounds like a fun place to work?? Hell no. If you you are a rockstar coder, this is your chance.

Get out of the hell hole now.