Google wave is a failure

Google wave has created a lot of frenzy lately. It has people begging for accounts and people who have invites selling them at e-bay. Deja-Vu! Gmail reloaded? Remember when everybody went crazy for the invitations and a lot of them were sold on ebay at that time. Google kept the supply scarce to drive this even more, tapping on a very basic instinct of human nature: elitism. "Oh_my_god_you_have_gmail" was like Harry potter's forehead scar and people loved to flaunt it. Google wave's launch have been similar. But there is a difference between Gmail and Wave.

Gmail was/is an awesome product and path breaking in many ways. Google provided 1GB storage when yahoo gave 4MB! Not only that, compared to Yahoo's interface Gmail was light-years ahead. It did what every great design does i.e. get out of the way and let them do what they want to. Gmail had features which people needed and they used the right technology to deliver it. It was first free ajaxified email client(a major one at least) and allowed people to do everything without reloading the page.

Wave, on the other hand, is almost obnoxious. It is as if Google is trying very hard to show off what it can do instead of delivering what people need/want. For instance, that live typing thing. I hate that feature. It is like the unix "talk" on console back in college and I hated it back then too. I do not want people to see what I am typing. I write angry things sometimes which I never send and therefore I do not want people to read what I type. I have talked to people about it and everybody feels that real time typing is just overkill. If you think about it e-mails allow people to speak freely without the fear of others stepping on your feet. It also allows you to think about what you want to convey, double/triple check what you are sending, check facts etc. Google wave just kills all this in one go!

I am sure Google engineers/product managers know that but they are too obsessed with the technology to kill this feature. Or may be they do not realize it. That is a bigger problem which means they do not understand the domain problem well and do not know what problem they intend to solve.

Even if Google gives and option to switch it off, I do not think Wave is going anywhere in this avatar. Wave UI sucks: It is unintuitive, slow and generally not well thought out. So even if they give an option to switch off the instant typing thing, this is not so easy to fix. For instance, if there is a long wave and there are to new waves at different ends of the conversation, I do not know how to find them. The enter key doesn't work as expected. There are so many "unsynced waves" notification. I can go on and on and on.

Proactive India

I love the fact that this Government is very proactive:

Going after the naxals proactively
Naxals have been a big menace for this country. They started off as a movement to protect the interest of Poorest of poor. However it has mutated into a cult which promotes crime and law and order which in turns makes sure poores-of-poor remain just that. It was abut time that govt. capture/shoot all the big naxal ideologists and they did just that when they captured Kobard Ghandhy. And now comes the pounding of the naxal land. Hope this seals the fate of Naxals.

Dealing with Pakistan in a firm manner
Pakis have been very adventurous last year with Mumbai attack. India responded really well by making them pay at various fronts like diplomacy and threatening stability of Pakistan. India also managed to launch the long needed eye-in-the-sky. They were all set to launch this Israeli "eye" in 2006 when America intervened. This is a crucial realtime element which has been missing since the days of Operation Blue star.

Dealing with China
China has been feeling adventurous lately and that is not good for us. India has shown China that it is not the 1962 India and will not take things lying down. They are activating advance airstrips, deploying Mig 29s, Brahmos missiles, building more roads and infrastructure to make sure China understands clearly.

Keep up the good work

One line window pagination in Ruby

def getPages(current_page, minimum=1, maximum=20, window=2)
return((minimum+window<current_page ? minimum.upto(window).collect : minimum.upto(current_page+window).collect) + (current_page-window > minimum+window ? [".."] : []) + (current_page>minimum+window ? (current_page-window > minimum+window ? current_page-window : minimum+window).upto(current_page+window > maximum ? maximum : current_page+window).collect : [])+(current_page+window+1<maximum-window ? [".."] : [])+(current_page<maximum-2*window ? maximum-window : current_page+window+1).upto(maximum).collect)
end

Output:
>> 1.upto(20){|x| print x; print " : "; p getPages(x,1,20,1)}

1 : [1, 2, "..", 19, 20]
2 : [1, 2, 3, "..", 19, 20]
3 : [1, 2, 3, 4, "..", 19, 20]
4 : [1, "..", 3, 4, 5, "..", 19, 20]
5 : [1, "..", 4, 5, 6, "..", 19, 20]
6 : [1, "..", 5, 6, 7, "..", 19, 20]
7 : [1, "..", 6, 7, 8, "..", 19, 20]
8 : [1, "..", 7, 8, 9, "..", 19, 20]
9 : [1, "..", 8, 9, 10, "..", 19, 20]
10 : [1, "..", 9, 10, 11, "..", 19, 20]
11 : [1, "..", 10, 11, 12, "..", 19, 20]
12 : [1, "..", 11, 12, 13, "..", 19, 20]
13 : [1, "..", 12, 13, 14, "..", 19, 20]
14 : [1, "..", 13, 14, 15, "..", 19, 20]
15 : [1, "..", 14, 15, 16, "..", 19, 20]
16 : [1, "..", 15, 16, 17, "..", 19, 20]
17 : [1, "..", 16, 17, 18, 19, 20]
18 : [1, "..", 17, 18, 19, 20]
19 : [1, "..", 18, 19, 20]
20 : [1, "..", 19, 20]

Dunno if I should be proud of this or worried!

Dump those fixtures

I hate testing/debugging a rails application on "unreal" data. At the same time I find writing fixtures by hand is a big pain. I wrote this rake task to dump all the data in fixtures folder by (rake test:fixtures:dump) on a production environment, bring it to development environment and do a rake test:fixtures:load

desc "Dump all the data from DB to fixtures in YML format"
namespace :db do
  namespace :fixtures do
    task :dump => :environment do
      require 'active_record/fixtures'
      ActiveRecord::Base.establish_connection(Rails.env)
      base_dir = ENV['FIXTURES_PATH'] ? File.join(Rails.root, ENV['FIXTURES_PATH']) : File.join(Rails.root, 'test', 'fixtures')
      fixtures_dir = ENV['FIXTURES_DIR'] ? File.join(base_dir, ENV['FIXTURES_DIR']) : base_dir

      Dir.entries(File.join(RAILS_ROOT, "app", "models")).each{|filename|
        if /\.rb$/.match(filename)
          model = Kernel.const_get(filename.gsub(".rb", "").camelcase)
          if model.ancestors.include?(ActiveRecord::Base) and model.table_exists?
            puts "Dumping #{model}"
            file = File.open(File.join("test", "fixtures", model.table_name+".yml"), "w")
            counter = 1
            model.find_in_batches(:batch_size => 5000){|rows|
              rows.each{|row|
                file.puts "#{counter}:"
                file.puts row.attributes.to_yaml.split("\n")[1..-1].collect{|x| "\s\s"+x}.join("\n")
                counter+=1
              }
            }
            file.close
            puts " -- Created a file with #{counter} rows"
          end
        end
      }
    end
  end
end

Shootout: Jruby1.3.1 vs Ruby1.8.7 vs Ruby1.9.2 vs Python2.6 vs Java6

Click on the image to view it clearly

Jruby is the fastest among Jruby, Ruby and Python. Benchmark program was a fractal geometry program posted here.

Language	Time for 100 iterations	times slower than java with -server
java –server	0.18	1
Ruby1.8	7.78	44.07
Ruby1.9.2	4.2	23.78
Jruby	2.5	14.16
Jruby1.3.1—sever	2.31	13.1
java	0.18	1.01
python	3.04	17.21

cricketr

A few weeks ago I decided to quit my job and do something on my own. An idea, to let cricket crazy fans do commentary about anything and everything related to cricket, has been lingering on in my head for very long. I have been thinking on and off about this idea for about an year now. It is finally launched and the URL is cricketr. As such the would cricketr is a typo for cricketer but the "r" actually signifies that cricketr is a web2.0 site.
The idea itself is not very unique. It is basically a twitter clone for cricket. There are, however, a few very subtle but a few important differences.

1. Unlike twitter, it does cater only one domain and hence domain ontology can be used to make sense of a commentary. For instance, if I type "Sachin just hit a massive six", it can deduce that sachin is a player who belongs to Mumbai Indians(IPL context). Not only that, if a match is going on, or has just finished/about to start it will also identify the match and tag this commentary to that match tag. This way it is possible to generate crowdsourced commentary which is extremely fast. On a demo run today I found this to be about 2 minutes faster than cricinfo score and commentary updates and about 1 minute faster than cricbuzz's score and commentary updates.

2. Personal interaction is important but not the goal. Idea is to let users talk about cricket and derive knowledge out of it.

It is still in a very fluid state(it is basically a product of 4 days of almost 24hr marathon hacking sessions) and a lot of things need to be decided upon. Some big features planned pretty soon. Hope people will like it and do a lot of commentary on it :)

Fingers crossed!

Mongrel as a stand alone server

I have been using rails for quiet sometime now. Rails is easy but I feel stupid using it. It is bulky and slow for some of my use cases. I wanted something lean. I wanted something more geeky and powerful to handle lots of long running requests(more than 100K/day with avg. 30sec processing time).
I have been playing with mongrel handlers for about an year now and they work like a charm. In this post I will give a sneak peek into how to write a highly scalable back-end for doing real stuff. Mongrel is a very fast server and can take heavy load. So here we go writing our own mongrel server for heavy processing

require 'rubygems'
require 'mongrel'
require 'mysql'

PORT=4444
class LogHandler < Mongrel::HttpHandler
  def initialize
    @@mysql=Mysql.connect("host", "username", "password", "databse")
  end

   def process(request, response)
     response.start(200) do |head, out|
       logs = @@mysql.query("select * from huge")
       # Do some heavy processing on this data
       sleep 10
       # done
       logs.each{|row|
         out.write(row)
       end
     end
   end
end

config = Mongrel::Configurator.new :host => "0.0.0.0", :port => PORT do
  daemonize(:cwd => Dir.pwd, :log_file => "server.log")
  listener(:num_processors => 150, :timeout => 300) do
    uri "/", :handler => LogHandler.new
  end
   trap("INT") { stop }
end

config.run.join

This piece of code registers a url "/" on the machine on port provided (in this case 4444) and serves a log huge file. Not very developer friendly is it ?


Well we can use ERB along with it and that will make things look a little easier.
Lets add some more code to class LogHandler for ERB stuff.

class LogHandler < Mongrel::HttpHandler
   def initialize
      @mutex = Mutex.new
   end
# This is for making instance variables of this class available at the template
  def get_binding
    binding
  end

# We change the process function to render a rhtml file called view.rhtml
  def process(request, response)
     response.start(200) do |head, out|
       head["Content-Type"] = "text/html"
       logs = @@mysql.query("select * from huge order by id DESC limit 20")
       sleep 20 # Some heavy processing on logs      
       rhtml = ERB.new(File.read("view.rhtml"))
       @mutex.synchronize{
          @logs = logs
          out.write rhtml.result(self.get_binding) 
      }
     end
   end

Mutex lock is important to implement here as mongrel reuses instance variables of this class for subsequent requests and may lead to a race condition.

View.rhtml looks something like this

<html><body><title>Log console</title>
<h1>Last 20 hits on our page</h3>
<table>
        <th>IP</th>
        <th>URL fetched</th>
        <th>Came from</th>
        <th>time taken</th>
     <%@logs.each{|td|%>
<tr>
        <td><%=row[0]%></td>
        <td><%=row[1]%></td>
        <td><%=row[2]%></td>
        <td><%=row[3]%></td>
</tr>
<%}%>
</table>
</div>

Not bad right ? I can now render rhtml as I 'd do from a rails application.

But hey what about the Routing, MVC stuff, activerecord, logging, form helpers, javascript helpers, view side helpers, callbacks, migrations etc etc ? Well well! This is NOT a full scale framework or a Rails substitute. If you want to do view-side-heavy things use Rails or Merb. If you want to do processing heavy jobs which results in simple-html/no-html then use this.

That said we can very easily sneak in a few of the Rails goodies.

1. Active record - That is easy. Just require 'active_record'; establish_connection; create models by doing this

def User < ActiveRecord::Base
end

2. Logging
Rails does logging in two parts. Request logging and response logging. You may add callbacks in the 'process' function to log a request and reponse at start and end of the function respectively.
Something like

def process
   requestLogger(request)
   # Do stuff
   responseLogger(response)
end

3. Routing
This one much more difficult/the most difficult to implement. However if you have only a few urls to match and most of them are not dynamic it is a easier to hard code them. However it is dirty to do so. To implement a light weight routing is not that difficult and not so dirty. We take this path.

First step to implement routing it to get all the parameters. Both post and get parameters. You may use something like this:

  def post_params(request)
    post_params = {}
    request.body.readlines.first.split('&').each{|x|
      k,v=x.split('=')
      post_params[k.to_sym] = CGI.unescape(v)
    }
    return post_params
  end

   def get_params(request)
    get_params = {}
    request.params["QUERY_STRING"].split("&").each{|x|
      k, v = x.split("=")
      get_params[k.to_sym] = CGI.unescape(v)
    }
    return get_params
  end

Actual implementation of routing is a little complex and is not easy to cover in one blog post. Also I may have done it wrong so I do not want to put it out there. I will cover this in detail when I am sure about it.

The more stuff you add to this thing the more it will start looking like Rails! IMHO it is not a bad idea to implement your own framework. I did implement a framework to run rails code as it is. But I never used it on my production servers. Apparently that is the why how it should be :)

Update 1: I have changed some code to highlight that mysql query is not the heavy call and processing is being on done elsewhere (sleep in this case). Default mysql libraries are not thread safe however one may use something like Neverblock