Microfinance as a business

Microfinance as an industry has started to grow up. However it is not all rosy. An impending IPO of SKS has added more fuel to the naysayers calling MFIs yet another loan sharks, sum-prime like bubble, profiteers and what not.

A lot of these are coming from western media like WSJ who have started covering MFI sector almost every week and obviously a lot of it is negative. They generally pick up on small village and show how MFIs have flushed them with loans or how they have all defaulted as a community. A lot of it, I think, is plain BS! These Americans have no freaking idea of whats happening on the ground. I think what western media clearly fails to comprehend is the sheer size of the market and diversity of this country...or they don't want to. Paint a bleak picture and their readers would get some kind of sadistic pleasure out reported failing of yet-another-poverty-elimination-experiment.

Viability of Microfinance: A thought experiment

Microfinance institutions(MFIs) lend to poor people who actually do some 'real' work and repay it back to MFIs overtime. For this MFIs charge a good amount of interest rate. Now lets take an example of a farmer taking a loan for cow. A good cow(not a jersey) which costs about INR 10,000 gives about 10 lts of milk. This in rupees terms comes to about INR 80 per day. Even if we take 20% of this as costs of medicines etc and assume that a cow gives milk only half the year then he earns about INR 30 or per week INR210. Now a typical MFI loan of Rs.10000 with 50 installments has an EMI of about Rs. 230. I don't understand who it is difficult for a farmer to service this loan when cows are just a side business for a farmer. Moreover he'd get good returns on his 10K loans after he has paid of the loan in one year.

Why scaling up is a necessity for MFIs

Scaling up is not a luxury but a necessity. Those who do not scale their operations cannot ever make significant dent in the bleak universe of poverty. A typical MFI starts with about 100 clients and quickly grows to about 1000 clients. The typical loan size is about 10000 which means the amount of money disbursed is about 1 crore INR. That is a large village or a small town to whom they can reach. After this they start to stagnate in need of better management, IT, manpower etc.

India has 110 crore people. 30-70% of which are in dire need of funds to scale up whatever they do and become "non-poor". The market size is about 100 million clients. If an MFI just reaches 1000 people then it is not doing enough. In my view when they reach about a million clients then all they have captured is 1% of the market which still is clearly not enough. Even if we have 100 such players still there would be room for more MFIs to service cycle 2 and 3 loans of these people. In India MFIs of size serving more than a million customers was 3 according to this report in October 2009(page 11). Therefore saying MF sector is a bubble is a joke! There is a clearly a lot of room to grow.

There are many more challenges for smaller MFIs. Fraud by their employees due to weak monitoring, difficulty is securing credit lines, higher cost of loans etc. Larger MFIs with good governance can tide over these difficulties with ease.

It is About time we started looking at MFIs as proper for profit businesses. We have reached a point where MF sector needs to kick off the training wheels and switch to corporate model and government should respond with creating solid guildlines and rules for this sector. RBI has very clearly shown that with the banks and now is the time that they do it for MFIs/NBFCs. They also need good governance and that needs talent. Without money this sector cannot bring in talented people. This sector needs good IT, solid reporting tools, CIBIL like credit bureau, professional consulting firms etc. All of this can only come along if these MFIs start scaling up, bringing to the table economics of scale to pay off corporate salaries and put in good solid systems.

Profits not extraordinary profits

What Md. Yunus started was a NGO like Bank for poor but clearly the self governing, non profit model is not for everybody and that model cannot scale for everybody. Not everybody is Md. Yunus. He has very clearly shown us a path. Not everybody is a as talented or motivated. Moreover greed is the primary reason why farmers have suffered for so many years. Allowing it all over again would be foolish. This is a capitalist economy and everybody needs to earn his/her bread. If poor are benefiting from the loans MFIs provide them and earn profit on it then why should MFIs not be allowed to earn some profit. Having said that extra-ordinary profits is something what good RBI regulation should prevent from happening.

Note: This post is about what I think & feel and not of my employer.

Featuring MOSTFIT

I have been working on Mostfit for about 6 months now. It is an open source MIS (management information system) for MFIs(Microfinance institutions). For the uninitiated MFIs provide loans to the poor without any collaterals. It is a social lending model where a group takes a joint responsibility of timely repayment of loans. MFIs are almost like the infamous money-lenders with much larger scale and lower interest rates. This idea was first started by Md. Yunus of Bangladesh who later received Noble peace prize for it.

MFIs now have started to scale up and have started adding hundereds and thousands of clients to their books every week. Most of the MFIs are managed by excel sheets and due to the size of operations it quickly starts becoming a problem for them. Let me give you an idea of this size: SKS microfinance which arguably is India's largest MFI has about 1 crore clients and about Rs. 4000 crore in loan outstanding. They are even going for an IPO very soon.

This is where MOSTFIT comes into picture. It is designed to suit the needs of a diverse model MFI with many loan products. It is built from grounds up to be completely flexible. It is written in ruby, datamapper, haml and rspec. Code resides here. We even provide an instant MIS instance on http://mostfit.in

So any MFI out there looking for a MIS should give it a try at http://mostfit.in

Google wave is a failure

Google wave has created a lot of frenzy lately. It has people begging for accounts and people who have invites selling them at e-bay. Deja-Vu! Gmail reloaded? Remember when everybody went crazy for the invitations and a lot of them were sold on ebay at that time. Google kept the supply scarce to drive this even more, tapping on a very basic instinct of human nature: elitism. "Oh_my_god_you_have_gmail" was like Harry potter's forehead scar and people loved to flaunt it. Google wave's launch have been similar. But there is a difference between Gmail and Wave.

Gmail was/is an awesome product and path breaking in many ways. Google provided 1GB storage when yahoo gave 4MB! Not only that, compared to Yahoo's interface Gmail was light-years ahead. It did what every great design does i.e. get out of the way and let them do what they want to. Gmail had features which people needed and they used the right technology to deliver it. It was first free ajaxified email client(a major one at least) and allowed people to do everything without reloading the page.

Wave, on the other hand, is almost obnoxious. It is as if Google is trying very hard to show off what it can do instead of delivering what people need/want. For instance, that live typing thing. I hate that feature. It is like the unix "talk" on console back in college and I hated it back then too. I do not want people to see what I am typing. I write angry things sometimes which I never send and therefore I do not want people to read what I type. I have talked to people about it and everybody feels that real time typing is just overkill. If you think about it e-mails allow people to speak freely without the fear of others stepping on your feet. It also allows you to think about what you want to convey, double/triple check what you are sending, check facts etc. Google wave just kills all this in one go!

I am sure Google engineers/product managers know that but they are too obsessed with the technology to kill this feature. Or may be they do not realize it. That is a bigger problem which means they do not understand the domain problem well and do not know what problem they intend to solve.

Even if Google gives and option to switch it off, I do not think Wave is going anywhere in this avatar. Wave UI sucks: It is unintuitive, slow and generally not well thought out. So even if they give an option to switch off the instant typing thing, this is not so easy to fix. For instance, if there is a long wave and there are to new waves at different ends of the conversation, I do not know how to find them. The enter key doesn't work as expected. There are so many "unsynced waves" notification. I can go on and on and on.

Proactive India

I love the fact that this Government is very proactive:

Going after the naxals proactively
Naxals have been a big menace for this country. They started off as a movement to protect the interest of Poorest of poor. However it has mutated into a cult which promotes crime and law and order which in turns makes sure poores-of-poor remain just that. It was abut time that govt. capture/shoot all the big naxal ideologists and they did just that when they captured Kobard Ghandhy. And now comes the pounding of the naxal land. Hope this seals the fate of Naxals.

Dealing with Pakistan in a firm manner
Pakis have been very adventurous last year with Mumbai attack. India responded really well by making them pay at various fronts like diplomacy and threatening stability of Pakistan. India also managed to launch the long needed eye-in-the-sky. They were all set to launch this Israeli "eye" in 2006 when America intervened. This is a crucial realtime element which has been missing since the days of Operation Blue star.

Dealing with China
China has been feeling adventurous lately and that is not good for us. India has shown China that it is not the 1962 India and will not take things lying down. They are activating advance airstrips, deploying Mig 29s, Brahmos missiles, building more roads and infrastructure to make sure China understands clearly.

Keep up the good work

One line window pagination in Ruby

def getPages(current_page, minimum=1, maximum=20, window=2)
return((minimum+window<current_page ? minimum.upto(window).collect : minimum.upto(current_page+window).collect) + (current_page-window > minimum+window ? [".."] : []) + (current_page>minimum+window ? (current_page-window > minimum+window ? current_page-window : minimum+window).upto(current_page+window > maximum ? maximum : current_page+window).collect : [])+(current_page+window+1<maximum-window ? [".."] : [])+(current_page<maximum-2*window ? maximum-window : current_page+window+1).upto(maximum).collect)
end

Output:
>> 1.upto(20){|x| print x; print " : "; p getPages(x,1,20,1)}

1 : [1, 2, "..", 19, 20]
2 : [1, 2, 3, "..", 19, 20]
3 : [1, 2, 3, 4, "..", 19, 20]
4 : [1, "..", 3, 4, 5, "..", 19, 20]
5 : [1, "..", 4, 5, 6, "..", 19, 20]
6 : [1, "..", 5, 6, 7, "..", 19, 20]
7 : [1, "..", 6, 7, 8, "..", 19, 20]
8 : [1, "..", 7, 8, 9, "..", 19, 20]
9 : [1, "..", 8, 9, 10, "..", 19, 20]
10 : [1, "..", 9, 10, 11, "..", 19, 20]
11 : [1, "..", 10, 11, 12, "..", 19, 20]
12 : [1, "..", 11, 12, 13, "..", 19, 20]
13 : [1, "..", 12, 13, 14, "..", 19, 20]
14 : [1, "..", 13, 14, 15, "..", 19, 20]
15 : [1, "..", 14, 15, 16, "..", 19, 20]
16 : [1, "..", 15, 16, 17, "..", 19, 20]
17 : [1, "..", 16, 17, 18, 19, 20]
18 : [1, "..", 17, 18, 19, 20]
19 : [1, "..", 18, 19, 20]
20 : [1, "..", 19, 20]

Dunno if I should be proud of this or worried!

Dump those fixtures

I hate testing/debugging a rails application on "unreal" data. At the same time I find writing fixtures by hand is a big pain. I wrote this rake task to dump all the data in fixtures folder by (rake test:fixtures:dump) on a production environment, bring it to development environment and do a rake test:fixtures:load

desc "Dump all the data from DB to fixtures in YML format"
namespace :db do
  namespace :fixtures do
    task :dump => :environment do
      require 'active_record/fixtures'
      ActiveRecord::Base.establish_connection(Rails.env)
      base_dir = ENV['FIXTURES_PATH'] ? File.join(Rails.root, ENV['FIXTURES_PATH']) : File.join(Rails.root, 'test', 'fixtures')
      fixtures_dir = ENV['FIXTURES_DIR'] ? File.join(base_dir, ENV['FIXTURES_DIR']) : base_dir

      Dir.entries(File.join(RAILS_ROOT, "app", "models")).each{|filename|
        if /\.rb$/.match(filename)
          model = Kernel.const_get(filename.gsub(".rb", "").camelcase)
          if model.ancestors.include?(ActiveRecord::Base) and model.table_exists?
            puts "Dumping #{model}"
            file = File.open(File.join("test", "fixtures", model.table_name+".yml"), "w")
            counter = 1
            model.find_in_batches(:batch_size => 5000){|rows|
              rows.each{|row|
                file.puts "#{counter}:"
                file.puts row.attributes.to_yaml.split("\n")[1..-1].collect{|x| "\s\s"+x}.join("\n")
                counter+=1
              }
            }
            file.close
            puts " -- Created a file with #{counter} rows"
          end
        end
      }
    end
  end
end

Shootout: Jruby1.3.1 vs Ruby1.8.7 vs Ruby1.9.2 vs Python2.6 vs Java6

Click on the image to view it clearly

Jruby is the fastest among Jruby, Ruby and Python. Benchmark program was a fractal geometry program posted here.

Language	Time for 100 iterations	times slower than java with -server
java –server	0.18	1
Ruby1.8	7.78	44.07
Ruby1.9.2	4.2	23.78
Jruby	2.5	14.16
Jruby1.3.1—sever	2.31	13.1
java	0.18	1.01
python	3.04	17.21

cricketr

A few weeks ago I decided to quit my job and do something on my own. An idea, to let cricket crazy fans do commentary about anything and everything related to cricket, has been lingering on in my head for very long. I have been thinking on and off about this idea for about an year now. It is finally launched and the URL is cricketr. As such the would cricketr is a typo for cricketer but the "r" actually signifies that cricketr is a web2.0 site.
The idea itself is not very unique. It is basically a twitter clone for cricket. There are, however, a few very subtle but a few important differences.

1. Unlike twitter, it does cater only one domain and hence domain ontology can be used to make sense of a commentary. For instance, if I type "Sachin just hit a massive six", it can deduce that sachin is a player who belongs to Mumbai Indians(IPL context). Not only that, if a match is going on, or has just finished/about to start it will also identify the match and tag this commentary to that match tag. This way it is possible to generate crowdsourced commentary which is extremely fast. On a demo run today I found this to be about 2 minutes faster than cricinfo score and commentary updates and about 1 minute faster than cricbuzz's score and commentary updates.

2. Personal interaction is important but not the goal. Idea is to let users talk about cricket and derive knowledge out of it.

It is still in a very fluid state(it is basically a product of 4 days of almost 24hr marathon hacking sessions) and a lot of things need to be decided upon. Some big features planned pretty soon. Hope people will like it and do a lot of commentary on it :)

Fingers crossed!

Mongrel as a stand alone server

I have been using rails for quiet sometime now. Rails is easy but I feel stupid using it. It is bulky and slow for some of my use cases. I wanted something lean. I wanted something more geeky and powerful to handle lots of long running requests(more than 100K/day with avg. 30sec processing time).
I have been playing with mongrel handlers for about an year now and they work like a charm. In this post I will give a sneak peek into how to write a highly scalable back-end for doing real stuff. Mongrel is a very fast server and can take heavy load. So here we go writing our own mongrel server for heavy processing

require 'rubygems'
require 'mongrel'
require 'mysql'

PORT=4444
class LogHandler < Mongrel::HttpHandler
  def initialize
    @@mysql=Mysql.connect("host", "username", "password", "databse")
  end

   def process(request, response)
     response.start(200) do |head, out|
       logs = @@mysql.query("select * from huge")
       # Do some heavy processing on this data
       sleep 10
       # done
       logs.each{|row|
         out.write(row)
       end
     end
   end
end

config = Mongrel::Configurator.new :host => "0.0.0.0", :port => PORT do
  daemonize(:cwd => Dir.pwd, :log_file => "server.log")
  listener(:num_processors => 150, :timeout => 300) do
    uri "/", :handler => LogHandler.new
  end
   trap("INT") { stop }
end

config.run.join

This piece of code registers a url "/" on the machine on port provided (in this case 4444) and serves a log huge file. Not very developer friendly is it ?


Well we can use ERB along with it and that will make things look a little easier.
Lets add some more code to class LogHandler for ERB stuff.

class LogHandler < Mongrel::HttpHandler
   def initialize
      @mutex = Mutex.new
   end
# This is for making instance variables of this class available at the template
  def get_binding
    binding
  end

# We change the process function to render a rhtml file called view.rhtml
  def process(request, response)
     response.start(200) do |head, out|
       head["Content-Type"] = "text/html"
       logs = @@mysql.query("select * from huge order by id DESC limit 20")
       sleep 20 # Some heavy processing on logs      
       rhtml = ERB.new(File.read("view.rhtml"))
       @mutex.synchronize{
          @logs = logs
          out.write rhtml.result(self.get_binding) 
      }
     end
   end

Mutex lock is important to implement here as mongrel reuses instance variables of this class for subsequent requests and may lead to a race condition.

View.rhtml looks something like this

<html><body><title>Log console</title>
<h1>Last 20 hits on our page</h3>
<table>
        <th>IP</th>
        <th>URL fetched</th>
        <th>Came from</th>
        <th>time taken</th>
     <%@logs.each{|td|%>
<tr>
        <td><%=row[0]%></td>
        <td><%=row[1]%></td>
        <td><%=row[2]%></td>
        <td><%=row[3]%></td>
</tr>
<%}%>
</table>
</div>

Not bad right ? I can now render rhtml as I 'd do from a rails application.

But hey what about the Routing, MVC stuff, activerecord, logging, form helpers, javascript helpers, view side helpers, callbacks, migrations etc etc ? Well well! This is NOT a full scale framework or a Rails substitute. If you want to do view-side-heavy things use Rails or Merb. If you want to do processing heavy jobs which results in simple-html/no-html then use this.

That said we can very easily sneak in a few of the Rails goodies.

1. Active record - That is easy. Just require 'active_record'; establish_connection; create models by doing this

def User < ActiveRecord::Base
end

2. Logging
Rails does logging in two parts. Request logging and response logging. You may add callbacks in the 'process' function to log a request and reponse at start and end of the function respectively.
Something like

def process
   requestLogger(request)
   # Do stuff
   responseLogger(response)
end

3. Routing
This one much more difficult/the most difficult to implement. However if you have only a few urls to match and most of them are not dynamic it is a easier to hard code them. However it is dirty to do so. To implement a light weight routing is not that difficult and not so dirty. We take this path.

First step to implement routing it to get all the parameters. Both post and get parameters. You may use something like this:

  def post_params(request)
    post_params = {}
    request.body.readlines.first.split('&').each{|x|
      k,v=x.split('=')
      post_params[k.to_sym] = CGI.unescape(v)
    }
    return post_params
  end

   def get_params(request)
    get_params = {}
    request.params["QUERY_STRING"].split("&").each{|x|
      k, v = x.split("=")
      get_params[k.to_sym] = CGI.unescape(v)
    }
    return get_params
  end

Actual implementation of routing is a little complex and is not easy to cover in one blog post. Also I may have done it wrong so I do not want to put it out there. I will cover this in detail when I am sure about it.

The more stuff you add to this thing the more it will start looking like Rails! IMHO it is not a bad idea to implement your own framework. I did implement a framework to run rails code as it is. But I never used it on my production servers. Apparently that is the why how it should be :)

Update 1: I have changed some code to highlight that mysql query is not the heavy call and processing is being on done elsewhere (sleep in this case). Default mysql libraries are not thread safe however one may use something like Neverblock

Desperate times calls for desperate measures

These are desperate times and they surely call for some desperate measures. However when I look around I find people doing not-so-desperate things. The only mantra for cost cutting seems to be employee cutting! The truth is that most of the corporations are so bloated and top-heavy that they cannot think about anything else. It is like having a tummy so big that you are unable to see the feet, the truth!, the foundation.

Simple things, like Lehman brothers new york office not switching off external lights/display in months of acute crisis, strike me really really odd. I am not sure I do understand it correctly but isn't it very counter intuitive that when you are undergoing such a bad time you should spend thousands of dollars on some lousy display that nobody cares about ? Have corporations completely forgotten "saving" a primary and basic principle of survival ?

How can we save at least some money in these times ?

1. Start spending as less as possible on stationary. Ask employees to bring pens and diaries.
2. Keep that office printer starved of paper. Better still decommission those which are not really needed or are back up. Put per person quota on printers.
3. Do the same with phones. Ask employees to not use phones for personal purposes. Monitor the bills and slap it on people if it exceeds a certain limit.
4. Sublet office space if you can.
5. Sell that office car if it is not really used
6. Stops those team meets which require a long distance call or travel
7. Rather, stop meetings as much as possible
8. Start switching off everybody's computers at night.
9. Restrict internet usage to bare minimum
10. Stop that friday free lunch, ask people to pay up.
11. Make the coffee dispensing machine pay-on-use
12. Reduce the office cooling up to 25 deg C
13. Make the office gym pay-per-use
14. Switch off all the servers that you don't really need
15. Remove those tissue papers from cafeteria and bathroom. Ask people to carry handkerchiefs
16. Allow people to work from home
17. If possible move the office to a place where rent is lower (suburbs)
18. Completely stop those corporate gifts.
19. Try to move your call center to a web chat based model and save on phone and employee costs. (This may be a good idea in all times)
20. Motivate people to work more, put in more efforts by assuring them that no firing is going to take place.

More to follow as I observe more useless expenses happening around me.
In my view this is much better than firing people left, right and center.