IE sucks - To open a new tab you need windows live toolbar enabled! How crazy is that!

In M$ world you cannot open a new tab unless you have "Windows live toolbar" enabled!!!!!
WTF!! Seriously WTF!

Orkut security: Blocked or locked album viewing

A few days back orkut unveiled a good privacy measure to lock albums, scrapbooks et al for users who do not want them to be available for public consumption. Great feature! The only problem though is that it doesn't work or rather there is a workaround it! This is how:
1. Go to the profile page of the person whose album is locked.

2. View html source of the page and search for 'medium'

3. This will give you an url like http://img<2,3 or 4>.orkut.com/images/medium//.jpg

4. Open another tab paste this url and replace 'medium' with 'milieu/1'(http://img3.orkut.com/images/milieu/2//.jpg) and press enter(or go button of your browser and bang! the first picture in his/her album (thumbnail) will be presented to you.

5. To view other pictures in the album change the 'milieu/1' to 'milieu/2' and so on till you have seen all the pictures.

Security blunder of this magnitude was not expected from google!

Mongrelhandler + erb

A few weeks back I wrote a MongrelHandler to handle concurrent requests on a site to
process a slow running process(read scraping) in background. Here is how it looks like (in essence)

class  < Mongrel::HttpHandler
def get_binding
binding
end
def process(request, response)
response.start(200) do |head, out|
  head["Content-Type"] = "text/html"
  params={}
  request.params["QUERY_STRING"].split('&').each{|x|
    k,v=x.split('=')
    params[k.to_sym]=v
  }
  params[:id]=request.params["PATH_INFO"]
  @posts=Post.find(params[:id]) #or whatever
  template=""
  File.open("#{RAILS_ROOT}/app/views/<path>/<filename>", "r"){
    |f| template=f.readlines.to_s
  }
  rhtml = ERB.new(template)
  str=rhtml.result(self.get_binding)
  out.write str
end
end
end

To start it call
mongrel_rails start -e environment -p port -S path-to-mongrel-handler

Session less rails application

Sessions are used to store user login information on the server because http is a stateless protocol and need either sessions or cookies to know pass data between user's subsequent page requests. However there has been a lot of talk and benchmarks which show that sessions are one of the things that slow down rails applications rather dramatically. It is also one of the key bottlenecks which with many other factors turns itself into a scalability nightmare.
File session stores are slow and when no. of servers are more than one you cannot easily make use of them without using something like NFS or load balancer hack. DB session store though are easily scalable when more than one servers are used are still quiet slow and moreover it would just become as unscable as no. of users and applications increase as the number of inserts and selects from DB will grow tremendously. Drb sessions store ...well nobody seems to use them :)

I decided to completely do away with sessions and use only cookies in my rails application. My application cookie is made up of various user params which are stored in a pipe seperated string and then Rijndael encryption encrypted using a key. Aes is quiet fast and there is a library for ruby which is a actually a wrapper on top of a C lib so it is quiet fast(feels fast enough...haven't done any benchmarks on it).
Now everytime my application needs to check for authentication it reads the cookie, decrypts it (after base decoding and url unescaping) and makes a user object of a user class using this decrypted info and thats it. Bye bye sessions :)

p.s. As for error/flash messages ...well thats a different story. If you want to disobey rails and do things your way, a lot of magic powers are rendered useless but it's not as if there are no ways to get around the problem.

p.p.s. This system was conceptualized by Saurabh Nanda who based it on Standford's WebAuth and JA-SIG CAS

The ruby code for unearthing vista multiplication(display) bug

is this
counter=0;total=0;
1.step(65535,0.1){|x| total+=1; y=((65535/x).to_s).delete('.').split(//).last; if(y.to_i==5 or y.to_i==0); counter+=1; puts "#{65535/x},#{x}-#{65535/x*x}"; end


I love ruby :)

Actually this code spits out 72000 such numbers however the bug is present in around 12000 such numbers. Seems my pattern is only 1/6 true :)

I found 12000 recurrence of excel 2007 multiplication bug

1.016265520786650 64486.1

7.988663375388550 8203.5
7.989832120258950 8202.3
7.994608046453750 8197.4
7.994803103498750 8197.2
7.996656620257950 8195.3
7.997827705300150 8194.1
8.001245330012450 8190.6
8.001831501831500 8190.0
8.006621788371550 8185.1
9.396911429431750 6974.1
9.399067766224450 6972.5
9.401899460576150 6970.4
9.406892789986650 6966.7
9.410675052772150 6963.9
9.412026598112850 6962.9
9.415812991192650 6960.1
9.417436663840550 6958.9

15.278360609875500 4289.4
15.290123888850000 4286.1
15.329824561403500 4275.0
15.356766256590500 4267.5
15.384163947510500 4259.9
15.387414886123500 4259.0
15.401156232374500 4255.2
15.430892394631500 4247.0
15.498403689251500 4228.5
15.536616011948500 4218.1
15.539563227657500 4217.3
15.559486217621500 4211.9
15.563550869193500 4210.8
15.573536750552500 4208.1
15.595031292387500 4202.3
15.712059458163500 4171.0
15.712436164856500 4170.9
15.712812889613500 4170.8
15.713943172281500 4170.5
19.260837619397500 3402.5
19.275000000000000 3400.0
19.358126070774500 3385.4
19.392495709297500 3379.4
19.436206180675000 3371.8
19.510851766947500 3358.9
19.647139944837500 3335.6
24.679897567221500 2655.4
24.689195298372500 2654.4
24.800378429517500 2642.5
24.925832952989500 2629.2
25.097656250000000 2611.2
25.113044144696500 2609.6
25.117856732206500 2609.1
25.174784880147500 2603.2
25.213527239150500 2599.2
25.322642967542500 2588.0
25.364786933467500 2583.7
25.520853615795000 2567.9
25.536764992401500 2566.3
25.572638233113500 2562.7
25.577628600421500 2562.2
25.645691476872500 2555.4
25.708065275380500 2549.2
25.718153991052500 2548.2
25.722191694795500 2547.8
25.872483221476500 2533.0
26.144977260033500 2506.6
26.182580902916500 2503.0
26.287605294825500 2493.0
26.294988564779500 2492.3
26.353144603506500 2486.8
26.417946547345500 2480.7
26.447798539085500 2477.9
26.480927751737500 2474.8
26.549586776859500 2468.4
26.561423418311500 2467.3
42.533099688473500 1540.8
42.602223233439500 1538.3
42.643805309734500 1536.8
42.780207585351500 1531.9
42.850137308748500 1529.4
42.970952724411500 1525.1
43.004790340573500 1523.9
46.330858960763500 1414.5
46.380042462845000 1413.0
46.485317066250500 1409.8
46.495211067754500 1409.5
46.657411362665500 1404.6
46.690652607580500 1403.6
46.720610251657500 1402.7
47.035814253929500 1393.3
47.137308494569500 1390.3
47.242647058823500 1387.2
47.300613496932500 1385.5
57.618252154035500 1137.4
57.750264363764500 1134.8
58.062372641091500 1128.7

78.929302661688500 830.3
79.072152509652500 828.8
79.253839642036500 826.9
79.292196007259500 826.5
79.426736153193500 825.1
79.571393880524500 823.6
79.648760330578500 822.8
79.794228661877500 821.3
79.823386114494500 821.0
80.018315018315000 819.0
80.194566813509500 817.2
80.273150416462500 816.4
80.430780559646500 814.8
80.450527866437500 814.6
80.549410029498500 813.6
80.658461538461500 812.5
80.967383246849500 809.4
81.067540821375500 808.4
81.137798687631500 807.7
99.870466321243500 656.2
100.390625000000000 652.8
104.073368270605000 629.7
104.272076372315000 628.5
105.074555074555000 623.7
105.923711006950000 618.7
106.250000000000000 616.8
107.030867221950000 612.3
109.480454393585000 598.6
123.047315058205000 532.6
126.588757967935000 517.7
138.230331153765000 474.1
139.140127388535000 471.0
142.250922509225000 460.7
142.995854243945000 458.3
143.089519650655000 458.0
148.135171790235000 442.4
153.298245614035000 427.5
153.874148861235000 425.9
154.308923946315000 424.7

157.120594581635000 417.1
158.144305019305000 414.4
160.546300832925000 408.2
160.901055732875000 407.3
162.496900570295000 403.3
164.826458752515000 397.6
171.692428608855000 381.7
177.169505271695000 369.9
198.771610555050000 329.7
200.596877869605000 326.7
200.781250000000000 326.4
207.520582647245000 315.8
208.179796696315000 314.8

269.802387813915000 242.9
287.812911725955000 227.7
298.157415832575000 219.8
305.382106244175000 214.6
312.220104811815000 209.9
312.816229116945000 209.5
315.223665223665000 207.9
315.679190751445000 207.6
317.053701015965000 206.7
318.131067961165000 206.0
321.092601665850000 204.1
331.654858299595000 197.6
334.532924961715000 195.9
337.287699433865000 194.3
342.219321148825000 191.5
345.648734177215000 189.6
351.771336553945000 186.3
381.017441860465000 172.0

695.700636942675000 94.2
706.196120689655000 92.8
715.447598253275000 91.6
720.164835164835000 91.0
726.552106430155000 90.2
753.275862068965000 87.0
766.491228070175000 85.5
772.818396226415000 84.8
773.730814639905000 84.7
782.974910394265000 83.7
788.628158844765000 83.1
798.233861144945000 82.1
800.183150183150000 81.9
803.125000000000000 81.6
850.000000000000000 77.1
915.293296089385000 71.6
947.037572254335000 69.2
951.161103047895000 68.9
992.954545454545000 66.0
1391.401273885350000 47.1
1430.895196506550000 45.8
1606.250000000000000 40.8
2332.206405693950000 28.1
2400.549450549450000 27.3
2436.245353159850000 26.9
3181.310679611650000 20.6
3212.500000000000000 20.4
3810.174418604650000 17.2
4174.203821656050000 15.7
5601.282051282050000 11.7
6425.000000000000000 10.2
7201.648351648350000 9.1
9929.545454545450000 6.6
12850.000000000000000 5.1

Excel 2007 bug

The now (in)famous excel 2007 multiplication bug is quiet interesting. First people thought it is a floating point bug but it seems it may not be the case. I did some analysis and found that the bug can be reproduced like this. Take two numbers who when multiplied give 65535(2^16-1). Suppose one of them is A and the other one is B such that A*B=65535. Now if A is a number such that A*10 is an integer and (B*10^N is and integer and B*10^N is of the form 5M (divisible by 5) (where M and N are interger) then this bug can be reproduced.
For instance 10.2(A) * 6425(B)

Choose the right tool

Everyday on Slashdot and elsewhere one comes across comments like PhP sucks, Ruby is slow, Rails doesn't scale, Perl is obfuscated, Java is slow etc. Most of these comments are posted by newbies(I being one of them) who don't have much idea about the "right tool" for a particular usage. So how do we choose the right tool ?
Well being an engineer I can tell you that this decision is should be based on optimal combinations of these factors(not in order of priority):
1. Load expected on the system
2. Amount of development time available
3. Quality of development resources available
4. Importance of the software being produced(critical/non critical)
5. Complexity

Expected load on the system: I think this one factor should play a major role in deciding the kind of tools to be used. If the expected no. of users on the system is in the order of 2000 requests/day, we can safely say that load is not much. However we must also keep in mind what is the distribution of this load. If there is a spike in the load at any given time of day/week/month/year which reaches, say , 200 req/sec for ten seconds and there is no load after that throughout the day, then the system must be capable of handling such loads. Therefore expected maximum load/sec multiplied by some safety factor (say 2) should be our design goal.
We must also consciously avoid premature optimization. Building a Google like system when we have a user base of 2000 is stupidity to say the least.

Amount of development time available(deadline): Some times it may be necessary to churn out some cool feature or application in short amount of time because it is sought after by the users. We cannot ignore that demand and sketch out an elaborate and mammoth design which would take months. It would be a better idea to code using some sort of "rapid prototyping tool" to gauge the user perception and then if it becomes popular then recode it into some other high performance language or "harden" the existing app by a code review.
Some amount of time should be kept aside for murphy 's sake

Quality of development resources available: This is one thing which should directly affect the tool being used. If we have a "super geek" team we can probably use any tool on earth. But sadly this is not the case most of the time and hence it is better to choose something which has a short learning curve and clear demarcations are available in the design to distribute it efficiently over a large non-high-quality team.

Importance of the software being produced: Well if we need an application which is not very critical or say some downtime would not hurt anything on the production side, then using a tool which makes life easy is a good idea. Also spending development time of a better quality resource is not that good an idea.

Complexity: Choice of tool should also depend on the complexity of the software that needs to be built. We cannot expect to build a complex application which say needs neural networks, backward chaining and symbolic calculations in php! Lisp would be a btter idea.

In my view before even we start designing a system we should create a list of the goals along with reasonable expected figures of key statistics. this would not only help us design the system better but also allow to take important desicions like no. of developers needs, technology to be used, amount of time expected for completion etc.

Gotchas!

Oracle:
If you have worked only with MySQL till now and planning to shift you apps in production on Oracle then you may run into a few very nasty gotchas.

1. "30 character" limit: I had a long (19 characters) table name (required due to rails convention) and some pretty long column names (due to legacy schema thingy). I wrote the find methods and bang! An error which said Oracle doesn't support identifiers longer than 30 characters! Whoa!!! I still don't understand what could be the logic behind a 30 character limit!

2. Oracle doesn't have a 'limit' keyword. Yes you read it correctly ! You cannot say something like this "select * from < table > limit 10, 100"! Oracle doesn't support this at all. You would need sub selects and stuff

3. Some of the table names which are allowed in mysql are not in Oracle like "date"...if I am not mistaken

Rails:
If you serialize a column and try to read the column if no data is there it returns a rather nasty error which says "ActiveRecord::SerializationTypeMismatch:"
http://dev.rubyonrails.org/changeset/6880

A new nigeria ???

I received this spam today morning which indicates that Nigeria's domination over scam mails may soon be threatened!!! ...On a second thought what if this mail actually originated from Nigeria :)


U.S ARMY PERSONNEL
1st Armored Division (IRAQ)
Tikitit Regitment Base.

Dear Friend,

With a very desperate need for assistance, I have summed up
courage to
Contact you. My name is SGT Michael Albert, I am an American
soldier,
serving in the military of the 1st Armored Division in
Iraq,and i
actually found your contact particulars in a business
journal.I am
seeking your Experience and assistance to evacuate the sum of
($28.2Million United Sates Dollars to USA) or any other safe
country
and there are no risks involved.

Right now Iraq is a war zone We have being attacked by
insurgents
everyday and car bombs and I have survived two suicide bomb
attacks by
the grace of God.We managed to move funds belonging to Saddam
Hussein's
family. We want to move this money to you, so that you may
invest it
for us and keep our share for banking. I do not know for how
long we
will remain here.

One passionate appeal I will make to you is not to discuss
this matter
with any third party,as any leakage of this information will
be too bad
for me.It may be difficult for us to communicate on phone for
security
reasons,my job is to find a good partner that we can trust and
assist
us. Can I trust you? When you receive this Fund, kindly send
me an e-
mail signifying your interest we shall also discuss about
percentage
when it is convenient to do so.

I will guide you with more information. But only when I
ascertained
your seriousness and totality to commit your time in seeing to
the
success of this business.

I wait for your urgent and good response.

Respectfully,

Sgt.Michael Albert

Wanna make your Rails app faster?

(courtesy: http://scott.elitists.net/sessions.html)

Get rid of sessions!
Yes you can get rid of sessions for apps that run too slow for your comfort. Ofcourse flash errors won't show up and other stuff that uses sessions. But definitely there are other ways to do flash errors and logins and other "session" stuff in a sessions disabled environment.

So go on and say
session :disabled => 'true'
in your application controller and feel the difference

Finch

Chat in console with finch...It's very fast and eats very less memory, supports all the major protocols but the best part is that your boss won't be able to catch you :)

Writing a custom mongrel handler

Wrote my first custom mongrel handler and it is ugly. I have not used ERB, eruby or erubis yet to generate HTML.

Benchmark results for mongrel handler

Transactions: 5000 hits
*Elapsed time: 54.55 secs
*Data transferred: 2.90 MB
*Response time: 0.45 secs
Transaction rate: 91.66 trans/sec
*Throughput: 0.05 MB/sec
*Longest transaction: 1.63
*Shortest transaction: 0.00

A similar app on rails was made and here are the benchmark figures

Transactions: 5000 hits
*Elapsed time: 195.34 secs
*Data transferred: 3.39 MB
*Response time: 3.29 secs
**Transaction rate: 25.60 trans/sec
*Throughput: 0.02 MB/sec
*Longest transaction: 7.63
*Shortest transaction: 0.01


There is almost a 4 fold jump in transaction rate and therefore Mongrel handler can be used to handle pages which have very high hit counts
However using plain vanilla mongrel handlers to write pages is very difficult as it does not automagically 'renders' html pages.

Ruby

After 3 months with Ruby, I think it's a great language. It may not be as fast as C or Perl but it is very developer-friendly. It's 100% object oriented. That means everything is an object and hence everyobject can be manipulated by some very useful set of methods that are provided with the language
So you can say something like this
12.next => 13
12.remainder(5) => 2

It also supports some of the very powerful features of Lisp(Greatest language ever !) like unnamed functions, blocks, iterators etc

"A quick brown fox jumped over a lazy dog".downcase!.split(//).sort.uniq.each {|char| print char+" "}
gives
a b c d e f g i j k l m n o p q r u v w x y z

Being 100% OOPs is one of the strongest features of this language

1.upto(10){|i| puts "9 X #{i} = #{9*i}"}
will give you table of 9
9 X 1 = 9
9 X 2 = 18
9 X 3 = 27
9 X 4 = 36
9 X 5 = 45
9 X 6 = 54
9 X 7 = 63
9 X 8 = 72
9 X 9 = 81
9 X 10 = 90

It offers "minimum surprise" to a newbie and the code is very easy to read.

Though there are some downsides also as it is interpreted only language and can be slow. But come Christmas 2007 a new ruby version is being released which hopes to solve this problem to some extent.

Konqueror: A better browser

I just ran a css compatibility test on konqueror and firefox and amazingly
konqueror passed all the tests while firefox flunked many!

Firefox: From the 43 selectors 26 have passed, 10 are buggy and 7 are unsupported (Passed 357 out of 578 tests)

Konqueror: From the 43 selectors 43 have passed, 0 are buggy and 0 are unsupported (Passed 578 out of 578 tests)

You can test you own browser at http://www.css3.info/selectors-test/test.html

Hmm... Now I remember somebody told me a few years ago that Konqueror is a better browser...Okay...point noted

Rails Roko

I never follow any cult and nor do I despise any "particular" OS because every other geek in town does. But if I had to follow one, I would have joined a "rails-roko" cult :P. People argue that rails is in the same stage as Php was 9 years ago. Fine! I agree that things take time to stabilize but then you should not publicize it as the next big thing after bread!
Rails, the idea, is great. Web programming is mostly CRUD and having a framework that makes it all so easy should be encouraged. But then encouraging and supporting it one thing and marketing it as if it is enterprise ready is quite another.
Rails has a number of show stoppers like mongrels wake up issue, performance issue, freedom issues!.
I have tried to deploy a third party rails application in a production environment with "minor changes". These minor changes turned out to be monstrous. Switching to a only-cookie-based-system (to make the system scalable) turned out to be a nightmare. Now flash notices and errors don't work as they use sessions and I have completely disabled them. Running a mongrel cluster behind a apache frontend is also a big nightmare. Mongrels die (or go into deep sleep) overnight and wake up after a long time (~10 mins) after a page is requested. Per mongrel transaction count is also low.
It is very easy to do stuff in rails which result in a huge number of database queries like a 'forum.topic.post.last.created_at' in the view against each forum makes life miserable for the rails app as it first hits a method_missing in ruby interpreter and then rails tries to generate a function out of it after looking at all class declarations! and that is slow like anything. I switched it off and found that requests/sec count went up from 6-7 req/sec to a mind boggling 70-100 req/sec!
So all the marketing gimmicks are not going to work unless the stuff actually works!
More rails cribbing later

A few thoughts about Ruby on Rails

1. Starting is fun and easy. A few configurations, sqls and commands and "almost" everything you need is done.
2. First road block comes when you create views which query multiple tables in one go.
3. Life becomes easy again when you have to modify designs and templates. Page templates resides in different folder so html headache is minimum
4. You really have to bang your head in the wall if you try to integrate your app in a portal with an already-present-cookie-based login system!
5. Never ever try to do #4 with a thrid party app.
6. Rails somehow seems very immature with the way DHH changes stuff. Apps created in rails 1.1 doesn't work with rails 1.2. Even ruby versions and "optional" gems create problem and to add to the misery these errors are either cryptic or so insanely-damn-fucking stupid that you won't ever realize that it's a version clash. (A "wrong number of argument "... error surfaced because I had a older RedCloth gem and it was a discovery almost by accident when I out of sheer frustration uninstalled everything and installed everything again)
7. Rails has weird gotcha and it stems from the fact that it is cryptic. An innocent looking column name in a table would give you so much pain that you feel like banging your head against your monitor. Don't do that becuase first it can be dangerous and second it's not your fault. Check out these links
http://wiki.rubyonrails.org/rails/pages/Gotcha
http://blogs.thewehners.net/josh/view/178
8. Performance of a "default" app sucks big time. A lot of tweaking is necessary to make it fast enough to be barely usable.
More on rails bitching later

Business Idea #1

Okie here goes my first one:
1. General Knowledge on the move
Wikipedia is a free source of knowledge but this knowledge base is restricted to web. What if there be a tool which summarizes those articles (possibly divding it into multiple pieces and shorten it) and makes it available over mobile phones. For instance you want to look up "Lake saimaa" (It is the largest lake in finland and is very beautiful), it picks up the relevant article from wiki and sends back the user a sumamry.

In this case since the article size itself is small (character count 1025) so summarizing it (making it 320 character long) won't be a big issue as such. Some of the popular "sms" shortcut transforms can be used. But what if somebody searches for say "India". Then the summarizing code has to be more intelligent to take the article and ascertain which are the important parts of this particular article (in this case geography, political system, economics, population, military, culture etc). Then it should return say about 200 character long article on India and list it's population, GDP, Average expectancy of life etc and say "make more relevant search by saying ex. India GDP" etc.
This way people are intrigued to send more smses to search server for more content which would in turn result in more revenue.
Now what if somebody sends a query about some recent happening or some obscure topic which wikipedia has no knowledge about. Then we land in a soup. Wikipedia alone cannot cater to all the knowledge "needs". We will need more content, news, articles and such.

Okay then so much for mobile content.

Have switched jobs and joined a travel search company. Work is good here and environment is well energetic. Seems I am gonna love it here. Did I tell ya that they code here in lisp ?

Catching Up

A lot has happened in the tech world and in my world. I have kinda drifted away from this blog due to extremely stupid and irritating web filtering at work. One cannot access even blogs!! There is this really irritating filter called websense. It categorizes all the websites into various categories and the admin can blog them...category by category! Shopping, tasteless!, dating and personal , GMAIL, webmail, proxy avoidance, Message boards and clubs, Lifestyle and society! and Sports!! are some of the categories that are banned there. Well not for very long ...I have resigned and will be joining a travel search firm.
It is amazingly stupid that some people think that you can actually make people work in a organization by limiting access to the net and setting entry and exit restrictions! Maybe in a mill or something but where you have people working on large projects and most of the work is accomplished by taking a personal interest, a ban on things junta does in leisure time is suicidal. No wonder out of four people in my cubicle 3 have already put in their papers.

I think the new India and new Indians are a very different from the old India that was there 20-30 years ago. Censoring tactics like china is not going to work in a free country like India. Some may argue that censorship does exists in movies. Well hindi movies anyways don't have a lot of free speech and change the world kind of stuff. Moreover there is no choice. Job scene on the other hand is a very different ball game altogether!

Tech world in the next post.

Traffic forecasting

Is it possible to forecast traffic ?
Definitely it is!!

Indian Social Networking -- Part 2

Sometimes this whole idea of Indian Social Networking reminds me of 1999. A disaster prone, completely unfeasible idea of selling a service on internet (free substitutes of which are already available). Indian public just won't buy anything online. But then whats the harm in trying (if money is not from your pocket). Maybe they can sell cakes, cards, t-shirts, CDs, DVDs, flowers etc. Then they can also act as job search websites! ..Imagine naukri turning into a social networking website!!!