The now (in)famous excel 2007 multiplication bug is quiet interesting. First people thought it is a floating point bug but it seems it may not be the case. I did some analysis and found that the bug can be reproduced like this. Take two numbers who when multiplied give 65535(2^16-1). Suppose one of them is A and the other one is B such that A*B=65535. Now if A is a number such that A*10 is an integer and (B*10^N is and integer and B*10^N is of the form 5M (divisible by 5) (where M and N are interger) then this bug can be reproduced.
For instance 10.2(A) * 6425(B)
Friday, September 28, 2007
Monday, September 10, 2007
Choose the right tool
Everyday on Slashdot and elsewhere one comes across comments like PhP sucks, Ruby is slow, Rails doesn't scale, Perl is obfuscated, Java is slow etc. Most of these comments are posted by newbies(I being one of them) who don't have much idea about the "right tool" for a particular usage. So how do we choose the right tool ?
Well being an engineer I can tell you that this decision is should be based on optimal combinations of these factors(not in order of priority):
1. Load expected on the system
2. Amount of development time available
3. Quality of development resources available
4. Importance of the software being produced(critical/non critical)
5. Complexity
Expected load on the system: I think this one factor should play a major role in deciding the kind of tools to be used. If the expected no. of users on the system is in the order of 2000 requests/day, we can safely say that load is not much. However we must also keep in mind what is the distribution of this load. If there is a spike in the load at any given time of day/week/month/year which reaches, say , 200 req/sec for ten seconds and there is no load after that throughout the day, then the system must be capable of handling such loads. Therefore expected maximum load/sec multiplied by some safety factor (say 2) should be our design goal.
We must also consciously avoid premature optimization. Building a Google like system when we have a user base of 2000 is stupidity to say the least.
Amount of development time available(deadline): Some times it may be necessary to churn out some cool feature or application in short amount of time because it is sought after by the users. We cannot ignore that demand and sketch out an elaborate and mammoth design which would take months. It would be a better idea to code using some sort of "rapid prototyping tool" to gauge the user perception and then if it becomes popular then recode it into some other high performance language or "harden" the existing app by a code review.
Some amount of time should be kept aside for murphy 's sake
Quality of development resources available: This is one thing which should directly affect the tool being used. If we have a "super geek" team we can probably use any tool on earth. But sadly this is not the case most of the time and hence it is better to choose something which has a short learning curve and clear demarcations are available in the design to distribute it efficiently over a large non-high-quality team.
Importance of the software being produced: Well if we need an application which is not very critical or say some downtime would not hurt anything on the production side, then using a tool which makes life easy is a good idea. Also spending development time of a better quality resource is not that good an idea.
Complexity: Choice of tool should also depend on the complexity of the software that needs to be built. We cannot expect to build a complex application which say needs neural networks, backward chaining and symbolic calculations in php! Lisp would be a btter idea.
In my view before even we start designing a system we should create a list of the goals along with reasonable expected figures of key statistics. this would not only help us design the system better but also allow to take important desicions like no. of developers needs, technology to be used, amount of time expected for completion etc.
Well being an engineer I can tell you that this decision is should be based on optimal combinations of these factors(not in order of priority):
1. Load expected on the system
2. Amount of development time available
3. Quality of development resources available
4. Importance of the software being produced(critical/non critical)
5. Complexity
Expected load on the system: I think this one factor should play a major role in deciding the kind of tools to be used. If the expected no. of users on the system is in the order of 2000 requests/day, we can safely say that load is not much. However we must also keep in mind what is the distribution of this load. If there is a spike in the load at any given time of day/week/month/year which reaches, say , 200 req/sec for ten seconds and there is no load after that throughout the day, then the system must be capable of handling such loads. Therefore expected maximum load/sec multiplied by some safety factor (say 2) should be our design goal.
We must also consciously avoid premature optimization. Building a Google like system when we have a user base of 2000 is stupidity to say the least.
Amount of development time available(deadline): Some times it may be necessary to churn out some cool feature or application in short amount of time because it is sought after by the users. We cannot ignore that demand and sketch out an elaborate and mammoth design which would take months. It would be a better idea to code using some sort of "rapid prototyping tool" to gauge the user perception and then if it becomes popular then recode it into some other high performance language or "harden" the existing app by a code review.
Some amount of time should be kept aside for murphy 's sake
Quality of development resources available: This is one thing which should directly affect the tool being used. If we have a "super geek" team we can probably use any tool on earth. But sadly this is not the case most of the time and hence it is better to choose something which has a short learning curve and clear demarcations are available in the design to distribute it efficiently over a large non-high-quality team.
Importance of the software being produced: Well if we need an application which is not very critical or say some downtime would not hurt anything on the production side, then using a tool which makes life easy is a good idea. Also spending development time of a better quality resource is not that good an idea.
Complexity: Choice of tool should also depend on the complexity of the software that needs to be built. We cannot expect to build a complex application which say needs neural networks, backward chaining and symbolic calculations in php! Lisp would be a btter idea.
In my view before even we start designing a system we should create a list of the goals along with reasonable expected figures of key statistics. this would not only help us design the system better but also allow to take important desicions like no. of developers needs, technology to be used, amount of time expected for completion etc.
Gotchas!
Oracle:
If you have worked only with MySQL till now and planning to shift you apps in production on Oracle then you may run into a few very nasty gotchas.
1. "30 character" limit: I had a long (19 characters) table name (required due to rails convention) and some pretty long column names (due to legacy schema thingy). I wrote the find methods and bang! An error which said Oracle doesn't support identifiers longer than 30 characters! Whoa!!! I still don't understand what could be the logic behind a 30 character limit!
2. Oracle doesn't have a 'limit' keyword. Yes you read it correctly ! You cannot say something like this "select * from < table > limit 10, 100"! Oracle doesn't support this at all. You would need sub selects and stuff
3. Some of the table names which are allowed in mysql are not in Oracle like "date"...if I am not mistaken
Rails:
If you serialize a column and try to read the column if no data is there it returns a rather nasty error which says "ActiveRecord::SerializationTypeMismatch:"
http://dev.rubyonrails.org/changeset/6880
If you have worked only with MySQL till now and planning to shift you apps in production on Oracle then you may run into a few very nasty gotchas.
1. "30 character" limit: I had a long (19 characters) table name (required due to rails convention) and some pretty long column names (due to legacy schema thingy). I wrote the find methods and bang! An error which said Oracle doesn't support identifiers longer than 30 characters! Whoa!!! I still don't understand what could be the logic behind a 30 character limit!
2. Oracle doesn't have a 'limit' keyword. Yes you read it correctly ! You cannot say something like this "select * from < table > limit 10, 100"! Oracle doesn't support this at all. You would need sub selects and stuff
3. Some of the table names which are allowed in mysql are not in Oracle like "date"...if I am not mistaken
Rails:
If you serialize a column and try to read the column if no data is there it returns a rather nasty error which says "ActiveRecord::SerializationTypeMismatch:"
http://dev.rubyonrails.org/changeset/6880
Sunday, August 26, 2007
A new nigeria ???
I received this spam today morning which indicates that Nigeria's domination over scam mails may soon be threatened!!! ...On a second thought what if this mail actually originated from Nigeria :)
U.S ARMY PERSONNEL
1st Armored Division (IRAQ)
Tikitit Regitment Base.
Dear Friend,
With a very desperate need for assistance, I have summed up
courage to
Contact you. My name is SGT Michael Albert, I am an American
soldier,
serving in the military of the 1st Armored Division in
Iraq,and i
actually found your contact particulars in a business
journal.I am
seeking your Experience and assistance to evacuate the sum of
($28.2Million United Sates Dollars to USA) or any other safe
country
and there are no risks involved.
Right now Iraq is a war zone We have being attacked by
insurgents
everyday and car bombs and I have survived two suicide bomb
attacks by
the grace of God.We managed to move funds belonging to Saddam
Hussein's
family. We want to move this money to you, so that you may
invest it
for us and keep our share for banking. I do not know for how
long we
will remain here.
One passionate appeal I will make to you is not to discuss
this matter
with any third party,as any leakage of this information will
be too bad
for me.It may be difficult for us to communicate on phone for
security
reasons,my job is to find a good partner that we can trust and
assist
us. Can I trust you? When you receive this Fund, kindly send
me an e-
mail signifying your interest we shall also discuss about
percentage
when it is convenient to do so.
I will guide you with more information. But only when I
ascertained
your seriousness and totality to commit your time in seeing to
the
success of this business.
I wait for your urgent and good response.
Respectfully,
Sgt.Michael Albert
U.S ARMY PERSONNEL
1st Armored Division (IRAQ)
Tikitit Regitment Base.
Dear Friend,
With a very desperate need for assistance, I have summed up
courage to
Contact you. My name is SGT Michael Albert, I am an American
soldier,
serving in the military of the 1st Armored Division in
Iraq,and i
actually found your contact particulars in a business
journal.I am
seeking your Experience and assistance to evacuate the sum of
($28.2Million United Sates Dollars to USA) or any other safe
country
and there are no risks involved.
Right now Iraq is a war zone We have being attacked by
insurgents
everyday and car bombs and I have survived two suicide bomb
attacks by
the grace of God.We managed to move funds belonging to Saddam
Hussein's
family. We want to move this money to you, so that you may
invest it
for us and keep our share for banking. I do not know for how
long we
will remain here.
One passionate appeal I will make to you is not to discuss
this matter
with any third party,as any leakage of this information will
be too bad
for me.It may be difficult for us to communicate on phone for
security
reasons,my job is to find a good partner that we can trust and
assist
us. Can I trust you? When you receive this Fund, kindly send
me an e-
mail signifying your interest we shall also discuss about
percentage
when it is convenient to do so.
I will guide you with more information. But only when I
ascertained
your seriousness and totality to commit your time in seeing to
the
success of this business.
I wait for your urgent and good response.
Respectfully,
Sgt.Michael Albert
Monday, July 23, 2007
Wanna make your Rails app faster?
(courtesy: http://scott.elitists.net/sessions.html)Get rid of sessions!
Yes you can get rid of sessions for apps that run too slow for your comfort. Ofcourse flash errors won't show up and other stuff that uses sessions. But definitely there are other ways to do flash errors and logins and other "session" stuff in a sessions disabled environment.
So go on and say
session :disabled => 'true'
in your application controller and feel the difference
Friday, July 13, 2007
Finch
Chat in console with finch...It's very fast and eats very less memory, supports all the major protocols but the best part is that your boss won't be able to catch you :)
Thursday, July 12, 2007
Writing a custom mongrel handler
Wrote my first custom mongrel handler and it is ugly. I have not used ERB, eruby or erubis yet to generate HTML.
Benchmark results for mongrel handler
Transactions: 5000 hits
*Elapsed time: 54.55 secs
*Data transferred: 2.90 MB
*Response time: 0.45 secs
Transaction rate: 91.66 trans/sec
*Throughput: 0.05 MB/sec
*Longest transaction: 1.63
*Shortest transaction: 0.00
A similar app on rails was made and here are the benchmark figures
Transactions: 5000 hits
*Elapsed time: 195.34 secs
*Data transferred: 3.39 MB
*Response time: 3.29 secs
**Transaction rate: 25.60 trans/sec
*Throughput: 0.02 MB/sec
*Longest transaction: 7.63
*Shortest transaction: 0.01
There is almost a 4 fold jump in transaction rate and therefore Mongrel handler can be used to handle pages which have very high hit counts
However using plain vanilla mongrel handlers to write pages is very difficult as it does not automagically 'renders' html pages.
Benchmark results for mongrel handler
Transactions: 5000 hits
*Elapsed time: 54.55 secs
*Data transferred: 2.90 MB
*Response time: 0.45 secs
Transaction rate: 91.66 trans/sec
*Throughput: 0.05 MB/sec
*Longest transaction: 1.63
*Shortest transaction: 0.00
A similar app on rails was made and here are the benchmark figures
Transactions: 5000 hits
*Elapsed time: 195.34 secs
*Data transferred: 3.39 MB
*Response time: 3.29 secs
**Transaction rate: 25.60 trans/sec
*Throughput: 0.02 MB/sec
*Longest transaction: 7.63
*Shortest transaction: 0.01
There is almost a 4 fold jump in transaction rate and therefore Mongrel handler can be used to handle pages which have very high hit counts
However using plain vanilla mongrel handlers to write pages is very difficult as it does not automagically 'renders' html pages.
Monday, July 09, 2007
Ruby
After 3 months with Ruby, I think it's a great language. It may not be as fast as C or Perl but it is very developer-friendly. It's 100% object oriented. That means everything is an object and hence everyobject can be manipulated by some very useful set of methods that are provided with the language
So you can say something like this
12.next => 13
12.remainder(5) => 2
It also supports some of the very powerful features of Lisp(Greatest language ever !) like unnamed functions, blocks, iterators etc
"A quick brown fox jumped over a lazy dog".downcase!.split(//).sort.uniq.each {|char| print char+" "}
gives
a b c d e f g i j k l m n o p q r u v w x y z
Being 100% OOPs is one of the strongest features of this language
1.upto(10){|i| puts "9 X #{i} = #{9*i}"}
will give you table of 9
9 X 1 = 9
9 X 2 = 18
9 X 3 = 27
9 X 4 = 36
9 X 5 = 45
9 X 6 = 54
9 X 7 = 63
9 X 8 = 72
9 X 9 = 81
9 X 10 = 90
It offers "minimum surprise" to a newbie and the code is very easy to read.
Though there are some downsides also as it is interpreted only language and can be slow. But come Christmas 2007 a new ruby version is being released which hopes to solve this problem to some extent.
So you can say something like this
12.next => 13
12.remainder(5) => 2
It also supports some of the very powerful features of Lisp(Greatest language ever !) like unnamed functions, blocks, iterators etc
"A quick brown fox jumped over a lazy dog".downcase!.split(//).sort.uniq.each {|char| print char+" "}
gives
a b c d e f g i j k l m n o p q r u v w x y z
Being 100% OOPs is one of the strongest features of this language
1.upto(10){|i| puts "9 X #{i} = #{9*i}"}
will give you table of 9
9 X 1 = 9
9 X 2 = 18
9 X 3 = 27
9 X 4 = 36
9 X 5 = 45
9 X 6 = 54
9 X 7 = 63
9 X 8 = 72
9 X 9 = 81
9 X 10 = 90
It offers "minimum surprise" to a newbie and the code is very easy to read.
Though there are some downsides also as it is interpreted only language and can be slow. But come Christmas 2007 a new ruby version is being released which hopes to solve this problem to some extent.
Saturday, June 23, 2007
Konqueror: A better browser
I just ran a css compatibility test on konqueror and firefox and amazingly
konqueror passed all the tests while firefox flunked many!
Konqueror: From the 43 selectors 43 have passed, 0 are buggy and 0 are unsupported (Passed 578 out of 578 tests)
You can test you own browser at http://www.css3.info/selectors-test/test.html
Hmm... Now I remember somebody told me a few years ago that Konqueror is a better browser...Okay...point noted
konqueror passed all the tests while firefox flunked many!
Firefox: From the 43 selectors 26 have passed, 10 are buggy and 7 are unsupported (Passed 357 out of 578 tests)
Konqueror: From the 43 selectors 43 have passed, 0 are buggy and 0 are unsupported (Passed 578 out of 578 tests)
You can test you own browser at http://www.css3.info/selectors-test/test.html
Hmm... Now I remember somebody told me a few years ago that Konqueror is a better browser...Okay...point noted
Tuesday, June 05, 2007
Rails Roko
I never follow any cult and nor do I despise any "particular" OS because every other geek in town does. But if I had to follow one, I would have joined a "rails-roko" cult :P. People argue that rails is in the same stage as Php was 9 years ago. Fine! I agree that things take time to stabilize but then you should not publicize it as the next big thing after bread!
Rails, the idea, is great. Web programming is mostly CRUD and having a framework that makes it all so easy should be encouraged. But then encouraging and supporting it one thing and marketing it as if it is enterprise ready is quite another.
Rails has a number of show stoppers like mongrels wake up issue, performance issue, freedom issues!.
I have tried to deploy a third party rails application in a production environment with "minor changes". These minor changes turned out to be monstrous. Switching to a only-cookie-based-system (to make the system scalable) turned out to be a nightmare. Now flash notices and errors don't work as they use sessions and I have completely disabled them. Running a mongrel cluster behind a apache frontend is also a big nightmare. Mongrels die (or go into deep sleep) overnight and wake up after a long time (~10 mins) after a page is requested. Per mongrel transaction count is also low.
It is very easy to do stuff in rails which result in a huge number of database queries like a 'forum.topic.post.last.created_at' in the view against each forum makes life miserable for the rails app as it first hits a method_missing in ruby interpreter and then rails tries to generate a function out of it after looking at all class declarations! and that is slow like anything. I switched it off and found that requests/sec count went up from 6-7 req/sec to a mind boggling 70-100 req/sec!
So all the marketing gimmicks are not going to work unless the stuff actually works!
More rails cribbing later
Rails, the idea, is great. Web programming is mostly CRUD and having a framework that makes it all so easy should be encouraged. But then encouraging and supporting it one thing and marketing it as if it is enterprise ready is quite another.
Rails has a number of show stoppers like mongrels wake up issue, performance issue, freedom issues!.
I have tried to deploy a third party rails application in a production environment with "minor changes". These minor changes turned out to be monstrous. Switching to a only-cookie-based-system (to make the system scalable) turned out to be a nightmare. Now flash notices and errors don't work as they use sessions and I have completely disabled them. Running a mongrel cluster behind a apache frontend is also a big nightmare. Mongrels die (or go into deep sleep) overnight and wake up after a long time (~10 mins) after a page is requested. Per mongrel transaction count is also low.
It is very easy to do stuff in rails which result in a huge number of database queries like a 'forum.topic.post.last.created_at' in the view against each forum makes life miserable for the rails app as it first hits a method_missing in ruby interpreter and then rails tries to generate a function out of it after looking at all class declarations! and that is slow like anything. I switched it off and found that requests/sec count went up from 6-7 req/sec to a mind boggling 70-100 req/sec!
So all the marketing gimmicks are not going to work unless the stuff actually works!
More rails cribbing later
Subscribe to:
Posts (Atom)