I want generate thumbnails with 100x150, but "Admin->options->theme->Thumb size" accepts only one parameter. ie, it is generating 150x150. how can i change this to 100x150.
Note: i unchecked "Crop thumbnails:" option.

Visit the documentation a look for customalbumthumb functions (don't remember hte exact name right now) and change your theme to use them.

if you want to fit your thumbs within the width x height you have set you have to use the maxspace variant.