Optimizing Modules w/o Kernel Recompile

I'm a little gun-shy about recompiling my own kernel, but I'm hoping to optimize my modules and hopefully decrease my memory usage a little.

First of all, is there any benefit for not loading modules that aren't useful for a given machine? For example, joydev, psmouse, snd*? Will I free up memory this way?

If I were to prevent something from loading in debian, which syntax is best?

blacklist module_name

alias module_name off

Thanks!