Gimp makes it really easy to create multi-res .ico files. In fact, this is one of the few areas where Gimp beats Photoshop hands down.
The trick is to use multiple layers with different sizes.

Here are the basic steps.