Well, by default Crossover should be installing into your system applications folder. When you first run Crossover after un-zipping the installer, Crossover will ask you if you want to "move to applications folder" (which you should do) - this will copy Crossover to your system Applications folder.
When you install a given windows application in Crossover, the launcher icon for that windows app goes by default into your user's Applications folder, i.e.
/Users/jon/Applications/CrossOver/[windows app]
You can change this location within Crossover's Preferences>System Integration menu, but in general I'd recommend that you leave this setting alone. Note that these are just the launcher icons for the windows apps. The apps' program files are in
/Users/jon/Library/Application Support/CrossOver/Bottles
If you want your bottles (the folders that contain your windows applications) stored somewhere other than default, some methods for altering Crossover's settings to achieve this are discussed, here:
http://www.codeweavers.com/support/wiki/mac/faq/cxofficemacbottlelocation
For seldom-used windows apps, it's not too much of a problem to just copy the bottle's directory to an external drive and then copy it back into place and launch Crossover (which will recognize the bottle and generate a new icon for it in the Programs menu) when you need it. You could also try symlinking that bottle after moving it from the original location to your USB, so the link within the standard bottles directory remains and points to the new location:
$ ln -s /Volumes/drive/bottlename/ /Users/jon/Library/Application Support/CrossOver/Bottles/bottlename
