cp123, you are actually correct. One major cause (according to the guy at my LBS) of feet numbness is wearing your shoes too tight, as it cuts off the supply to your foot (the main artery is on the top of your foot). Loosening you shoes should help.
I'm just guessing here, but another problem may be if the soles of your shoes are fairly flexy, you may be putting pressure on some nerves in your foot (hence the fact that it goes away when you release your foot). I used to get this with my old shoes (Shimano basic mountain bike ones - they just had laces), then I upgraded to Specialized Sport MTB shoes (very stiff), and my feet are now as comfy as ever