Most of the hair on your body isn't really about warmth. The hair on your head will keep your head warm, but not by much. A lot of your body hair though is to help keep you cool, keep dirt out of certain areas, and can also help you to feel things when something moves or touches them.
Much of the hair in weird places is used to help trap dirt. For instance, nose hair helps keep particles and dust in the air from entering into your lungs. Hairs in your ears serve the same purpose (Boogers and earwax also act like this... they're kind of like flypaper for dirt.)
Armpit hair is another weird one. I'd say it was to help cool you when you sweat, by taking the hot sweat off your body and putting it where it can be dissipated.
So, hair doesn't just keep you warm, but it also keeps you from getting dirt where you don't really need it, and is effective in cooling yourself off. I would say that pubic hair falls into the "keep dirt out" category, but at the same time, in some situations, I suppose it could also be used for cooling. Pardon me if I sound a little crude here, but think - when testicles get hot, they sweat, like other parts of the body. The hair there could do the same as on armpits, and move the liquid away to help cool them to a proper temperature. And on a woman, the hair would probably help keep dirt from getting up into her vagina more than anything else.