Adaptation: To me, one of the continually-remarkable things about cold-weather cycling is the way our bodies adapt when in use. I call it remarkable because it seems like I have to consciously remind myself that it's going to happen. If I dress to be comfortable for the first 7 minutes of cycling in the cold, it usually means I'm going to sweat up my gear thoroughly by the end of my 9 mile commute (and be pretty miserable at the end). So what I do is I suffer (just a teeny bit) for the first mile and a half, until I make it to the top of my first decent hill. By the time I've ridden that far, I'm quite comfortable. By "comfortable" I don't mean that I'm toasty warm, but I'm not feeling chilly, either, and the engine is just humming along. It is a bit of a strange sensation, feeling a little cold, but being very confident that the discomfort will pass, and it always does. It's a good exercise in delayed gratification.
By far the most difficult "extremities" problem is hands. I've discussed before my approach to gloves / mittens, and I've found the "Zoidbergs" with poly liners good down only to the about 20° F. (given my commute distance). Below that temperature, you can help comfort by wearing latex gloves against your skin as vapor barriers to prevent chapping. I recommend also applying a very thin coating of some anti-chap hand cream as soon as you can after riding. Chapped fingers are no fun.
Discussions on Particular Items of Clothing:
1. Tunic: I think finding a high-quality fleece tunic is fairly critical. This tunic from REI [UPDATE 10-19-09: the previous item is no longer available; this is the replacement] is what I use, and I like it quite a lot. It's stretchy, it's available in tall sizes, the fleece is very soft and comfy against my skin, it's very thermally forgiving, and it will dry quickly hanging on the back of my office door.
2. Windproof shell: There are many brands and varieties of windproof shell available. I've had mine for several years now, and I'll probably have to replace it in the next year or so. The one I have I bought several years ago at the LL Bean store in Freeport, ME, and I've worn it so much that replacing it will seem like replacing a friend. But the zipper has only so much time left. In any case: requirements are: reflectivity, drawstring or elastic waist (or both), big enough to fit over a couple of winter layers, but not excessively floppy, zipper pockets, velcro wind flaps over the zippers, and a stowable hood. This is the closest thing that Bean has to what I use now. In a perfect world, it would have under-arm "pit zips," but sigh, no.
3. Vapor barriers for feet: This makes a surprising difference in foot warmth. I use newspaper bags for simplicity, economy, and convenience. For weather that's not too cold, you can just pull them on over your wool socks. But your feet will definitely feel, uh, rather humid at the end of your ride, so it feels good to "unsack" your feet at the end of the ride to let 'em air out. This article suggests that I'm doing it wrong, and ought to wear the VB under my socks, so in colder weather I've worn thin sock liners (old dress socks work just fine) under the bags. VB socks are available from hunting-fishing places also, but the paper bags are a pretty green solution, and zero cost (if you're one of the dinosaurs still reading newpapers!)
4. Headwear: I've become pretty attached to wearing a balaclava in temperatures below freezing, although above freezing, a thin fleece stocking-cap will do fine. In very cold weather, I might be tempted to put the (windproof) hood up on my parka.
So, that's about it. Nothing earth-shattering here, and (as always) your mileage may vary (along with your cold-bloodedness), so you can adjust the temps at the top of the chart. Remember to keep in mind some of my other recommendations for winter cycling, and get out there.