A couple of others touched on this earlier, but it bears repeating: good gear is far more comfortable.
We've covered the safety factors pretty well, but the fact remains that a properly fitting full-face helmet is by far the most comfortable thing to have on your head for a day of riding.
Same for a good armored jacket appropriate for the weather (I have three or four to choose from depending on weather, from waterproof touring jacket to mesh), gloves (25+ pairs...

), pants (waterproof armored overpants for winter commuting, armored mesh overpants for hot weather, etc.), and even boots.
Work boots protect pretty well and function fine, but if you try riding with real motorcycle boots, you'll understand how they enhance control and comfort, and make riding that much more pleasurable.
Honestly, people who think they're more comfortable in a beanie and t-shirt have simply never tried wearing proper gear. Good gear makes riding more pleasurable, not less -- you can ride further and faster with less fatigue and stress.
(By the way, "good gear" does NOT include assless chaps, unarmored and unvented black leather jackets, leather or denim vests festooned with patches, leather do-rags, slip-on (and slip-off) harness boots, fingerless gloves, and faded blue jeans. These items are costumes, not riding gear.)
People who resist wearing full-face lids have simply never ever tried one that fits properly. There were a lot of really bad, uncomfortable helmets in the '80s and '90s that caused a lot of people to just give up on them, but these days you can get lots of different top-of-the-line extremely high quality full-face helmets for under $100. They're light weight, aerodynamically excellent, and VERY comfy. You're cooler in the summer and warmer in the winter.
You do have to try several different brands to find one that fits -- they're all a wee bit different -- and you should get advice on fit from someone with a lot of experience and/or helmet fitting training. Just about everyone buys their first helmet too big unless they seek advice.
Once you find a brand that works for you, you can usually order that brand in the future from the internet. For example, I have a pretty standard HJC XL head, so I've purchased my last four or five helmets over the internet. I rotate between two HJC AC-12 helmets at the moment. The second was purchased for just $80 on clearance because the graphics were unpopular. It feels and looks the same from the inside...
Here's a ton of material on helmet fit, dozens of helmet reviews, etc.:
http://www.webbikeworld.com/motorcycle-helmets/motorcycle-helmets.htm