I don’t know if it is just me, but I think that there are many more possibilities when designing hats for women. Women can get away with any kind of colour or any amount of frilliness they want, although it still takes a certain personality to pull some more outlandish things off. I can see most of the hats I make being worn by women, but a small subset can also be worn by men. When I look at the hats around me (which of course I do constantly), I think most men tend to wear simple watch caps or ski hats. Of my hats, more of the ones that men might wear are either off white, dark colours or shades of grey.
Unless you are my wonderful nephew Ben, for whom everything in life has always been possible.