>..when I want to dress up or wear a mini-skirt or whatever, I feel like I'm standing out a bit too much..

This is my experience too. Whenever I think about wearing something a bit fancy (that would be totally acceptable in another field) I feel a bit self conscious.