There's a place on South William Street (McGregor's Plastics) which sells all sorts of display stuff, generally aimed at shops and businesses, but sells to the public too. I know they have things like hooks especially for hat displays, and they're wall mountable. Even if you didn't buy the hooks, having a look around might give you some ideas.
This is an image of one of the hat displays they have. The wooden part minimises damage to the inside of the hat. Might even be something you could make a version of yourself?