320kbps Sounds Worse Than A Lower Bit Rate?

Ive been converting my whole itunes library to 320Kbps and noticed some songs sound worse at 320 than some lower ones. When i say worse I mean a little bit muffled or less vibrant sounding. I am using AVS audio converter. Yes I have an HD audio card and HD $300 Bose headphones.