Is Sad Music Actually Sad?

Be it Elliott Smith or Queen, classical or dub step, there's usually a clear understanding that some songs are sad, and some songs are happy. But what is it about the music that makes us feel these feelings we're feeling? You might think it has something to do with the notes or how our brain's natural response to these sounds, but you're wrong. Or at the very least incomplete in your thinking. We've just been culturally trained to respond to music in certain ways because if you ignore the lyrics, music itself doesn't actually contain any emotion at all.