What happens if you only use "one" speaker from the set?

It should be fine. You might just be missing one of the channels of audio. Try putting the source on mono mode if it's possible.