I think it's VERY annoying. I have no music on my sites. I have surfed the web from work so I know how "nice" it's to have the boss think you're working and then hear the music
Here is the part of the article about music on your site...
4) Music – Forcing people to listen to your choice of music is one of the professional annoyers most deadly weapon. Used well this technique can guarantee that the hapless visitor to your site will leave quickly never to return. For a really top-level ‘grade A’ annoyance put the music on ‘loop’ and place the controls to stop the music somewhere difficult to find preferably at the bottom of the site. This technique can be enhanced even further by combining with the obscure plug-in
It does depend on the site itself. I have one site that has a holiday theme and I let visitors choose wether they want to listen to the song by clicking on a "play music" and "stop playing music" button. Works great for that site.