This sounds suspiciously like a partially clogged pilot jet. I had a Yamaha Virago that did this same thing, and it turned out one of the pilot jets had a little debris in it. Those holes are not much bigger than the diameter of a sewing needle, and I was quite surprised to learn how much they affect performance. If I remember correctly, the easiest way to check is to idle the bike on the center stand and disconnect one spark plug. If it continues to run (albeit roughly) then that cylinder is fine. Disconnect the other. If it dies instantly, it's most likely that pilot jet.