Thanks for the suggestions guys, I wiggled it around and it seems to not be stuck anymore, however, the mystery continues.
Now when I try to start the bike, it just cranks and cranks but wont start. If i move the choke all the way towards me (towards the rear of the bike), the bike will stop cranking completely. Like it will crank once and then silence. Could this be related to the leaking choke or should I just call it a day and pull out the carbs for a cleaning (which I would like to avoid if possible, as the bike ran before this gas leak)?