Cage The Elephant singer Matthew Shultz has been arrested for illegal gun possession. As the New York Post reports, Shultz was charged with criminal possession of a firearm after police found two loaded weapons in his room at the Bowery Hotel in Manhattan on Thursday night. A hotel worker claimed Shultz pulled one of the weapons from his pocket in a public part of the building.

Police found the guns along with Polaroids of them and handwritten notes “one of which states in substance: ‘I will protect myself if I am attacked,'” prosecutors allege according to the Post. They also say Shultz appeared intoxicated.

Shultz’s attorney say the weapons were purchased legally and registered, but not in New York. The musician is currently out on $10,000 bail and will return to court Feb 22.