The Ontario Appeal Court ruling strikes down a plank of the federal Conservatives' 2008 omnibus bill, which raised the penalty from one year.
The court found a three-year prison sentence to be "cruel and unusual punishment" for a first offence.
The court says the ruling has no significant impact on sentences for people engaged in criminal conduct or who pose a danger to others, saying they should continue to receive sentences to emphasize deterrence and denunciation.
The Appeal Court heard six appeals together in February because each involved a constitutional challenge to a mandatory minimum sentence for various firearm offences.
The minimum sentence was originally struck down last year by the Ontario Superior Court in the case of Leroy Smickle, who was caught alone in his boxers in his cousin's apartment posing with a loaded handgun while taking pictures of himself to post on Facebook. Police burst in, looking for his cousin.
Federal government lawyers had argued in support of the law, pointing to a spate of gun violence in 2005, which first prompted Ottawa to propose the stiffer penalties.
Critics in the legal community, however, say mandatory minimums don't reduce crime and do more harm than good.