Productive members of our Society with felony backgrounds (1 out of 100 Americans) are not allowed to vote and/or bear arms. Our Rights are not privileges our government can pick and choose who can utilize them. If we as citizens allow our government to dictate which sub-groups within our citizenry are allowed certain Rights that we hold so dear, then what is to stop other sub-groups (like yours) to be included in these restriction(s) of Rights?
"First they came for the socialists,
and I didn't speak out because I wasn't a socialist.
Then they came for the trade unionists,
and I didn't speak out because I wasn't a trade unionist.
Then they came for the Jews,
and I didn't speak out because I wasn't a Jew.
Then they came for me,
and there was no one left to speak for me."-Martin Niemölle