Felons' Rights: A Paradox?

It's generally been concluded that felons do not share the same constitutional rights as law-abiding citizens. Tradition holds that a citizen who commits a felony has violated his end of the social contract, voiding the obligation of society to give him, in return, his own constitutional rights.
Some states have always restored these rights after the felon has paid their debt to society. Others, like Maine, can permanently suspend the rights of felons to vote, own firearms, obtain certain occupational licenses, sit on a jury (is this really a right, or an obligation?) or run for public office.
Florida, on the other hand, has a state-run board to determine whether or not a felon deserves to have these rights restored, a concept they have been taking fire for.
To be sure, some felonies are worse than others and a value judgement must be made by society: "Was the crime this person committed an atrocious act against society? Is it worth punishment for life?"
This may not be the case for some nonviolent crimes like drug trafficking, grand theft, cannabis cultivation and fraud, but where do we draw the line? If that line is violence, people like KennethLay and Jeffrey Skilling, who aren't exactly run-of-the-mill thieves, may be getting off too easily.
I agree some felons have paid their debt to society and deserve to renew their societal contract. On the other hand, if we are to make concessions for some felons, a common-sense system outlining the exact criteria for rights resortation must be outlined to avoid the slippery slope.