tldr; "The data so far provide little support for the strong claims about legalization made by either opponents or supporters."
Except that putting people in jail and fining them is cruel and we shouldnt do it without a really fucking good reason.
http://www.cdcr.ca.gov/Reports_Research/Offender_Information...
It's true federal penitentiaries have a larger percentage of inmates in for simple possession, but the federal prison population is a tiny fraction of the number of people incarcerated by the states.
And while I don't have any evidence to back it up, the comparatively small number of people in federal prison for MJ possession suggests they're either the most unlucky people on the planet or they're really there for some other reason even if the government could only make the drug charge stick. Think Al Capone and tax evasion.
tl;dr: Legalizing pot isn't going to affect the prison population very much.