He was 76.
His long-time friend and co-accused, John Artis, said Carter died in his sleep early Sunday morning after a lengthy battle with prostate cancer.
"It's a big loss to those who are in institutions that have been wrongfully convicted," said Artis.
"He dedicated the remainder of his life, once we were released from prison, to fighting for the cause of those who have been wrongfully convicted."
Artis quit his job state side and moved to Toronto to act as Carter's caregiver after his friend was diagnosed with cancer nearly three years ago.
During the final few months, as Carter's health took a turn for the worse, Artis said the man who was immortalized in a Bob Dylan song had come to grips with the fact that he was dying.
"He tried to accomplish as much as he possibly could prior to his passing," Artis said of the man who founded the Association in Defense of the Wrongly Convicted.
"He didn't express very much about his legacy. That'll be established for itself through the results of his work. That's primarily what he was concerned about — his work," Artis said.
"He was a very selfless person," Artis added.
Artis, who had known Carter for 48 years, said Carter's family members — including two daughters — are all in the United States.
He said Carter's personal artifacts, including his writings and photos, are being donated to a Boston university.
Carter was convicted along with Artis of murdering three people in a New Jersey bar in 1966.
Their convictions were overturned in 1975, but both were found guilty a second time in 1976.
After serving 19 years, Carter was freed in 1985 when a federal judge overturned the second convictions.