Cale was the man behind the classics "Cocaine" and "After Midnight," which Eric Clapton covered and turned into hits.
Lynyrd Skynyrd hit "Call me the Breeze" and Waylon Jennings' "Clyde" were also penned by Cale.
Cale's official website confirmed that he passed away at Scripps Hospital in La Jolla, Calif. after suffering a heart attack Friday night.
"We've lost a great artist and a great person tonight," a statement on his Facebook page read.
Cale was born John Weldon Cale in Oklahoma City and influenced decades of musicians by pioneering the "Tulsa Sound," a blend of rockabilly, country, jazz and blues.
He was well respected among some of rock's biggest stars and could call musicians like Neil Young, Mark Knopfler, Eric Clapton and Bryan Ferry fans.
Singer Neil Young said, "Of all the players I ever heard, it's gotta be Hendrix and JJ Cale who are the best electric guitar players."
Clapton told Vanity Fair magazine that Cale was the living person he most admired.
Clapton and Cale later collaborated on the album "The Road to Escondido," which won the Grammy for contemporary blues album in 2008.
"I'd probably be selling shoes today if it wasn't for Eric," Cale told the Associated Press in 2006.