Carl was born near Huntingdon, Tennessee on 22^nd August 1942. He grew up in the backwoods, listening to local radio stations which fed a diet of classic hillbilly and honky tonk into the family wireless set. Carl's love of music was nurtured by his family, firstly in church, then by listening to the mother church of country on the radio. He was on stage at the Grand Ole Opry at the age of nine and later on the local radio stations talent shows. By 1950 he was playing guitar, and in 1952, Jackson radio station WDXI gave him a regular spot.

In 1954 Carl formed a band with fellow youngsters and were billed as Carl Mann and the Kool Kats. A year or so later when his piano player quit, he learned to play the piano because there was no one else around to play it. They played local churches, schools and the Junior Opry on WSM Nashville. His professional career began in 1958 when he recorded his first record for Memphis based Jaxon label called Gonna Rock N Roll Tonite / Rockin’ Love. In early 1959 he signed a three year contract with Sun Records and in April, his first record Mona Lisa was released.

It was a reasonable success and shot up the national charts with sales of over a half a million copies. It would have done better if not for Conway Twitty's cover version being released at the same time. He had several follow-up hits but the fast pace was too much for the eighteen year old and he had to take time off to recuperate. In the meantime his contract had expired and Sun owner Sam Phillips had lost interest in him. Over the next two decades he recorded for various labels in both the US and Europe, without much success.