James was born in 1902 in Bentonia, Mississippi and brought up on a plantation by his mother. His father was a bootlegger and had left his mother when James was only five years old. He was 10 years old when he got his first guitar and he learned his first blues techniques from other workers who played guitar on the plantation. His teenage years were troubled, running away from home on occasions, but he returned to attend High School and at this time he also learned to play the piano. James dropped out of High School early and went on the road, working and playing at sawmills and construction sites around the Mississippi Delta. He would also venture into local towns to play on street corners for any tips he could get. When he was 19 years old he moved to Arkansas and teamed up with Will Crabtree, who was both a pianist and a pimp.

James worked with Crabtree for a while, playing the blues and also pimping for a local brothel, and then moved to Memphis where he got a job as a pianist in a brothel. The Prohibition years were lean ones for the bars and brothels, and James returned to Mississippi and worked as a sharecropper. To augment his income he also began running illicit liquor to pay for the high style clothes and jewellery that he had got used to in Memphis. During this time James developed his trademark "three finger picking" guitar playing technique and this, coupled with his unusual high pitched singing voice, brought him to the attention of Paramount Records. In 1931 he recorded eighteen songs for Paramount, all of which were released and included "Special Rider Blues," and "Devil Got My Woman". About this time James met up with his father who was now a reformed character and Baptist minister. He went to Texas with his father, forming a gospel group, and later attended a seminary in Alabama where he was ordained himself. However he eventually returned to Mississippi when his mother became ill and decided to stay there after she died, working as a farmer. Along with other Mississippi blues singers, James was "re-discovered in the 1960ís and began recording again. He also began touring but became ill in the late 1960ís and died in 1969 aged 67.