In 1940, federal charges were filed against Baker for mail fraud and he spent four years in prison.