Rather than run for re-election, in 1966, Callaway ran as for governor of Georgia. The gubernatorial election was exceptionally close due to a split within the state Democratic Party between supporters of segregationist Lester Maddox and liberal former governor Ellis Arnall; after Maddox won the Democratic nomination, Arnall continued his campaign as a write-in candidate. Ultimately, Callaway won a plurality, but not a majority of votes cast, which under Georgia law meant that the election was thrown to the Georgia General Assembly. After a series of lawsuits reaching the United States Supreme Court, the authority of the legislature was ultimately upheld, and Maddox was elected governor by the heavily Democratic legislature.[9]
Callaway was succeeded in Congress by Jack Brinkley.
Callaway served as Ford's campaign manager,[12] but resigned following accusations that he had used undue political influence to ensure the expansion of a ski resort;[12] he was replaced by Rogers Morton.[13]