He was admitted to the bar in 1856.[1] He entered private practice in Keokuk, Iowa from 1856 to 1857.[1] He was a member of the Iowa House of Representatives in 1857, resuming private practice in Keokuk from 1858 to 1861.[1] He was a member of the Iowa Senate from 1861 to 1865,[4] again resuming private practice in Keokuk from 1862 to 1869.[1][5]
In McCrary's first month in Congress, he received national attention for refusing to support an appropriation for a federal courthouse in Keokuk because the nation was in debt and he could not support such a courthouse in every district.[7] He published A Treatise on the American Law of Elections, in 1875.[3][8] In the 44th United States Congress, as a member of the United States House Committee on the Judiciary, he was the author of a farsighted (but unsuccessful) bill to reorganize the federal courts to enable reasonable and prompt judicial review.[9] He helped create the Electoral Commission to resolve the outcome of the 1876 Presidential Election, and served on the committee that investigated the Crédit Mobilier scandal.[3]
Maintaining his passion for law, McCrary established an expertise in contested elections and laws pertaining to elections.[10] He published A Treatise on the American Law of Elections in 1875, which later underwent four editions.
McCrary was the 33rd United States Secretary of War in the cabinet of President Rutherford B. Hayes from March 12, 1877, to December 11, 1879, when he resigned.[12] As Secretary, McCrary withdrew federal troops from the remaining reconstruction governments in South Carolina and Louisiana, and used federal troops in the 1877 railway strike and in Mexican border disturbances.[3] The greatest military conflicts during his watch occurred in the American West, in battles with certain Native American tribes in Colorado, New Mexico, and elsewhere.[13]
Following his resignation from the federal bench, McCrary resumed private practice in Kansas City, Missouri from 1884 to 1890.[1] He served as general counsel for the Atchison, Topeka and Santa Fe Railroad Company in Kansas City, Missouri from 1884 to 1890.[3][14] He died on June 23, 1890, in St. Joseph, Missouri,[1] after suffering from a stomach tumor.[2] He was interred in Oakland Cemetery in Keokuk.[6]