He married Lizzie Moorhead, a native of Erie, of which her father was one of the oldest citizens. She attended the Erie Academy, in which she afterwards taught. Stone taught in this academy, and it is in this way they met. They had 4 daughters and 2 sons. The eldest son acted as Stone's private secretary, and the youngest attended the preparatory department of Columbia College. The eldest daughter married Mr. Allen, of Warren. The second daughter, Ann, was an interesting young woman and a favorite in Washington society and attended school near Philadelphia and afterwards traveled in Europe. The third daughter, Bessie, attended Baltimore College.[1]
Stone was elected as a Republican to the Fifty-first Congress to fill the vacancy caused by the death of Lewis F. Watson. He was reelected to the Fifty-second and to the three succeeding Congresses. He served as chairman of the United States House Committee on Coinage, Weights, and Measures during the Fifty-fourth and Fifty-fifth Congresses. He was an unsuccessful candidate for reelection in 1898. He was also an unsuccessful candidate for Governor of Pennsylvania in 1898. He resumed the practice of law and died at his home near Warren, Pennsylvania. Interment in Oakland Cemetery in Pleasant Township, Pennsylvania.