Scarbrough was succeeded by his eldest son, the fifth Earl. He represented Lincoln in Parliament. On his death the titles passed to his younger brother the aforementioned the Hon. Richard Lumley, the sixth Earl. He also sat as Member of Parliament for Lincoln. He was succeeded by his younger brother John, the seventh Earl. He assumed the additional surname of Savile in 1807 in accordance with the will of his uncle Sir George Savile, and inherited the Savile estates on his brother's death. Lord Scarbrough was a clergyman. His son, the eighth Earl, represented Nottinghamshire and Nottinghamshire North in the House of Commons and served as Lord Lieutenant of Nottinghamshire. In 1836, he assumed by royal licence the additional and principal surname of Savile.[4]
The eighth earl had several illegitimate children (see below) but never married. He was succeeded by his first cousin once removed, the ninth Earl. He was the grandson of the Hon. Frederick Lumley, fifth son of the fourth Earl. His son, the tenth Earl, was a soldier and also served as Lord Lieutenant of the West Riding of Yorkshire. He was succeeded by his nephew, the eleventh Earl. He was the son of Brigadier General the Hon. Osbert Lumley, younger son of the ninth Earl. Lord Scarbrough represented Kingston upon Hull East and York in Parliament and served as Governor of Bombay. His son, the twelfth Earl, was Lord Lieutenant of South Yorkshire. As of 2017[update] the titles are held by the latter's eldest son, the thirteenth Earl, who succeeded in 2004.[3]