A short-serving Prime Minister, Cavendish was elected the Member of Parliament for Derbyshire in 1741 and again in 1747. He was a supporter of Walpole, and later the Pelhams. He was appointed Master of the Horse, having to leave the Commons for the Lords by a writ of acceleration to become Baron Cavendish, and served in this role until 1755, when he succeeded to his father's Dukedom, and was also appointed Lord Lieutenant of Ireland.

In 1756, George II asked Cavendish to form an administration which he accepted on the proviso that it last only until the end of the parliamentary session. His government was mostly run by William Pitt, but lasted only until May 1757 when George II once again called on the Duke of Newcastle. Cavendish served in Newcastle's government as Lord Chamberlain and remained close with him. George III believed Newcastle and Cavendish to be plotting against him, and in late 1762 the King personally struck Cavendish's name from the list of Privy Councillors. Cavendish died two years later.