He incurred the resentment of the Scottish government by rescuing, in June 1676, his brother-in-law Kirkton, a Presbyterian minister who had illegally been seized and confined in a house by Carstairs, an informer. He was fined £500, remaining in prison for four months and then being liberated on paying one-half the fine to Carstairs. In despair at the state of his country he determined in 1683 to emigrate to South Carolina, but the plan came to nothing.
The same year Baillie, with some of his friends, went to London and entered into communication with Monmouth, Russell and their party in order to obtain redress; and on the discovery of the Rye House Plot he was arrested. Questioned by the king himself he repudiated any knowledge of the conspiracy, but with striking truthfulness would not deny that he had been consulted with the view of an insurrection in Scotland. He was subsequently loaded with irons and sent back a prisoner to Scotland. Though there was no evidence whatever to support his connexion with the plot, he was fined £6000 and kept in close confinement.
He was already in a languishing state when on December 23 1684 he was brought up again before the high court on the charge of treason. He was pronounced guilty on the following day and hanged the same afternoon at the market cross at Edinburgh with all the usual barbarities. His shocking treatment was long remembered as one of the worst crimes committed by the Stuart administration in Scotland.
Bishop Burnet, who was his cousin, describes him as "in the presbyterian principles but ... a man of great piety and virtue, learned in the law, in mathematics and in languages." He married a sister of Sir Archibald Johnston, Lord Warriston, and left a son, George, who took refuge in Holland, afterwards returning with William III and being restored to his estates.