He became a judge in 1863. He presided over the Haymarket Square case in 1886, sentencing anarchists August Spies, Michael Schwab, Samuel Fielden, Albert Parsons, Adolph Fisher, George Engel, and Louis Lingg to death and Oscar W. Neebe to fifteen years.
There was no evidence that any of the defendants had any connection with the bombing. Gary allowed them to be convicted on the theory that their speeches had encouraged the unknown bomber to commit the act.
In 1888 he was appointed by the Supreme Court to the Appellate Court for the First District of Illinois; he returned to the Cook County Superior Court in 1897. He was still active as a judge at the time of his death.
He has no connection to the city of Gary, Indiana, which was named after Judge Elbert Gary, a business associate of J.P. Morgan, and the first president of U.S. Steel Corp.