The Greek study of mathematics was closely related to that of religion. Plato is quoted as saying "God ever geometrizes" and Pythagoras as saying "Numbers rule the Universe".
Johannes Kepler stated that "The chief aim of all investigations of the external world should be to discover the rational order and harmony which has been imposed on it by God and which He revealed to us in the language of mathematics."
Isaac Newton became extremely religious in his old age, and devoted the rest of his life to the study of religion.
Leopold Kronecker is quoted as saying "God made the integers, all the rest is the work of man."
James Jeans said "From the intrinsic evidence of his creation, the Great Architect of the Universe begins to appear as a pure mathematician".
According to Henri Poincare, "If God speaks to man, he undoubtedly uses the language of mathematics."
Georg Cantor equated what he called the Absolute Infinite with God. He held that the Absolute Infinite had various mathematical properties, including (if I recall correctly) that every property of the Absolute Infinite is also held by some smaller object.
St. Anselm's ontological argument sought to use logic to prove the existence of God. A more elaborate version was given by Gottfried Leibniz; this is the version that Gödel studied and attempted to clarify with his ontological argument.
Kurt Gödel created a formalization of St. Anselm's ontological argument for God's existence known as Gödel's ontological proof.
While Gödel was deeply religious, he never published his argument because he feared that it would be mistaken as establishing God's existence beyond doubt. Instead, he only saw it as a logical investigation and a clean formulation of Leibniz' argument with all assumptions spelled out.