Born at Lorient, in Brittany, he was the son of a lawyer who held posts of great importance at Rennes and in Paris. He bequeathed the little ancestral estate of Keroman where the philosopher-essayist died. The writer was a first-class student in Rennes and obtained honours as a law graduate at the famous College Louis-le-Grand in Paris, but declined that profession due to its moral ambivalence. Partly under the influence of the works of Jean-Baptiste Henri Lacordaire, Jules Amédée Barbey d'Aurevilly and Louis Veuillot, the latter two being the most brilliant and feared polemical crusaders of the Church in the press, he founded a newspaper Le Croisé ("The Crusader") in 1859 but it only lasted two years due to a disagreement with his co-founder. This was one of the greatest disappointments of his life. Thereafter his essays appeared throughout France as well as in Belgium and New Orleans' "Le Propagateur".

Frail from infancy, he also suffered from a spinal or bone disease. This struggle probably tinged his prose with a melancholy strain, which is strikingly original as mentioned in J.-K. Huysmans' work (they shared a veneration of the mystic John of Ruysbroeck). Both writers, like Leon Bloy, are almost impossible to translate.[dubious – discuss] In 1857 he married Zoë Berthier, an army officer's daughter and talented writer herself, who was ten years older and a friend for some years before their marriage. She became his devoted nurse, which brought upon herself abuse from gutter journalists of the time for her estimable guardianship.