[Ohio]

This is not exactly a GKChesterson quote, rather a distortion ascribed to Pres. JFK:

Bartlett’s Familiar Quotations says that the quotation, “Don’t ever take a fence down until you know the reason why it was put up,” was ascribed to Chesterton by John F. Kennedy in a 1945 notebook.The quotation you’re looking for is from Chesterton’s 1929 book, The Thing, in the chapter entitled, “The Drift from Domesticity”:

"In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it."