Did you have an ancestor that helped take one of the U.S. censuses?
My grandfather, Henry Melville Potter, was an enumerator in his little town of Ashley, Illinois, for the 1910 census. This was his silver badge. I even had the little pocket notebook where he wrote down the census info for his neighbors in the town…… a clearly illegal thing to do but oh so fun to find today. And since the census was handwritten, I have several pages written in his handwriting which is also such fun.
If your (male) ancestor lived in a rural town, is it possible that he was a Census Taker for that town? Have you looked?