Where Did Samual Glover Go

Caroline Brimmer of Scoto, NY was married to Samuel Glover and had a child Samuel Glover born "1853". Census list baby Samuel but no mention of his father. On Samuel (death cert) it does list his dad as Samuel and mother Caroline Brimmer.

Where did you go?