Pioneer of the theory that genetic traits affect survival of both individual organisms and species, Darwin wondered in his own lifetime if his marriage to first cousin Emma Wedgwood was having “the evil effects of close interbreeding” that he had observed in plants and animals.
Three of their children died before age 10, two from infectious diseases. The survivors were often ill, and out of the six long-term marriages that resulted, only half produced any children. According to researchers at Ohio State University and Spain's Universidad de Santiago de Compestela, that alone is a "suspicious" sign that the Darwins suffered from reproductive problems.
You can read more in an article by Zoё Macintosh in the LiveScience web site at http://www.livescience.com/history/charles-darwin-inbreeding-100503.html