I think it would be asking a lot for all changelings to be attributable to the same phenomenon.
For one thing, some changelings were adults. In the case of the burning of Bridget Cleary in 1895, both the defendant - her husband, Michael Cleary - and the victim's own family either believed, or expected their neighbors to believe, that the Bridget who died might not have been Bridget at all, but a breathing, moving, speaking ringer brought in by fairies who had abducted her. This case demonstrates the flexibility of the changeling concept, which could be used to cover a wide variety of mental, social, and physical states; much the same way that vague medical diagnoses (Irritable Bowel Syndrome! Hysteria! Nervous disorders! SIDS! PMS! Mid-life crisis! Stress!) have been called upon to render unmanageable situations less intimidating by applying a label ever since the Enlightenment.
In the Cleary case, after the burning and his arrest, Michael became passionately insistent that he must be released in order to go rescue his wife from the fairies - having killed the imposter, he must be in the right place at the right time and perform the right actions in order to free her. It's not hard to imagine yourself into his shoes at this point - whatever he, in his heart, believed at the moment he took the actions that led to Bridget's death, once she was dead she had to be a changeling, because otherwise, he was a wife-murderer. Had the case occurred in 1395 instead of 1895, we would now have a version of it in which Bridget's behavior and physical condition fully justified this belief.
Similarly, if ill-treatment or neglect (easily prompted by stress, ignorance, or postpartum depression) causes the death of a child, the child's relatives will be anxious to justify their own behavior in their own eyes. A little exagerration here, a little memory burp there, a little cooperation from people who want to believe because not believing is unbearable - anything from Down's Syndrome to ADHD, Munchhausen's by Proxy to Shaken Baby Syndrome, can become a changeling story, complete with Evil Imposter Baby.