Could part of it be that UFOs have moved steadily into weirder and weirder directions, so that now they ARE more akin to ghosts, fairies, demons, etc... and are thus even MORE marginalized? Never to go away -- we still have the occasional stories re ghosts and the like -- but never to really regain their earlier even marginally respectable status? (and I lean HEAVILY on that "marginal"...*S*)
I probably am not up to the challenge of expressing this well... :cross eye ... but there was a time.. certainly well into the 70s... that UFOs were seen as primarily "technological"... from my limited knowledge, many of the reports from the 50s and 60s suggested metallic CRAFT, nuts and bolts, tangible, 'real', technologically advanced "ships" from somewhere, with occupants who might be green or red or blue but who were..well.. "real" .. not metaphysical, but "spacemen" or travellers of some kind. I am not underselling the weirdness of some earlier UFO reports... *S*.. but the majority I recall coming across in UFO books, etc, of my youth suggested "real" (if fantastic!) craft of some kind, with "real" (if fantastic) occupants.
As time has passed... as the general public has gained some small measure of understanding of how vast space is, of the amazing distances involved, and as technology has made it more and more difficult for the layman to accept an undetected "spaceship" zipping about (yes, even with stealth! *S*), it seems to me that both UFOs and their occupants have become more and more "metaphysical", more and more akin to spirits and ghosts, and less and less like the nuts-and-bolts "invaders" of the 50s and 60s. Time-warping, reality-altering "beings" travelling in ships of light,etc. And, as they have done so, they have moved more and more into the margins (while certainly still occupying a fairly prominent cultural place -- the aforementioned impossible to avoid Greys, etc).
In some ways, I am reminded a bit of the way science fiction has changed over the years. I read a lot of 50s/60s science fiction in my youth (our local library had an extensive, if aged collection of B-grade science fiction) and such books were FULL of folks "hopping" into their rocket ships and venturing to other worlds, etc, encountering all kinds of alien races -- very much akin to the sci-fi/fantasy worlds of Star Wars, as opposed to the colder, "more real" world of, say, 2001... Well, you really don't see a LOT of that kind of science fiction anymore, for good or ill, in part because there has been SO much discussion of the vast distances, the vastness of space, etc. Fantasy has moved into this niche, by and large... Star Trek and Star Wars aside, the universe has become an infinitely larger, lonlier, sparsely populated place.
I also wonder... somewhat different approach.. if we aren't just plain "thinking smaller" these days. Again, hard to capture in words, but I recall that even amid the turmoil of the 60s, etc, there was a kind of bigger-picture core optimism about the future that I don't feel now... sadly. I recall the excitement about the space race, the almost "certainty" that by now we would have folks LIVING on Mars, etc. May seem like romantic, stupid drivel today, but... there was a sense of a bigger, wider, future that I don't think we have now. Maybe UFOs don't fit so well in our smaller, darker, more cynical, more "insignificant" worldview. I am not ASSUMING that UFOs are spacecraft, but... I think that WAS the general perception and I think it may simply be harder to sell the notion that anyone would cross enormous gulfs to come HERE.... *S* Again, wish I was more eloquent, could explain it better, but...
Interesting thread. Thanks for putting up with my rambling, free-form, half-baked musings...*S*
Shadow