The earliest version of what's now considered a 'men in black (MIB)' incident occurred in 1947. It occurred in relation to the Maury Island Incident:
http://www.mufon.com/maury-island-incident---1947.html
One of the two reporting witnesses (Harold Dahl) claimed he was visited by a man in a black suit who advised him to keep quiet about the incident. This alleged visit was never substantiated beyond Dahl's claim, which was made long after the fact. However, there were actual events involving visits, investigations, and warnings ...
Within a month of the alleged incident Dahl and his boss (
not a witness himself) were visited by Kenneth Arnold (the original 'flying saucer' witness, who'd transitioned into the first dedicated UFO researcher). Two USAAF officers who'd flown in also met with them while Arnold was there. Dahl allegedly gave some evidence to the USAAF officers, who departed and were killed when their plane crashed
Owing to the plane crash and deaths, the USAF and the FBI got involved with formal investigations. Their initial conclusion was that the Maury Island incident was a hoax which had resulted in the deaths of two officers.
Those were the days of J. Edgar Hoover and his dress code - meaning that any FBI agents Dahl met were wearing dark business suits.
The FBI, at least, warned Dahl that if he'd shut up about the alleged UFO incident there'd be no referral for prosecution on charges of fraud.
So there you have it - all the elements of the MIB mythos, originating in one known incident.
By the time another couple of decades had passed the UFO community of interest had grown, fed on itself over and over, and began getting paranoid to the point conspiratorial repression had become part of the canon. It was in this later timeframe that today's MIB mythos was born, reflecting elements that dated back to the earliest highly-publicized UFO sightings / mania.
I don't doubt that folks reporting UFO incidents received contacts and / or personal visits. These would have included official Project Blue Book investigators, law enforcement personnel, self-anointed UFO experts, and anyone else who had an interest.
I strongly doubt all the reported such visits actually occurred or that those visits that did occur involved agents operating for a single organization under a single agenda.