I don't rememeber 'men' being prominent in the packaging and we just called the people 'legos' like the rest of the sets. So i'm half with you on them being kegos and not minifigs, but all of the people in the sets are just legos.
Can't think of a situation where that distinction would be necessary. Like for the castle set we called them knights or soldiers, police were police, that guy was that guy, etc.