Marc — Of course your children don’t see you as untrustworthy, and of course, you haven’t taught them to. But about whom are we speaking? I referred to “cultural truisms.”
Although I do so very hesitantly, let’s put aside the possibility that the do-gooders would treat you no differently, with respect to your own children, than they would some other presumedly predatory male. The point remains that this accepted level of precaution includes their father. In other words, that male hand on the billboard could be dad’s. The soccer coach whom the league has instructed not to have physical contact with the children is dad. Whatever signals your children absorb from the society around them with respect to men’s “potential for violence and sexual aggressiveness” will present no exception for their father. (And here I emphasize that fathers are — or should be — the central male figures in children’s lives.)
Do you think your children will not pick up on the differences in the ways in which others treat you and your wife? Your comments about one’s own familial circle of trust are inarguable (although applicable to both genders)*, but it was the public profiling for which you initially admitted support. The different mentions of coaches draw the distinction most clearly: In one post, they’re to be warned beyond an invisible barrier from the children; in the next, they’re within the trusted circle. (I note also that you seem to be shifting from “like all other men” to “strangers” in general.)
Obviously, there’s a churning range of causes and effects, from the explicit directives of an individual parent to the vague impressions perpetuated in popular culture, and to some extent we’re arguing at different ends. I’m not arguing that any particular person “acts in an untrustworthy manner” because he feels “that people unfairly don’t trust” him. I’m arguing that the general image — the general profile — of men as vessels of predation in varying degrees of restraint has repercussions in the relationships that girls and women form with them. (I actually had females in mind, as much as males, as those reaping loneliness and isolation.)
It seems to me that this shift in social interactions, subtle though it may be, will tend to increase the number of grown men with a poorly formed ability to develop relationships. The effect is heightened, I’d say, by the barriers that are increasingly being built between boys and men, hindering the ability of the former to find healthy routes toward becoming the latter. (There’s a whole ‘nother tangent, here, in the power that then moves to government and cultural players to offer images for emulation.)
Because society doesn’t function in an “us getting along with them” manner (we’re all both us and them), your prescription against the stranger’s isolation misses the mark:

But I don’t think that being mistrusted simply because you are a stranger is a permanent sentence to loneliness or isolation. By engaging your community-volunteering, church, school — you can cease becoming “the other” and begin becoming part of the neighborhood. But it takes time.

If part of the problem is that men are sliding into a sense of isolation, society must draw them into involvement. Any man who takes determined steps to become involved is heading in the right direction of his own accord. (Although any man who takes too determined steps to gain trust would surely arouse suspicion.) In other words, in the modern attempt to lay out clear rules and procedures, society increasingly pushes aside the very people who most needed its direction.
* The manner in which and reasons for which we treat men and women differently when it comes to trusting them with our children relates to a peculiar obsession with sexuality and sexual acts in our era. Surely there are other ways in which adults can abuse children, some of which might actually do more harm in the long run than fondling.