A few things I've found out researching the subject of removing the hat:

"A relic of the ancient custom of taking off the helmet when no danger is nigh. A man takes off his hat to show that he dares stand unarmed in your presence."

"Taking the hat off is the modern remains of the ancient custom of Knights who removed their helmets in the presence of those they felt their friends and thus, before those they wished to honor by showing that they trusted them. A man removes his hat before a woman to show his respect. Touching the brim is but a perfunctory salute."

"Taking off hat/cap - in medieval Europe knights and soldiers had a helmet as a protection. Entering a home with a helmet on was a cause of serious concern. It was considered entirely unfriendly and a new custom emerged of taking of helmet and later whatever you were wearing on your head (ladies excluded)."

"In the days of knight errantry, every one meeting a stranger had to suppose him an enemy; ten to one he was. And the sign and proof of friendly intention was raising the right hand without a weapon in it. The hand was raised high, to be seen as far as they could shoot with a bow, and a further proof was added when they raised the vizor and exposed the face. The danger of the highway continued long after knights ceased to wear armour; so, with the same meaning, the same gesture was used, but with a lifting of the hat. If a man did not do it, he was either showing contempt or hostility for the other, or proving himself an ignorant brute. So, in all civilized countries, lifting the hat is a sign of mutual confidence and respect."

"Man needs a hat to keep the sun off his head in the summer, and to keep his head warm in the winter, but when he is inside he might pray or prophesy so he removes his hat in honor to God. 1 Corinthians 11:4 - Every man who prays or prophesies with his head covered dishonors his head."



There are many other references out there but too many to list here.

Remember, wherever you go that's where you are.