Attach your strap lower, so when you stand up, the strap is about even with your beltline.
That way it falls down and out of the way.
That's why I use a safety "Belt" for bow hunting.
I am not saying you should use a belt, because a harness is much safer and less likely to cause injury.
But the belt is my personal choice because it allows me more freedom of movement.