Maybe instead of working against him, you could work with him to help each other and see if he will monitor your land - not hunt it, but help keep watch - for poachers or illegal activities.
He has 4 acres. You have 320. He's doing things, obviously, that he believes are helping him from a management strategy. You're doing the same things, by your account.
Unless there's some kind of prior bad blood, team up and help make each other's place somewhere both of you enjoy.
If you're in central Florida, pissing him off might not be the best thing for you if he knows you don't get up there often.
Good luck.
Maybe the deer are coming across the line from FL too!! Then what??
Then the rut dates change!
