You are giving turkeys too much credit for their intellect. Deer hunt it and don't worry about it. If it's good habitat, they'll be back. Turkeys are chased by every carnivore that lives in the woods so they're accustomed to being flushed out of spots very frequently. Being pushed by you on a few deer hunts each year won't make a difference.
Bottom line they know when hunting season is. And granted you live in an area where there are not many poachers or a fall season, they don't mind the summer/fall woods very much