I wouldn't worry about it honestly. You just never know about them things. He may be back in that same spot every day or he may move around a little. In my experience they are all a little different.
If you called to him earlier with a turkey call, he could have been simply just waiting for that "hen" to come to him. Someone else could have drove down the road and made him hold up in his tree a little more. Bottom line, don't worry about it and good luck in the morning, I would think he would be fairly close by.
