What it could be.
barrell heat up
are the brass fired in this rifle before?
could be a scope issue, Minimal possibility. But you have a minimal problem.
could be a flinch...Just sayin.
wind.
primer flash hole size or pluggage from case cleaner
type of bullet.
is it a mixture of military and other brass?
Question.
After the flyer does it settle back in the same hole cutting zero??
if this is the case.. and the flyer is enough to bother you while hunting, I recomend trying a different scope. at least eliminmate the problem,
Care to share your recipe?? Sounds interresting.