Round 1 - Carburetor Cleaner will kill them faster than wasp spray. Take the little red extension tube and run it up in the hole and give it a good soaking. That will keep them out of the hole for the rest of the season and kill whatever is in there. It is kinda like yellow jackets do it at night when they are in the hole. You'll knock a serious dent in the population.
Round 2 - The preventative maintenance is a couple of traps. I do not know how to permanently get rid of them apparently whatever they treat wood with these days doesn't bother them they'll bore a hole right into a piece of treated lumber.
