Are you sure it was a permit and not a pompano?
I guess I can’t say definitively, since we didn’t take a pic or contact an expert, but it was definitely different from the pompano we were catching. The color was much different, just gray/silver with a little hint of orange around the anal fin. No other bright color anywhere on it. Also, the nose/forehead bump (don’t know what it’s called) looked different. Did some quick googling while we had it out of the water and looked just like a young permit.