So I'm wondering if your permit did have 'restrictions' if he would have felt obliged to find out what handgun you had.
BTW, a text book script on handling a stop. With or without a firearm. I had a similar non-firearm stop last year. At the conclusion the officer and I shook hands. My compliments.