When the client authenticates using a smart card we store the smart card reader used as part of the single sign-on information. This is necessary in case multiple readers are available. Using the stored token passphrase with the wrong reader will likely not work.
Unfortunately there is currently no way for users to get access to this information. We have internal APIs for it that we used in our rdesktop integration (which is now removed), but we never provided a command for it.