Right now we require SHA1_RSA_PKCS; it would be useful to support others when this is not available.
We would still require the crypto algorithms that are necessary for SSH (i.e. RSA, and possible ECDSA in the future), so this is about doing the hashing and PKCS#7 stuff in tlclient and just use the PKCS#11 module for the raw crypto.
One question is we should support CKM_RSA_X_509, CKM_RSA_PKCS or both. NSS apparently only uses CKM_RSA_PKCS: https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/PKCS11_Implement
We need a SHA-1 implementation that we can link into tlclient. NetBSD seems to has one, which should be sufficient licence wise: https://github.com/search?l=c&q=%22public+domain%22+%22sha1%22&ref=searchresults&type=Code
We will probably need more hashing algorithms in the future, so let's use nettle which we already have in the build system. It is LGPL so there is no problem linking to it.
Works fine. Tester should verify that authentication still works. For the paranoid I've also built a special opensc without SHA1_RSA_PKCS support (~ossman/tmp/opensc-nosha.so).
Tested that login using smart card still works on MacOSX, ARM and Win64 with one of the new testcard with RSA2048 key.