TIL that the name of the PEM block of encoded public keys matters. I had a key that was using a block name
PUBLIC KEY, where it should’ve been
RSA PUBLIC KEY. Therefore, the client was rejecting the key as it was decoding it using the incorrect format. Here’s a very nice post about this.