Bug 1623374 Need to support the new PKCS #11 Message interface for AE…

…S GCM and ChaCha Poly

PKCS #11 defines a new interface for handling AEAD type ciphers that allow
multiple AEAD operations without repeating the key schedule. It also allows
tokens to keep track of the number of operations, and generate IVs (depending
on the cipher).

This patch:
    1. implement those new functions in softoken.
       With the addition of CKF_MESSAGE_* flags to various mechanism, we need
       to strip them when using the version 2 API of softoken (since there are
       no C_Message* function in version 2). For that we need a separate
       C_GetMechanismInfo function. We use the same trick we used to have
       a separate version function for the V2 interface.
       Also now that the new message functions are in their own file, they
       still need access to the common Session state processing functions.
       those have gone from static to exported within softoken to accomidate
       that. Same with sftk_MapDecryptError() (sftk_MapVerifyError() was also
       made global, though nothing else is yet using it).
       Only C_MessageEncrptInit(), C_EncryptMessage(), C_MessageEncryptFinal,
       C_MessageDecryptInit(), C_DecryptMessage(), and C_MessageDecryptFinal
       are implemented. C_EncryptMessageBegin(), C_EncryptMessageNext(),
       C_DecryptMessageBegin(), and C_DecryptMessageNext() are all
       part of the multi-part withing a multi-part operation and
       are only necessary for things like S/MIME (potentially). If we wanted
       to implement them, we would need more functions exported from freebl
       (and initaead, updateaead, finalaead for each mechanism type).
    2. make those interfaces call aes_gcm and chacha20_poly1503
    (and make adjustments for those ciphers).
      For AES, I added a new function AES_AEAD, which handles both encrypt
      and decrypt. Internally, the gcm functions (both the generic gcm and
      the intel gcm wrapper) had their init functions split into key scheduling
      and counter mode/tag initialization. The latter is still called from
      init, but the former is now for each update call. IV generation is
      handled by a single function in gcm.c, and shared with intel_gcm_wrapper.c
      Since the AES functions already know about the underlying PKCS #11
      mechanism parameters, the new AEAD functions also parse the PKCS #11 GCM
      parameters.
      For Chacha/Poly new aead update functions were created called
      ChaChaPoly1305_Encrypt and ChaChaChaPoly1305_Decrypt. There was no
      Message specific initialization in the existing chacha_init, so no
      changes were needed there. The primary difference between
      _Encrypt/_Decrypt and _Seal/_Open is the fact that the tag is put at
      the end of the encrypted data buffer in the latter, and in a generic
      buffer in the former.
    3. create new pk11wrap interfaces that also squash the api differences
    between the various mechanisms for aead (similiar to the way we do it for
    CBC and ECB crypto today).
      To accomplish this I added PK11_AEADOp() and PK11_AEADRawOp(). Both
      functions handle the case where the token only supports the single shot
      interface, by using the single short interface to simulate the
      Message interface. The PK11_AEADOp() also smooths out the
      differences in the parameters and symantics of the various mechanism
      so the application does not need to worry about the PKCS #11 differences
      in the mechanism. Both use contexts from the standard
      PK11_CreateContext(), so key schedules are done once for each key rather
      than once for each message. MESSAGE/AEAD operations are selected by adding
      the psuedo attribute flag CKA_NSS_MESSAGE to the requested operation
      (CKA_ENCRYPT, CKA_DECRYPT, CKA_SIGN, CKA_VERIFY).
    4. write tests for the new interfaces
      Tests were added to make sure the PK11_AEADRawOp interface works,
      The single shot interface is used to test output of the message interface
      we also use two test only functions to force the connection to use
      the simulation interface, which is also compared to the non-simulate
      inteface. The AES_GCM also tests various IV generators.

Differential Revision: https://phabricator.services.mozilla.com/D67552

automation/abi-check/expected-report-libnss3.so.txt

@@ -1,3 +1,9 @@
+4 Added functions:
+
+  'function SECStatus PK11_AEADOp(PK11Context*, CK_GENERATOR_FUNCTION, int, unsigned char*, int, const unsigned char*, int, unsigned char*, int*, int, unsigned char*, int, const unsigned char*, int)'    {PK11_AEADOp@@NSS_3.52}
+  'function SECStatus PK11_AEADRawOp(PK11Context*, void*, int, const unsigned char*, int, unsigned char*, int*, int, const unsigned char*, int)'    {PK11_AEADRawOp@@NSS_3.52}
+  'function PRBool _PK11_ContextGetAEADSimulation(PK11Context*)'    {_PK11_ContextGetAEADSimulation@@NSS_3.52}
+  'function SECStatus _PK11_ContextSetAEADSimulation(PK11Context*)'    {_PK11_ContextSetAEADSimulation@@NSS_3.52}
 
 1 function with some indirect sub-type change:
 

gtests/pk11_gtest/pk11_aes_gcm_unittest.cc

@@ -7,6 +7,7 @@
 #include <memory>
 #include "nss.h"
 #include "pk11pub.h"
+#include "pk11priv.h"
 #include "secerr.h"
 #include "sechash.h"
 
@@ -140,6 +141,211 @@ class Pkcs11AesGcmTest : public ::testing::TestWithParam<AesGcmKatValue> {
                         &output_len, output.size(), data.data(), data.size());
   }
 
+  SECStatus MessageInterfaceTest(int iterations, int ivFixedBits,
+                                 CK_GENERATOR_FUNCTION ivGen,
+                                 PRBool separateTag) {
+    // Generate a random key.
+    ScopedPK11SlotInfo slot(PK11_GetInternalSlot());
+    EXPECT_NE(nullptr, slot);
+    ScopedPK11SymKey sym_key(
+        PK11_KeyGen(slot.get(), mech, nullptr, 16, nullptr));
+    EXPECT_NE(nullptr, sym_key);
+
+    const int kTagSize = 16;
+    int cipher_simulated_size;
+    int output_len_message = 0;
+    int output_len_simulated = 0;
+    unsigned int output_len_v24 = 0;
+
+    std::vector<uint8_t> plainIn(17);
+    std::vector<uint8_t> plainOut_message(17);
+    std::vector<uint8_t> plainOut_simulated(17);
+    std::vector<uint8_t> plainOut_v24(17);
+    std::vector<uint8_t> iv(16);
+    std::vector<uint8_t> iv_init(16);
+    std::vector<uint8_t> iv_simulated(16);
+    std::vector<uint8_t> cipher_message(33);
+    std::vector<uint8_t> cipher_simulated(33);
+    std::vector<uint8_t> cipher_v24(33);
+    std::vector<uint8_t> aad(16);
+    std::vector<uint8_t> tag_message(16);
+    std::vector<uint8_t> tag_simulated(16);
+
+    // Prepare AEAD v2.40 params.
+    CK_GCM_PARAMS_V3 gcm_params;
+    gcm_params.pIv = iv.data();
+    gcm_params.ulIvLen = iv.size();
+    gcm_params.ulIvBits = iv.size() * 8;
+    gcm_params.pAAD = aad.data();
+    gcm_params.ulAADLen = aad.size();
+    gcm_params.ulTagBits = kTagSize * 8;
+
+    // Prepare AEAD MESSAGE params.
+    CK_GCM_MESSAGE_PARAMS gcm_message_params;
+    gcm_message_params.pIv = iv.data();
+    gcm_message_params.ulIvLen = iv.size();
+    gcm_message_params.ulTagBits = kTagSize * 8;
+    gcm_message_params.ulIvFixedBits = ivFixedBits;
+    gcm_message_params.ivGenerator = ivGen;
+    if (separateTag) {
+      gcm_message_params.pTag = tag_message.data();
+    } else {
+      gcm_message_params.pTag = cipher_message.data() + plainIn.size();
+    }
+
+    // Prepare AEAD MESSAGE params for simulated case
+    CK_GCM_MESSAGE_PARAMS gcm_simulated_params;
+    gcm_simulated_params = gcm_message_params;
+    if (separateTag) {
+      // The simulated case, we have to allocate temp bufs for separate
+      // tags, make sure that works in both the encrypt and the decrypt
+      // cases.
+      gcm_simulated_params.pTag = tag_simulated.data();
+      cipher_simulated_size = cipher_simulated.size() - kTagSize;
+    } else {
+      gcm_simulated_params.pTag = cipher_simulated.data() + plainIn.size();
+      cipher_simulated_size = cipher_simulated.size();
+    }
+    /* when we are using CKG_GENERATE_RANDOM, don't independently generate
+     * the IV in the simulated case. Since the IV's would be random, none of
+     * the generated results would be the same. Just use the IV we generated
+     * in message interface */
+    if (ivGen == CKG_GENERATE_RANDOM) {
+      gcm_simulated_params.ivGenerator = CKG_NO_GENERATE;
+    } else {
+      gcm_simulated_params.pIv = iv_simulated.data();
+    }
+
+    SECItem params = {siBuffer, reinterpret_cast<unsigned char*>(&gcm_params),
+                      sizeof(gcm_params)};
+    SECItem empty = {siBuffer, NULL, 0};
+
+    // initialize our plain text, IV and aad.
+    EXPECT_EQ(PK11_GenerateRandom(plainIn.data(), plainIn.size()), SECSuccess);
+    EXPECT_EQ(PK11_GenerateRandom(aad.data(), aad.size()), SECSuccess);
+    EXPECT_EQ(PK11_GenerateRandom(iv_init.data(), iv_init.size()), SECSuccess);
+    iv_simulated = iv_init;  // vector assignment actually copies data
+    iv = iv_init;
+
+    // Initialize message encrypt context
+    ScopedPK11Context encrypt_message_context(PK11_CreateContextBySymKey(
+        mech, CKA_NSS_MESSAGE | CKA_ENCRYPT, sym_key.get(), &empty));
+    EXPECT_NE(nullptr, encrypt_message_context);
+    if (!encrypt_message_context) {
+      return SECFailure;
+    }
+    EXPECT_FALSE(_PK11_ContextGetAEADSimulation(encrypt_message_context.get()));
+
+    // Initialize simulated encrypt context
+    ScopedPK11Context encrypt_simulated_context(PK11_CreateContextBySymKey(
+        mech, CKA_NSS_MESSAGE | CKA_ENCRYPT, sym_key.get(), &empty));
+    EXPECT_NE(nullptr, encrypt_simulated_context);
+    if (!encrypt_simulated_context) {
+      return SECFailure;
+    }
+    EXPECT_EQ(SECSuccess,
+              _PK11_ContextSetAEADSimulation(encrypt_simulated_context.get()));
+
+    // Initialize message decrypt context
+    ScopedPK11Context decrypt_message_context(PK11_CreateContextBySymKey(
+        mech, CKA_NSS_MESSAGE | CKA_DECRYPT, sym_key.get(), &empty));
+    EXPECT_NE(nullptr, decrypt_message_context);
+    if (!decrypt_message_context) {
+      return SECFailure;
+    }
+    EXPECT_FALSE(_PK11_ContextGetAEADSimulation(decrypt_message_context.get()));
+
+    // Initialize simulated decrypt context
+    ScopedPK11Context decrypt_simulated_context(PK11_CreateContextBySymKey(
+        mech, CKA_NSS_MESSAGE | CKA_DECRYPT, sym_key.get(), &empty));
+    EXPECT_NE(nullptr, decrypt_simulated_context);
+    if (!decrypt_simulated_context) {
+      return SECFailure;
+    }
+    EXPECT_EQ(SECSuccess,
+              _PK11_ContextSetAEADSimulation(decrypt_simulated_context.get()));
+
+    // Now walk down our iterations. Each method of calculating the operation
+    // should agree at each step.
+    for (int i = 0; i < iterations; i++) {
+      SECStatus rv;
+      /* recopy the initial vector each time */
+      iv_simulated = iv_init;
+      iv = iv_init;
+
+      // First encrypt. We don't test the error code here, because
+      // we may be testing error conditions with this function (namely
+      // do we fail if we try to generate to many Random IV's).
+      rv =
+          PK11_AEADRawOp(encrypt_message_context.get(), &gcm_message_params,
+                         sizeof(gcm_message_params), aad.data(), aad.size(),
+                         cipher_message.data(), &output_len_message,
+                         cipher_message.size(), plainIn.data(), plainIn.size());
+      if (rv != SECSuccess) {
+        return rv;
+      }
+      rv =
+          PK11_AEADRawOp(encrypt_simulated_context.get(), &gcm_simulated_params,
+                         sizeof(gcm_simulated_params), aad.data(), aad.size(),
+                         cipher_simulated.data(), &output_len_simulated,
+                         cipher_simulated_size, plainIn.data(), plainIn.size());
+      if (rv != SECSuccess) {
+        return rv;
+      }
+      // make sure simulated and message is the same
+      EXPECT_EQ(output_len_message, output_len_simulated);
+      EXPECT_EQ(0, memcmp(cipher_message.data(), cipher_simulated.data(),
+                          output_len_message));
+      EXPECT_EQ(0, memcmp(gcm_message_params.pTag, gcm_simulated_params.pTag,
+                          kTagSize));
+      EXPECT_EQ(0, memcmp(iv.data(), gcm_simulated_params.pIv, iv.size()));
+      // make sure v2.40 is the same. it inherits the generated iv from
+      // encrypt_message_context.
+      EXPECT_EQ(SECSuccess,
+                PK11_Encrypt(sym_key.get(), mech, &params, cipher_v24.data(),
+                             &output_len_v24, cipher_v24.size(), plainIn.data(),
+                             plainIn.size()));
+      EXPECT_EQ(output_len_message, (int)output_len_v24 - kTagSize);
+      EXPECT_EQ(0, memcmp(cipher_message.data(), cipher_v24.data(),
+                          output_len_message));
+      EXPECT_EQ(0, memcmp(gcm_message_params.pTag,
+                          cipher_v24.data() + output_len_message, kTagSize));
+      // now make sure we can decrypt
+      EXPECT_EQ(SECSuccess,
+                PK11_AEADRawOp(decrypt_message_context.get(),
+                               &gcm_message_params, sizeof(gcm_message_params),
+                               aad.data(), aad.size(), plainOut_message.data(),
+                               &output_len_message, plainOut_message.size(),
+                               cipher_message.data(), output_len_message));
+      EXPECT_EQ(output_len_message, (int)plainIn.size());
+      EXPECT_EQ(
+          0, memcmp(plainOut_message.data(), plainIn.data(), plainIn.size()));
+      EXPECT_EQ(
+          SECSuccess,
+          PK11_AEADRawOp(decrypt_simulated_context.get(), &gcm_simulated_params,
+                         sizeof(gcm_simulated_params), aad.data(), aad.size(),
+                         plainOut_simulated.data(), &output_len_simulated,
+                         plainOut_simulated.size(), cipher_message.data(),
+                         output_len_simulated));
+      EXPECT_EQ(output_len_simulated, (int)plainIn.size());
+      EXPECT_EQ(
+          0, memcmp(plainOut_simulated.data(), plainIn.data(), plainIn.size()));
+      if (separateTag) {
+        // in the separateTag case, we need to copy the tag back to the
+        // end of the cipher_message.data() before using the v2.4 interface
+        memcpy(cipher_message.data() + output_len_message,
+               gcm_message_params.pTag, kTagSize);
+      }
+      EXPECT_EQ(SECSuccess,
+                PK11_Decrypt(sym_key.get(), mech, &params, plainOut_v24.data(),
+                             &output_len_v24, plainOut_v24.size(),
+                             cipher_message.data(), output_len_v24));
+      EXPECT_EQ(output_len_v24, plainIn.size());
+      EXPECT_EQ(0, memcmp(plainOut_v24.data(), plainIn.data(), plainIn.size()));
+    }
+    return SECSuccess;
+  }
+
   const CK_MECHANISM_TYPE mech = CKM_AES_GCM;
 };
 
@@ -166,4 +372,57 @@ TEST_F(Pkcs11AesGcmTest, TwelveByteZeroIV) {
   EXPECT_EQ(SECSuccess, EncryptWithIV(iv));
 }
 
+// basic message interface it's the most common configuration
+TEST_F(Pkcs11AesGcmTest, MessageInterfaceBasic) {
+  EXPECT_EQ(SECSuccess,
+            MessageInterfaceTest(16, 0, CKG_GENERATE_COUNTER, PR_FALSE));
+}
+
+// basic interface, but return the tags in a separate buffer. This triggers
+// different behaviour in the simulated case, which has to buffer the
+// intermediate values in a separate buffer.
+TEST_F(Pkcs11AesGcmTest, MessageInterfaceSeparateTags) {
+  EXPECT_EQ(SECSuccess,
+            MessageInterfaceTest(16, 0, CKG_GENERATE_COUNTER, PR_TRUE));
+}
+
+// test the case where we are only allowing a portion of the iv to be generated
+TEST_F(Pkcs11AesGcmTest, MessageInterfaceIVMask) {
+  EXPECT_EQ(SECSuccess,
+            MessageInterfaceTest(16, 124, CKG_GENERATE_COUNTER, PR_FALSE));
+}
+
+// test the case where we using the tls1.3 iv generation
+TEST_F(Pkcs11AesGcmTest, MessageInterfaceXorCounter) {
+  EXPECT_EQ(SECSuccess,
+            MessageInterfaceTest(16, 0, CKG_GENERATE_COUNTER_XOR, PR_FALSE));
+}
+
+// test the case where we overflow the counter (requires restricted iv)
+// 128-124 = 4 bits;
+TEST_F(Pkcs11AesGcmTest, MessageInterfaceCounterOverflow) {
+  EXPECT_EQ(SECFailure,
+            MessageInterfaceTest(17, 124, CKG_GENERATE_COUNTER, PR_FALSE));
+}
+
+// overflow the tla1.2 iv case
+TEST_F(Pkcs11AesGcmTest, MessageInterfaceXorCounterOverflow) {
+  EXPECT_EQ(SECFailure,
+            MessageInterfaceTest(17, 124, CKG_GENERATE_COUNTER_XOR, PR_FALSE));
+}
+
+// test random generation of the IV (uses an aligned restricted iv)
+TEST_F(Pkcs11AesGcmTest, MessageInterfaceRandomIV) {
+  EXPECT_EQ(SECSuccess,
+            MessageInterfaceTest(16, 56, CKG_GENERATE_RANDOM, PR_FALSE));
+}
+
+// test the case where we try to generate too many random IVs for the size of
+// our our restricted IV (notice for counters, we can generate 16 IV with
+// 4 bits, but for random we need at least 72 bits to generate 16 IVs).
+// 128-56 = 72 bits
+TEST_F(Pkcs11AesGcmTest, MessageInterfaceRandomOverflow) {
+  EXPECT_EQ(SECFailure,
+            MessageInterfaceTest(17, 56, CKG_GENERATE_RANDOM, PR_FALSE));
+}
 }  // namespace nss_test