patch by Thomas Klausner
r=rrelyea

In the NetBSD configuration, the symbol hiding flags are not defined. This leads to conflicts when openssl and nss are linked into the same binary.
For a longer discussion on the topic, see
https://groups.google.com/a/mozilla.org/g/dev-tech-crypto/c/Al0Pt0zhARE


Match more closely to OpenBSD.mk, and in particular, hide symbols (MAPFILE).

- fix wrong value of CPU_ARCH on NetBSD/evbarm-earmv7f
- s/aarch64eb/aarch64/

Differential Revision: https://phabricator.services.mozilla.com/D114443

--HG--
extra : moz-landing-system : lando

Bug 1566124 - Optimize AES-GCM for ppc64le Differential Revision: https://phabricator.services.mozilla.com/D108221 r=bbeurdouche

Differential Revision: https://phabricator.services.mozilla.com/D112302

--HG--
extra : moz-landing-system : lando

Differential Revision: https://phabricator.services.mozilla.com/D108734

--HG--
extra : rebase_source : ffa43f8cd1f341d56e8782f81a2aa24243533614

The main changes here are:

* an update to HPKE -08
* a move to the single-byte configuration ID
* reordering of ECHConfig

The addition of the explicit configuration ID means that the API for
constructing ECHConfig(List) needs to change.  That means a name change,
unfortunately.  I took the opportunity to make further changes to the arguments.

Differential Revision: https://phabricator.services.mozilla.com/D108392

--HG--
extra : rebase_source : e66ae86be746afbadc6c444d0debcc5aaabd2eb5

This adds the final HPKE version string.

This removes the draft version markers from the implementation and stops
tracking the draft version with the exported syntax.

I've added the script that I used to convert the JSON test vectors from the
specification; that should allow us to pick up new tests relatively easily,
especially if we need to add new algorithms.

This change breaks several ECH test cases.  As fixing those tests is
extraordinarily fiddly, I'm going to defer making those changes until we need to
update ECH.  As we can't land this code until ECH is updated to depend on the
final HPKE and until we have coordinated with servers on when the ECH update can
be deployed, it should be OK to defer.

In short, don't land this without the matching ECH changes.

Differential Revision: https://phabricator.services.mozilla.com/D105256

--HG--
extra : rebase_source : b0717403cf5136efc14f85499182763aa551efc3

Also fixes:
Bug 452464 pk12util -o fails when -C option specifies AES or Camellia ciphers

Related:
Bug 1694689 Firefox should use modern algorithms in PKCS#12 files by default
Bug 452471 pk12util -o fails when -c option specifies pkcs12v2 PBE ciphers


The base of this fix is was a simple 3 line fix in pkcs12.c, changing the initial setting of cipher and cert cipher.

Overview for why this patch is larger than just 3 lines:
1. First issue was found in trying to change the mac hashing value.
   a. While the decrypt side knew how to handle SHA2 hashes, the equivalent code was not updated on the encrypt side. I refactored that code and placed the common function in p12local.c. Now p12e.c and p12d.c share common code to find the required function to produce the mac key.
   b. The prf hmac was hard coded to SHA1. I changed the code to pass the hmac matching the hashing algorithm for the mac. This required changes to p12e.c to calculate and pass the new hmac as well and adding new PK11_ExportEncryptedPrivateKey and PK11_ExportEncryptedPrivKey to take the PKCS #5 v2 parameters. I also corrected an error which prevented pkcs12 encoding of ciphers other than AES.
2. Once I've made my changes, I realized we didn't have a way of testing them. While we had code that verified that particular sets of parameters for pkcs12 worked together and could be listed and imported, we didn't have a way to verify what algorithms were actually generated by our tools.
  a. pk12util -l doesn't list the encryption used for the certs, so I updated pp to take a pkcs12 option. In doing so I had to update pp to handle indefinite encoding when decoding blocks. I also factored that decoding out in it's own function so the change only needed to be placed once. Finally I renabled a function which prints the output of an EncryptedPrivate key. This function was disabled long ago when the Encrypted Private key info was made private for NSS. It has since been exported, so these functions could easily be enabled (archeological note: I verified that this disabling was not a recent think I found I had done it back when I still have a netscape email address;).
 b. I updated tools.sh to us the new pp -t pkcs12 feature to verify that the key encryption, cert encryption, and hash functions matched what we expected when we exported a new key. I also updated tools.sh to handle the new hash variable option to pk12util.
 c. I discovered several tests commented out with comments that the don't work. I enabled those tests and discovered that they can now encrypt, but the can't decrypt because of pkcs12 policy. I updated the policy code, but I updated it to use the new NSS system wide policy mechanism. This enabled all the ciphers to work. There is still policy work to do. The pk12 policy currently only prevents ciphers from use in decrypting the certificates, not decrypting the keys and not encrypting. I left that for future work.
3. New options for pp and pk12util were added to the man pages for these tools.

---------------------------------------------------------------------------
With that in mind, here's a file by file description of the patch:

automation/abi-check/expected-report-libnss3.so.txt
-Add new exported functions. (see lib/nss/nss.def)

cmd/lib/basicutil.h:
-Removed the HAVE_EPV_TEMPLATE ifdefs (NSS has exported the Encrypted Private Key data structure for a while now.

cmd/lib/secutil.c:
global: Updated several functions to take a const char * m (message) rather than a char * m
global: Made the various PrintPKCS7 return an error code.
global: Added a state variable to be passed around the various PKCS7 Print functions. It gives the proper context to interpret PKCS7 Data Content. PKCS 12 used PKCS7 to package the various PKCS12 Safes and Bags.
-Updated SECU_StripTagAndLength to handle indefinite encoding, and to set the Error code.
-Added SECU_ExtractDERAndStep to grab the next DER Tag, Length, and Data.
-Updated secu_PrintRawStringQuotesOptional to remove the inline DER parsing and use SECU_ExtractDERAndStep().
-Updated SECU_PrintEncodedObjectID to return the SECOidTag just like SECU_PrintObjectID.
-Renable SECU_PrintPrivateKey
-Added secu_PrintPKCS12Attributes to print out the Attributes tied to a PKCS #12 Bag
-Added secu_PrintPKCS12Bag to print out a PKCS #12 Bag
-Added secu_PrintPKCS7Data, which uses the state to determine what it was printing out.
-Added secu_PrintDERPKCS7ContentInfo which is identical to the global function SECU_PrintPKCS7ContentInfo except it takes a state variable. The latter function now calls the former.
-Added secu_PrintPKCS12DigestInfo to print the Hash information of the Mac. DigestInfo is the name in the PKCS 12 spec.
-Added secu_PrintPKCS12MacData to print the Mac portion of the PKCS 12 file.
-Added SECU_PrintPKCS12 to print otu the pkcs12 file.

cmd/lib/secutil.h
-Added string for pkc12 for the command line of pp
reenabled SECU_PrintPrivateKey
-Added SECU_PrintPKCS12 for export.

cmd/pk12util/pk12util.c
-Added the -M option to specify a hash algorithm for the mac.
updated P12U_ExportPKCS12Object: pass the hash algorithm to the PasswordIntegrity handler.
-Added PKCS12U_FindTagFromString: generalized string to SECOidTag which only filters based on the oid having a matching PKCS #11 mechanism.
updated PKCS12U_MapCipherFromString to call use PKCS12U_FindTagFromString to get the candidate tag before doing it's post processing to decide if the tag is really an encryption algorithm.
-Added PKCS12U_MapHashFromString with is like MapCipherFromString except it verifies the resulting tag is a hash object.
-Updated main to 1) change the default cipher, change the default certCipher, and process the new hash argument. NOTE: in the old code we did not encrypt the certs in FIPS mode. That's because the certs were encrypted with RC4 in the default pkcs12 file, which wasn't a FIPS algorithm. Since AES is, we can use it independent on whether or not we are in FIPS mode.

cmd/pp/pp.c
-Added the pkcs12 option which calls SECU_PrintPKCS12 from secutil.c

lib/nss/nss.def
-Add exports to the new PK11_ExportEncryptedPrivKeyInfoV2 and PK11_ExportEncryptedPrivateKeyInfoV2 (V2 means PKCS 5 v2, not Version 2 of ExportEncrypted*Info).
-Add export for the old HASH_GetHMACOidTagByHashOidTag which should have been exported long ago to avoid the proliferation of copies of this function in places like ssl.

lib/pk11wrap/pk11akey.c
-Add PK11_ExportEncryptedPrivKeyInfoV2 (which the old function now calls), which takes the 3 PKCS 5 v2 parameters. The underlying pkcs5 code can fill in missing tags if necessary, but supplying all three gives the caller full control of the underlying pkcs5 PBE used.
-Add PK11_ExportEncryptedPrivateKeyInfoV2, same as the above function except it takes a cert which is used to look up the private key. It's the function that pkcs12 actually uses, but the former was exported for completeness.

lib/pk11wrap/pk11pub.h
-Added the new PK11_ExportEncryptedPriv*KeyInfoV2 functions.

lib/pkcs12/p12d.c
-Remove the switch statement and place it in p12local.c so that p12e.c can use the same function.

lib/pkc12/p12e.c
-Remove the unnecessary privAlg check so we can encode any mechanism we support. This only prevented encoding certificates in the pk12 file, not the keys.
-add code to get the hmac used in the pbe prf from the integrity hash, which is under application control.
-Do the same for key encryption, then use the new PK11_ExportEncryptedPrivateKeyInfo to pass that hash value.
-Use the new sec_pkcs12_algtag_to_keygen_mech so there is only one switch statement to update rather than 2.
-Update the hash data to old the length of the largest hash rather than the length of a SHA1 hash.

lib/pkcs12/p12local.c
- Add new function new sec_pkcs12_algtag_to_keygen_mech to factor out the common switch statement between p12e and p12d.

lib/pkcs12/p12local.h
-Export the new sec_pkcs12_algtag_to_keygen_mech

lib/pkcs12/p12plcy.c
-Map the old p12 policy functions to use the new NSS_GetAlgorithmPolicy. We keep the old table so that applications can change the policy with the old PKCS12 specific defines (so the old code keeps working). NOTE: policies now default to true rather than false.

lib/util/secoidt.h
-Add new NSS_USE_ALG_IN_PKCS12 used by pk11plcy.c
NOTE: I have not updated the policy table in pk11wrap/pk11pars.c, so we can't yet control pkcs12 policy with the nss system policy table. That's a patch for another time.

test/tools/tool.sh
-global: Remove trailing spaces
-global: DEFAULT is changed to 'default'
-Update the PBE mechanism to exactly match the string in secoid.c. PKCS #12 does case independent compares, so case doesn't matter there, but now I'm comparing to the output of pp, and I didn't want to spend the time to figure out case independent compares in bash.
-Add our defauts and shell variables at the top so there are easy to change in the future.
export_with_*** have all been colapsed into a single export_p12_file which handles taking 'default' and turning off that argument.
-Add for loops for the hash functions.
-Restore the camellia ciphers back now that they work.
-Restore the pkcs12V2pbe back now that they work.
-Collect various pbe types into single variables and use those variables in loops
-Reduce the number of tests ran in optimized mode (which takes 60x the time to do a pbe then than debug mode based on a larger iterator).
-Add verify_p12 which dumps out the p12 file and makes sure the expected CERT_ENCRYPTION, KEY_ENCRYPTION, and HASH are used.

doc/pp.xml
-Add pkcs12 option

doc/pk12util.xml
-Add -M option
-Update synopsis with options in the description but not in the synopsis

Differential Revision: https://phabricator.services.mozilla.com/D113699

Redhat has run our scanners on the full NSS tree and identifed 123
errors our security team determined where 'critical'. I've reviewed
the errors and identified a much smaller subset of errors that are
either real, or confusing enough to warrent suppression comments.

Many errors are in cmd and gtest. I've skip those commands red hat does
not ship in this report, and I've skipped the issues in gtest.

Also, There's a large number of leaked_storage errors because evidently
coverity gets confused when you have a pointer in a local variable and
you pass that pointer off to a global or a function variable. I've
skipped most of those as well.

changes:
crlutil.c: add missing arena free in error path. #def4
secutil.c: (Not coverity found) make sure we don't overflow our buffer
           in badly encoded ECC oids.
modultil.c: Fix incorrect free operation in pk11install case. #def6
signtool/javascript.c: free old archiveDir after use in PR_smprintf #def8
                       don't double free curitem (curitem is almost
                       certainly NULL at this point, so the current code
                       is a noop). #def9, #def10
signtool/list.c: remove unused ugly_list variable (which is leaked) #def12
signtool/util.c: don't leak 'dir' in error path #def13 #def14
sigver/pk7print.c: coverity: use static pk rather than an allocated and
                   leaked pointer. #def15
                   add code for EC
                   disbled DSA code that isn't actually working
                   (PQG params).
symkeyutil.c: free name (depends on PORT_Free null check). #def16
pkix_pl_nameconstraints.c: Fix coverity double free warning.
                           PKIX_ERROR_RECEIVED is almost
                           certainly false in this case, but by
                           setting arena to NULL we make
                           sure it's not used or freed again. #def99
pkix_pl_string.c: Fix varargs leak in the error path. #def109
pk11parse.c: secmod_doDescCopy can reallocate our newSpec, but the
             pointer we are passed is an offset from newSpec. Pass in
             both pointers and return our newly allocated spec and
             length in that case.#def113
cmsutil.c: suppress cppcheck warnings do to cmsutil use of unions to
           cast pointers. #def113-117
pkcs11.c: support coverity incorrect use_after_free warning. #def118

scanner errors:
Error: USE_AFTER_FREE (CWE-416): [#def4]
nss-3.60.1/nss/cmd/crlutil/crlutil.c:389: freed_arg: "PORT_FreeArena_Util" frees "modArena".
nss-3.60.1/nss/cmd/crlutil/crlutil.c:455: double_free: Calling "PORT_FreeArena_Util" frees pointer "modArena" which has already been freed.
453| }
454| if (modArena && (!modCrl || modCrl->arena != modArena)) {
455|-> PORT_FreeArena(modArena, PR_FALSE);
456| }
457| if (modCrl)

Error: BAD_FREE (CWE-590): [#def6]
nss-3.60.1/nss/cmd/modutil/install-ds.c:1046: address_free: "PR_Free" frees address of "_this->forwardCompatible".
1044| Pk11Install_PlatformName_delete(&_this->forwardCompatible[i]);
1045| }
1046|-> PR_Free(&_this->forwardCompatible);
1047| _this->numForwardCompatible = 0;
1048| }

Error: USE_AFTER_FREE (CWE-416): [#def8]
nss-3.60.1/nss/cmd/signtool/javascript.c:1346: freed_arg: "PR_Free" frees "archiveDir".
nss-3.60.1/nss/cmd/signtool/javascript.c:1347: pass_freed_arg: Passing freed pointer "archiveDir" as an argument to "PR_smprintf".
1345| warningCount++;
1346| PR_Free(archiveDir);
1347|-> archiveDir = PR_smprintf("%s.arc", archiveDir);
1348| } else {
1349| PL_strcpy(archiveDir + strlen(archiveDir) - 4, ".arc");

Error: USE_AFTER_FREE (CWE-416): [#def9]
nss-3.60.1/nss/cmd/signtool/javascript.c:1477: alias: Assigning: "entityListTail" = "entityItem". Now both point to the same storage.
nss-3.60.1/nss/cmd/signtool/javascript.c:1477: alias: Assigning: "entityList" = "entityListTail". Now both point to the same storage.
nss-3.60.1/nss/cmd/signtool/javascript.c:1623: alias: Assigning: "curitem" = "entityList". Now both point to the same storage.
nss-3.60.1/nss/cmd/signtool/javascript.c:1651: freed_arg: "PR_Free" frees "entityListTail".
nss-3.60.1/nss/cmd/signtool/javascript.c:1654: double_free: Calling "PR_Free" frees pointer "curitem" which has already been freed.
1652| }
1653| if (curitem) {
1654|-> PR_Free(curitem);
1655| }
1656| if (basedir) {

Error: USE_AFTER_FREE (CWE-416): [#def10]
nss-3.60.1/nss/cmd/signtool/javascript.c:1477: alias: Assigning: "entityListTail" = "entityItem". Now both point to the same storage.
nss-3.60.1/nss/cmd/signtool/javascript.c:1477: alias: Assigning: "entityList" = "entityListTail". Now both point to the same storage.
nss-3.60.1/nss/cmd/signtool/javascript.c:1623: alias: Assigning: "curitem" = "entityList". Now both point to the same storage.
nss-3.60.1/nss/cmd/signtool/javascript.c:1651: freed_arg: "PR_Free" frees "entityListTail".
nss-3.60.1/nss/cmd/signtool/javascript.c:1654: pass_freed_arg: Passing freed pointer "curitem" as an argument to "PR_Free".
1652| }
1653| if (curitem) {
1654|-> PR_Free(curitem);
1655| }
1656| if (basedir) {

Error: RESOURCE_LEAK (CWE-772): [#def12]
nss-3.60.1/nss/cmd/signtool/list.c:36: alloc_fn: Storage is returned from allocation function "PORT_ZAlloc_Util".
nss-3.60.1/nss/cmd/signtool/list.c:36: var_assign: Assigning: "ugly_list" = storage returned from "PORT_ZAlloc_Util(16UL)".
nss-3.60.1/nss/cmd/signtool/list.c:137: leaked_storage: Variable "ugly_list" going out of scope leaks the storage it points to.
135|
136| if (failed) {
137|-> return -1;
138| }
139| return 0;

Error: RESOURCE_LEAK (CWE-772): [#def13]
nss-3.60.1/nss/cmd/signtool/util.c:131: alloc_fn: Storage is returned from allocation function "PR_OpenDir".
nss-3.60.1/nss/cmd/signtool/util.c:131: var_assign: Assigning: "dir" = storage returned from "PR_OpenDir(path)".
nss-3.60.1/nss/cmd/signtool/util.c:139: identity_transfer: Passing "dir" as argument 1 to function "PR_ReadDir", which returns an offset off that argument.
nss-3.60.1/nss/cmd/signtool/util.c:139: noescape: Resource "dir" is not freed or pointed-to in "PR_ReadDir".
nss-3.60.1/nss/cmd/signtool/util.c:139: var_assign: Assigning: "entry" = storage returned from "PR_ReadDir(dir, PR_SKIP_BOTH)".
nss-3.60.1/nss/cmd/signtool/util.c:142: leaked_storage: Variable "entry" going out of scope leaks the storage it points to.
nss-3.60.1/nss/cmd/signtool/util.c:142: leaked_storage: Variable "dir" going out of scope leaks the storage it points to.
140| if (snprintf(filename, sizeof(filename), "%s/%s", path, entry->name) >= sizeof(filename)) {
141| errorCount++;
142|-> return -1;
143| }
144| if (rm_dash_r(filename))

Error: RESOURCE_LEAK (CWE-772): [#def14]
nss-3.60.1/nss/cmd/signtool/util.c:131: alloc_fn: Storage is returned from allocation function "PR_OpenDir".
nss-3.60.1/nss/cmd/signtool/util.c:131: var_assign: Assigning: "dir" = storage returned from "PR_OpenDir(path)".
nss-3.60.1/nss/cmd/signtool/util.c:139: identity_transfer: Passing "dir" as argument 1 to function "PR_ReadDir", which returns an offset off that argument.
nss-3.60.1/nss/cmd/signtool/util.c:139: noescape: Resource "dir" is not freed or pointed-to in "PR_ReadDir".
nss-3.60.1/nss/cmd/signtool/util.c:139: var_assign: Assigning: "entry" = storage returned from "PR_ReadDir(dir, PR_SKIP_BOTH)".
nss-3.60.1/nss/cmd/signtool/util.c:145: leaked_storage: Variable "entry" going out of scope leaks the storage it points to.
nss-3.60.1/nss/cmd/signtool/util.c:145: leaked_storage: Variable "dir" going out of scope leaks the storage it points to.
143| }
144| if (rm_dash_r(filename))
145|-> return -1;
146| }
147|

Error: RESOURCE_LEAK (CWE-772): [#def15]
nss-3.60.1/nss/cmd/signver/pk7print.c:325: alloc_fn: Storage is returned from allocation function "PORT_ZAlloc_Util".
nss-3.60.1/nss/cmd/signver/pk7print.c:325: var_assign: Assigning: "pk" = storage returned from "PORT_ZAlloc_Util(328UL)".
nss-3.60.1/nss/cmd/signver/pk7print.c:351: leaked_storage: Variable "pk" going out of scope leaks the storage it points to.
349| default:
350| fprintf(out, "%s=bad SPKI algorithm type\n", msg);
351|-> return 0;
352| }
353|

Error: RESOURCE_LEAK (CWE-772): [#def16]
nss-3.60.1/nss/cmd/symkeyutil/symkeyutil.c:289: alloc_fn: Storage is returned from allocation function "PK11_GetSymKeyNickname".
nss-3.60.1/nss/cmd/symkeyutil/symkeyutil.c:289: var_assign: Assigning: "name" = storage returned from "PK11_GetSymKeyNickname(symKey)".
nss-3.60.1/nss/cmd/symkeyutil/symkeyutil.c:298: noescape: Resource "name ? name : " "" is not freed or pointed-to in "printf". [Note: The source code implementation of the function has been overridden by a builtin model.]
nss-3.60.1/nss/cmd/symkeyutil/symkeyutil.c:306: leaked_storage: Variable "name" going out of scope leaks the storage it points to.
304| }
305| printf("\n");
306|-> }
307|
308| SECStatus

Error: USE_AFTER_FREE (CWE-416): [#def99]
nss-3.60.1/nss/lib/libpkix/pkix_pl_nss/pki/pkix_pl_nameconstraints.c:835: freed_arg: "PORT_FreeArena_Util" frees "arena".
nss-3.60.1/nss/lib/libpkix/pkix_pl_nss/pki/pkix_pl_nameconstraints.c:854: double_free: Calling "PORT_FreeArena_Util" frees pointer "arena" which has already been freed.
852| PKIX_CERTNAMECONSTRAINTS_DEBUG
853| ("\t\tCalling PORT_FreeArena).\n");
854|-> PORT_FreeArena(arena, PR_FALSE);
855| }
856| }

Error: VARARGS (CWE-237): [#def109]
nss-3.60.1/nss/lib/libpkix/pkix_pl_nss/system/pkix_pl_string.c:428: va_init: Initializing va_list "args".
nss-3.60.1/nss/lib/libpkix/pkix_pl_nss/system/pkix_pl_string.c:534: missing_va_end: "va_end" was not called for "args".
532| }
533|
534|-> PKIX_RETURN(STRING);
535| }
536|

Error: USE_AFTER_FREE (CWE-416): [#def112]
nss-3.60.1/nss/lib/pk11wrap/pk11pars.c:1099: alias: Assigning: "newSpecPtr" = "newSpec". Now both point to the same storage.
nss-3.60.1/nss/lib/pk11wrap/pk11pars.c:1156: freed_arg: "secmod_doDescCopy" frees "newSpecPtr".
nss-3.60.1/nss/lib/pk11wrap/pk11pars.c:1211: use_after_free: Using freed pointer "newSpec".
1209| /* no target found, return the newSpec */
1210| if (target == NULL) {
1211|-> return newSpec;
1212| }
1213|

Error: CPPCHECK_WARNING (CWE-562): [#def113]
nss-3.60.1/nss/lib/smime/cmsutil.c:309: error[returnDanglingLifetime]: Returning pointer to local variable 'digestedData' that will be invalid when returning.
307| }
308| }
309|-> return cinfo;
310| }
311|

Error: CPPCHECK_WARNING (CWE-562): [#def114]
nss-3.60.1/nss/lib/smime/cmsutil.c:309: error[returnDanglingLifetime]: Returning pointer to local variable 'encryptedData' that will be invalid when returning.
307| }
308| }
309|-> return cinfo;
310| }
311|

Error: CPPCHECK_WARNING (CWE-562): [#def115]
nss-3.60.1/nss/lib/smime/cmsutil.c:309: error[returnDanglingLifetime]: Returning pointer to local variable 'envelopedData' that will be invalid when returning.
307| }
308| }
309|-> return cinfo;
310| }
311|

Error: CPPCHECK_WARNING (CWE-562): [#def116]
nss-3.60.1/nss/lib/smime/cmsutil.c:309: error[returnDanglingLifetime]: Returning pointer to local variable 'genericData' that will be invalid when returning.
307| }
308| }
309|-> return cinfo;
310| }
311|

Error: CPPCHECK_WARNING (CWE-562): [#def117]
nss-3.60.1/nss/lib/smime/cmsutil.c:309: error[returnDanglingLifetime]: Returning pointer to local variable 'signedData' that will be invalid when returning.
307| }
308| }
309|-> return cinfo;
310| }
311|

Error: USE_AFTER_FREE (CWE-416): [#def118]
nss-3.60.1/nss/lib/softoken/pkcs11.c:2671: freed_arg: "PORT_Realloc_Util" frees "oldNscSlotList".
nss-3.60.1/nss/lib/softoken/pkcs11.c:2674: use_after_free: Using freed pointer "oldNscSlotList".
2672| nscSlotListSize[index] * sizeof(CK_SLOT_ID));
2673| if (nscSlotList[index] == NULL) {
2674|-> nscSlotList[index] = oldNscSlotList;
2675| nscSlotListSize[index] = oldNscSlotListSize;
2676| return CKR_HOST_MEMORY;

Some of these are false positives, but they are reasonable issues for the scanners to flag, so scanner suppression comments, along with human reviewer comments will be added.

Differential Revision: https://phabricator.services.mozilla.com/D111339

--HG--
extra : rebase_source : 45ad7590affb7fdb8a0e182e29c6c88f909ff260

FIPS 180-3 updated the standard for clearing sensitive key material in FIPS modules. I've done a complete review of the portions of NSS affected by the FIPS requirements and identified all the areas where we need to update. The report is available here: https://docs.google.com/document/d/1v9kedUiwVYYIUagyT_vQdtrktjGUrA3SFsVP-LA6vOw/edit?usp=sharing

This patch does the following:
    - Clears the stack in gcm and ecc to deal with large stack leakages.
This only happens in FIPS enabled case. The size of the stack is based on the
size of the leakage, with some extra to make sure we reach down into that area.
Most of the leakage happens in either auto generated code or machine dependent
acceleration code.
    - Clears hash related data that wasn't cleared previously
    - Clears public key exponents that wasn't cleared previously.
    - Clears components that should have been cleared previously but wasn't.

Usually clearing takes one of the following forms:
    PORT_Free(x) -> PORT_Free(x, size). This means we need to know what
the size is supposed to be. In some cases we need to add code to preserve
the size.
    PORT_Free(x.data) -> SECITEM_ZfreeItem(&x, PR_FALSE). In this case x is
a SECITEM, which carries the length. PR_FALSE means clear and free the data in
the item, not the item itself. The code should have had SECITEM_FreeItem before
anyway.
    SECIEM_FreeItem(item, bool) -> SECITEM_ZfreeItem(item, bool). Simply change
the normal SECITEM free call to the one that clears the item.
    PR_ArenaFree(arena, PR_FALSE) -> PR_ArenaFree(arena, PR_TRUE). The bool here
means whether or not to clear as well as free the data in the arena.
    PORT_Memset(value, 0, size). This the obvious clear operation. It happens
if the variable is a stack variable, or if the memory isn't cleared with one
of the three clearing functions above.

In addition this patch fixes the following:
    - moves the determination if whether or not a slot is in FIPS mode by
slotID to a macro. NSS allows user defined slots to be opened. If you open a
user defined slot using the FIPS slot, the resulting slots will also be FIPS
slots. In areas where the semantics change based on the slot, these slots should
have the FIPS semantics. Directly checking if the slot is the FIPS slot now
only happens when we really mean the main FIPS slot and not just any FIPS slot.
    - In handling the clearing of PSS and OAEP, I identified an issue. These
functions where holding a pointer to the pMechanismParams in their C_XXXXInit
calls for later use in the C_XXXXUpdate/C_XXXXFinal/C_XXXX calls. The problem
is applications are allowed to free their pMechanismParams once C_XXXXInit is
complete. We need to make a copy of the params to use them.

Differential Revision: https://phabricator.services.mozilla.com/D108223

--HG--
extra : rebase_source : 7ac6f9d40bffbaf1a45e9e0996893afe618d4fc5

FIPS requires that we supply a hash and sign interface for our supported
signing algorithms to be validated. We already have those interfaces in
softoken for RSA PKCS1, DSA, and ECDSA. However, we don't test those
interfaces, now do we supply a way for an application to access those
interfaces (usually applications use the VFY_ and SGN_ interfaces which
handles the hashing an verify/sign operations).

We also have a generic pk11_signature_tests class in pk11_gtest, but only ecdsa
and some rsa pss tests uses it.

This patch rectifies all of these deficiencies:

lib/softokn
1) Hash and sign/verify mechanisms have been added to softoken to support PSS
hash and sign.
2) The rsa, dsa, and ecdsa hash and sign algorithms were also cleaned up by
creating a fake CKM_SHA1 which matches CKM_SHA_1 so that we can fully use the
same macros for all the hash types.

1&2 was sufficient to provide the goals of this task, however we wanted to be
able to add tests for this functionality..

lib/pk11wrap
3) Two new functions were added: PK11_CreateContextByPubKey and
PK11_CreateContextByPrivKey. These allow you to create multipart contexts with
Public and Private keys. This is a requirement to support hash and sign, as
they are multi-part operations (rather then just signing a hash, which is a
single part operation). With these functions, you can now use the PK11_DigestOp
and PK11_DigestFinal to complete a signature or verify optiation.

gtests/pk11_gtest
4) Add hash and sign/hash and verify support to the generic
pk11_signature_tests.h.
5) pk11_dsa_unittest.cc, pk11_rsa_unittest.cc, and the remainder of
pk11_rsapss_unittest.cc (Wycheproof tests) were moved to use the
pk11_signature_tests common implementation rather then their own.
6) pk11_ecdsa_unittest.cc was updated to support the hash&sign/verify combo
mechanism.
7) With multiple functions using pk11_signature_tests.h, The large functions
are moved to pk11_signature_tests.cpp.
8) The test vectors themselves were not changes, now just test against the
traditional hash first then verify interfaces and the hash and verify
interfaces.

Differential Revision: https://phabricator.services.mozilla.com/D110641

--HG--
extra : rebase_source : d2ec6b9589562cedd4aca45b79a649162eadc5ec

1) Add code to treat softokn as a non-threadsafe module.
2) Add a cycle to test ssl against non-threadafe modules.
3) Fix deadlock by restricting the ContextMonitor to only be active around PKCS #11 function calls.

Differential Revision: https://phabricator.services.mozilla.com/D112092

Differential Revision: https://phabricator.services.mozilla.com/D112226

--HG--
extra : moz-landing-system : lando

Differential Revision: https://phabricator.services.mozilla.com/D112143

--HG--
extra : moz-landing-system : lando

Currently, NSS assumes that every PowerPC target supports the crypto
and VSX extensions of the PowerPC ABI. However, VSX was only introduced
with ISA version 2.06 and the crypto extensions with ISA version 2.07
and enabling them on older PowerPC targets will result in a SIGILL. Thus,
make their use configurable and enable them by default on ppc64le only.

Differential Revision: https://phabricator.services.mozilla.com/D105354

--HG--
extra : moz-landing-system : lando

Differential Revision: https://phabricator.services.mozilla.com/D108638

--HG--
extra : moz-landing-system : lando

Bug 1698320 - replace __builtin_cpu_supports("vsx") with ppc_crypto_support() for clang. r=bbeurdouche

Differential Revision: https://phabricator.services.mozilla.com/D108354

--HG--
extra : moz-landing-system : lando

Depends on D107221

Differential Revision: https://phabricator.services.mozilla.com/D107788

--HG--
extra : moz-landing-system : lando

Depends on D107220

Differential Revision: https://phabricator.services.mozilla.com/D107221

--HG--
extra : moz-landing-system : lando

Differential Revision: https://phabricator.services.mozilla.com/D107220

--HG--
extra : moz-landing-system : lando

Depends on D106881

Differential Revision: https://phabricator.services.mozilla.com/D107787

--HG--
extra : moz-landing-system : lando

Bug 1683520 - ECCKiila P384, change syntax of nested structs initialization to prevent build isses with GCC 4.8. r=bbrumley

Depends on D102389

Differential Revision: https://phabricator.services.mozilla.com/D106881

--HG--
extra : moz-landing-system : lando

Bug 1683520 - [lib/freebl/ecl] P-384: allow zero scalars in dual scalar multiplication r=bbeurdouche

Differential Revision: https://phabricator.services.mozilla.com/D102389

--HG--
extra : moz-landing-system : lando

Bug 1683520 - ECCKiila P521, change syntax of nested structs initialization to prevent build isses with GCC 4.8. r=bbrumley

Depends on D102406

Differential Revision: https://phabricator.services.mozilla.com/D106882

--HG--
extra : moz-landing-system : lando

Bug 1683520 - [lib/freebl/ecl] P-521: allow zero scalars in dual scalar multiplication r=bbeurdouche

Differential Revision: https://phabricator.services.mozilla.com/D102406

--HG--
extra : moz-landing-system : lando

Differential Revision: https://phabricator.services.mozilla.com/D107387

--HG--
rename : lib/freebl/verified/Hacl_Curve25519_51.c => lib/freebl/verified/Hacl_Bignum25519_51.h
extra : moz-landing-system : lando

Differential Revision: https://phabricator.services.mozilla.com/D106617

--HG--
extra : moz-landing-system : lando

r=ueno,bbeurdouche

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:85.0) Gecko/20100101 Firefox/85.0

Steps to reproduce:

Using NSS with PKCS #11 library that returns CKR_ATTRIBUTE_VALUE_INVALID when searching for CKA_CLASS = CKO_PROFILE.

Actual results:

PK11_InitToken calls pk11_ReadProfileList and passes on failures. Thus, if the profiles cannot be read the token cannot be initialized.
pk11_ReadProfileList in turn uses pk11_FindObjectsByTemplate to search for CKO_PROFILE objects. This function fails if C_FindObjectsInit fails.
However, it should be perfectly ok that C_FindObjectsInit fails if CKO_PROFILE is not known. In fact, CKR_ATTRIBUTE_VALUE_INVALID is a valid return code here since the library does not know (yet) the value CKO_PROFILE for CKA_CLASS and since the CKA_CLASS is a fixed list it the standard allows to return this error code.

Expected results:

PK11_InitToken should complete successfully.

Differential Revision: https://phabricator.services.mozilla.com/D106167

Depends on D104418

Differential Revision: https://phabricator.services.mozilla.com/D106144

--HG--
extra : moz-landing-system : lando

Differential Revision: https://phabricator.services.mozilla.com/D104418

--HG--
extra : moz-landing-system : lando

Bug 1693217 - Increase nssckbi.h version number for March 2021 batch of root CA changes, CA list version 2.48. r=KathleenWilson

Differential Revision: https://phabricator.services.mozilla.com/D105451

--HG--
extra : moz-landing-system : lando

Bug 1692094 - Set email distrust after to 21-03-01 for Camerfirma's 'Chambers of Commerce' and 'Global Chambersign' roots. r=KathleenWilson

Differential Revision: https://phabricator.services.mozilla.com/D105435

--HG--
extra : moz-landing-system : lando