What's NIST 800-38G, and what does it have to do with format-preserving encryption?
In April (2017) a cryptanalytic attack was discovered on the FF3 method for Format Preserving Encryption (FPE) by NIST (National Institute of Standards and Technology). As a result, FF3 was declared no longer suitable as a general-purpose FPE method. Since FF3 is utilized by organizations worldwide, cybersecurity teams are forced to look for alternative data security platforms to secure their sensitive data. The FF3 method of FPE is creating risk and potential non-compliance issues, but that has not always been the case.
In this blog, we will discuss Format-Preserving Encryption (FPE) as a whole and discuss what the NIST is doing to make the technology safe and secure again. Also, we will look at other FPE methods that NIST is no longer recognizing as cryptographically secure? Why are they facing such a high level of scrutiny now?
Format-Preserving Encryption: What is it, who uses it, and who are the vendors that provide it?
In cryptography, format-preserving encryption refers to encrypting data in such a way that the output (the ciphertext) is in the same format as the input (the plain text). Format-preserving encryption is mostly used in on-premise encryption and tokenization solutions. It is used to protect sensitive data sets such as payment card data, Social Security numbers, and country identifiers that are commonly used and stored in a number of industry databases and applications.
What is happening with the FPE methods?
The special publication NIST 800-38G addresses FPE from a technical perspective: “This recommendation specifies 3 methods for format-preserving encryption, called FF1, FF2, and FF3. Each of these methods is a mode of operation of the AES algorithm, which is used to construct a round function within the Feistel structure for encryption.”
The publication focuses on the methods used for format-preserving encryption. With FFX, FF2 did not make it to the publication, and FF3 was found to be no longer suitable as a format-preserving encryption method. Although FF1 and FF2 are still sound, there is a major concern for longevity. FF2 and FF3 derivations have been submitted for reconsideration, but the likelihood of success with these methods is very low.
Considering the work TokenEx has done in the format-preserving encryption space on our own, designing a usable method is going to be challenging. With the payment card data use case, there is simply not enough entropy to create a secure output that cannot be reverse-engineered.
Why is this concerning for the format-preserving encryption market?
The format-preserving encryption market appears to be losing stability because two of the three FFX methods used for format-preserving encryption are not considered to be cryptographically secure. Additionally, organizations that are producing format-preserving encryption solutions leveraging FF2 and FF3 methods are struggling to validate their solutions with NIST 800-38G, which will prove to be challenging. There only remains one patented method that can be used, limiting competition and increases in pricing.
Risks and Implications for Organizations
Solutions that leverage FF2 & FF3 methods for format-preserving encryption need to address the shortcomings of these methods. As noted above, the FF2 and FF3 methods are not considered “suitable” for format-preserving encryption. This presents a challenge and a perception issue for securing data in compliance with the Payment Card Industry Data Security Standard (PCI DSS).
In industries not concerned with PCI data, even though compliance with any one specific regulation might not be impacted, the overall security of the organization is more at risk. Other sensitive information or data sets outside of payment card data, like personally identifiable information (PII), nonpublic personal information (NPI), or protected health information (PHI) are equally risky to store using format-preserving encryption methods that have been invalidated, as they present the same possibility for a breach.
Next Steps for Organizations Currently Using Format-Preserving Technologies
If you are using format-preserving encryption to secure data, identify which methods are being used for the solution. Question these vendors about what is being done on their part to remedy the use of insecure format-preserving encryption methods to ensure your organization is neither noncompliant nor penalized financially. If your vendor is simply waiting in the ranks for NIST to validate the methods that have been re-submitted without additional plans to ensure the solution they’ve sold your organization, it’s time to start looking elsewhere. There should be an active or pending product update for ensuring the FPE-based solution will leverage a fully functioning, suitable, and adequate format-preserving encryption method for securing data within your environment.
TokenEx's Data Protection Platform provides a welcome solution to the format-preserving encryption problem. Tokenization is the process of turning sensitive data into nonsensitive data called "tokens" that can be used in a database or internal system without bringing it into scope.
Tokenization can be used to secure sensitive data by replacing the original data with an unrelated value of the same length and format. The tokens are then sent to an organization’s internal systems for use, and the original data is stored in a secure token vault.
Unlike data protected with FPE, tokenized data is undecipherable and irreversible. This distinction is particularly important: Because there is no mathematical relationship between the token and its original number, tokens cannot be returned to their original form.