No, no it doesn’t. Huawei’s code might as well be extremely secure. Their code is certainly the most scrutinized. But the recent UDG source code review is not an evidence of security.
ERNW, an independent IT security service provider in Germany, recently performed a technical review / audit of Huawei’s Unified Distributed Gateway (UDG) source code. Huawei made the summary report available here [PDF].
The review focused on the quality of the source code and the source code management practices. The report is overall positive and showed that Huawei has significantly improved its software engineering processes. At least for the UDG product.
It’s a welcome improvement. Huawei deserves some recognition for improving their software engineering practices. A year ago UK government’s Huawei Cyber Security Evaluation Centre (HCSEC) issued a report that “revealed serious and systematic defects in Huawei’s software engineering and cyber security competence”.
Based on the positive report by ERNW, Huawei has mounted a PR campaign implying that the report is a proof of their 5G core network being secure and reliable.
Dozens contacted me over the last few days asking what this report really means. Here’s my $0.02.
This discussion is purely from the software engineering and cybersecurity perspectives. And based only on the publicly available ERNW summary report. I am not offering any opinions on the whole Huawei controversy here or on the 5G and COVID-19 conspiracy theory.
The ERNW’s technical review scope was the code base of UDG 20.2.0 5G.
UDG is just one of many 5G-related products within Huawei’s Cloud Core Network product line. And only one version was tested.
Huawei’s Cloud Core Network product line comprises several product groups. 5G Core product group consists of 5G Core, UNC, UDM, UPCF and UDG. Other product groups within the Cloud Core Network include CS&IMS, Mobile Packet Core, SDM, SmartPCC, Signalling, SEQ Analyst, SingleOSS-CN and others—each with several individual products. To make things more complex, there are also products within other product lines that a telco would consider if they implement 5G SA with Huawei products.
Extrapolating the positive UDG source code quality finding to security and reliability of the whole of 5G core gear is like saying that one cashier drawer in a bank has a placeholder for an advanced lock, therefore the entire bank is secure.
The source code in scope of this audit contained approximately 30 million lines of code (LOC). While LOC is a silly metric, to give you an indication – it is similar in size to the Microsoft Office for Mac code base, i.e. it’s massive.
Consider the size of the code base of just this one product and the fact that each of these dozens of products are constantly being developed and you’ll understand why I’ve been arguing for years that Huawei source code reviews as undertaken by some governments are not a feasible approach to evaluating the security of the 5G gear.
McCabe cyclomatic complexity
ERNW analyzed the source code complexity using the industry-accepted approach—by measuring McCabe’s cyclomatic complexity. It’s a quantitative measure of a number of linearly independent paths through a program and can indicate the complexity of a program. Higher value makes the program more difficult to understand and maintain.
ERNW set the threshold value for cyclomatic complexity as 100 based on “industry best practices”. For the UDG 5G components the average complexity per file was calculated at 90.284 demonstrating that on average the source code complexity is below the threshold.
The Software Engineering Institute (SEI) at Carnegie Mellon University defines four ranges for cyclomatic complexity based on an industry-accepted way of calculating the complexity. Cyclomatic complexity values below 10 represent low inherent execution and maintainability risk. Values of over 50 represent an untestable program with a very high risk.
Without understanding the non-SEI ERNW’s cyclomatic complexity metric, how they calculated the total cyclomatic complexity, and how they defined the threshold, I can’t comment on the result other than saying:
5G is a mission-critical and safety-critical system. Its cyclomatic complexity should be much closer to 0 than being barely below an “industry best-practice” threshold. (Unless ERNW took the “industry” to mean avionics, aerospace, medical, weapons sytems, nuclear, or automotive domains.)
Cyclomatic complexity is correlated with security, but only indirectly. A complex system would be harder to understand and maintain, which would make it more time consuming to find and fix vulnerabilities. It also might make it easier to introduce vulnerabilities. But that’s all.
ERNW found code duplication to be 2.2% on average. Which is a relatively good result. For an average business software, but it might still be too much for a critical software.
Some code duplication is normal. I would even argue that excessive DRYing (“Don’t Repeat Yourself”) of a code could impact readability, and therefore failing to meet the objective of DRY.
However, in this case 2.2% of the 30 million LOC means that there is 660,000 LOC with higher risk. If a fix to a vulnerability includes editing any of those 660,000 LOC, the developer would have to track down the duplicate and repeat the edit.
While code duplication, like cyclomatic complexity, degrades the maintainability of the code base it is also only indirectly correlated to security.
Code duplication also should not be reduced to a single number with no other comments. There is a significant difference in risks depending on what kind of code is being duplicated. The summary report says nothing about it.
Auditors found that Huawei is putting right processes in place to reduce the use of unsafe functions and are avoiding them extensively. They did, however, found some usage of unsafe functions that they recommended to be reduced. Without knowing more details, I can only say—the tested code uses some unsafe functions for now. Use of unsafe functions is one of the highest cybersecurity related risks in software engineering.
Auditors found that representative unit test cases were technically suitable and recommended the test coverage to be increased to 75%. Here ERNW again seems to set target thresholds based on average business systems, not critical systems.
My take: Below 75% unit testing coverage for components with high value functionality is not adequate even for an average business system, let alone for a mission-critical or a safety-critical system.
Variant analysis of the course code
Variant analysis means that ERNW used bad patterns they found in the source code and use it as a seed to look for other similar problems in the code.
From the report: “The variant analysis identified additional bad patterns.”
In the dynamic analysis phase the auditors would use a fuzzing approach to subject a running code to a variety of inputs, boundary conditions, etc. The auditors found positive test cases with either crashes or undefined states. ERNW summarized the findings as: “The results are common for projects with similar characteristics in terms of complexity and size of the code base”.
My take is that 5G core component’s source code quality shouldn’t be compared with the quality of an average software product, but with those of mission-critical and safety-critical systems.
Build process and open-source lifecyle management
The last part of the summary report looked at the build process and open-source lifecycle management. Conclusion summaries from the report:
- Secure Compilation Options: The binaries are compiled comprehensively with secure compilation options.
- Binary Equivalence: All binaries are built with binary equivalence. Overall, an acceptable amount of binary equivalence is achieved.
- Only minor improvements for the open-source lifecycle management were recommended. From the auditor’s perspective the separation of the code, the handling, documentation and patch management are reasonable and meet all requirements of a state-of-the-art open-source lifecycle management.
- The test covered only a small subset of 5G core.
- Few positive source code quality findings only indirectly correlate with security.
- The approach and the aim of the audit did not cover source code analysis from the secure code perspective nor the review of secure software engineering practices.
- This report can’t tell us conclusively anything about the UDG security, let alone about Huawei’s 5G core security.