Enhanced Testing of SONiC NOS with BGP NetOps, Dual Stack and ECMP coverage
What is FTAS (Fabric Test Automation Suite)?
Fabric Test Automation Suite (FTAS) is a virtual machine containing an extensive array of test cases. FTAS aims to validate essential features and functions of SONiC NOS for Fabric Deployment Readiness. The test cases focus on qualifying the fabric across functions, features, scale, day-2 operations, and chaos scenarios.
FTAS evolves continuously, incorporating customer feedback and integrating new features from the latest SONiC releases. In a significant stride forward, we’re thrilled to announce the unveiling of FTAS 2.2 in our ongoing journey.
Exploring FTAS 2.2 Enhancements: A Brief Overview
Fabric Test Automation Suite (FTAS) is a robust collection of test cases meticulously crafted to evaluate SONiC’s preparedness before deployment. Our continuous commitment to improvement has led to FTAS evolving based on customer insights and the integration of new features into the latest SONiC releases.
In our progressive journey, we are thrilled to announce the launch of FTAS 2.2. This version brings forth a range of enhancements aimed at further optimizing SONiC assessment and streamlining pre-deployment testing.
Let’s delve into the key enhancements introduced in FTAS 2.2:
- Improved Test Cases: FTAS 2.2 boasts refined test cases tailored to comprehensively evaluate SONiC’s functionalities, ensuring a more robust assessment process.
- Enhanced Performance Metrics: We’ve implemented enhanced performance metrics in FTAS 2.2 to provide more detailed insights into SONiC’s performance, facilitating better decision-making during deployment.
- Expanded Compatibility Testing: FTAS 2.2 extends its compatibility testing suite to encompass a wider range of network configurations, ensuring SONiC’s readiness across diverse environments.
- User-Centric Interface Updates: With a focus on user experience, FTAS 2.2 introduces interface updates, making it more intuitive and user-friendly for seamless test execution and analysis.
- Day-2 NetOps Functionality Testing Support: Enhancing tools for post-deployment network operation tests.
- Dual Stack & Graceful Restart Scalability Testing Support: Tools to test scalability for IPv4/IPv6 coexistence and network device restart.
- ECMP Scalability Testing Support: Focus on testing Equal-Cost Multi-Path routing efficiency.
- Improved Root Cause Analysis: Streamlined troubleshooting for identifying and resolving rare network issues faster.
What are the new features and improvements that FTAS offers?
Use Cases Covered
Based upon the customer deployments, the FTAS latest release is packed with new test cases to enhance the existing coverage.
Layer 3
BGP NetOps
BGP is often used for node and link drain during NetOps operations, as it provides a mechanism for the controlled removal of routes from a network. When a node or link is being drained, BGP can be used to gradually decrease the amount of traffic flowing through that node or link, while ensuring that the remaining traffic is still able to reach its destination. This is accomplished by updating BGP routing tables to reflect the new topology of the network.
The latest addition to FTAS is coverage of Day-2 NetOps operations like link drain and node drain using route maps. This coverage has been added based upon the existing customer use cases and will be expanded with future releases.
| Test Scenario | Description |
|---|---|
| BGP Netops | eBGP multi-AS config, adjacency, route convergence and data path using routed interface |
| eBGP multi-AS route convergence and data path testing using loopback | |
| Node drain/restore by applying deny/permit route-maps | |
| Link drain/restore by applying deny/permit route-maps | |
| Node drain with IPv6 traffic | |
| Link drain with IPv6 traffic | |
| Node drain using prefix-lists | |
| Link drain using prefix-lists | |
| Node drain/restore using AS path prepend | |
| Link drain using AS path prepend |
The test cases perform various operations on this topology like node drain & restore using route maps, prefix lists and AS path prepend.
Similarly we have link drain/restore operations using route maps, prefix lists and AS path prepend.
Dual Stack & BGP Graceful Restart
FTAS 2.2 also adds dual stack scalability and BGP Graceful Restart coverage. The dual stack coverage involves testing prefix,host and LPM routes to different scale limits. The BGP Graceful Restart involves testing the BGP graceful restart functionality under various failure conditions.
| Test Scenario | Description |
|---|---|
| Dual Stack | BGP dual stack scale for 4K prefix routes |
| BGP dual stack scale for 16K prefix routes | |
| BGP dual stack scale for 32K prefix routes | |
| BGP dual stack scale for 64K prefix routes | |
| BGP dual stack scale for 128K prefix routes | |
| BGP dual stack scale for 32K host routes | |
| BGP dual stack scale for 32K LPM routes | |
| BGP dual stack scale for 32K prefix routes with subnet mask | |
| Graceful Restart | BGP graceful restart with dual stack scale to 32K prefix routes |
| Enable/disable BGP graceful restart with dual stack scale to 32K prefix routes | |
| BGP graceful restart with dual stack scale to 32K prefix routes and docker restart | |
| BGP graceful restart with dual stack scale to 32K prefix routes and kill BGP process |
ECMP Scalability
In the latest FTAS release, we have also added validating the device functionality with various ECMP path and route scalability combinations. Here are various scenarios covered in FTAS 2.2:
| Test Scenario | Description |
|---|---|
| ECMP Scale | 16 ECMP paths and 32K routes |
| 32 ECMP paths and 64K routes | |
| 64 ECMP paths and 128K routes | |
| Scalability to the “device ports” ECMP paths and 1K routes per path |
Besides the feature and scalability enhancements, the latest release has also added the coverage of Dynamic Port Breakout feature. Dynamic Port Breakout in SONiC enables the administrator to change the breakout mode of the port without having to change the HWSKU/Profile of the device which requires device reboot. This FTAS release has basic coverage of Dynamic Port Breakout testing. This coverage is going to increase in future releases.
In previous FTAS releases, SONiC resilience was tested with port-channel configurations on DUT links. Thanks to valuable customer feedback, these tests can now be conducted without the need for port-channel configurations as well
Enhanced Failure Analysis
We pay close attention to the inputs from FTAS users. These users feed us data about how easy or difficult it is to operate FTAS UI and the changes they would like to see to make their experience better. FTAS 2.2 introduces significant enhancements aimed at elevating the user’s debugging experience to new levels of efficiency and effectiveness.
Recognizing the significance of tech support data in pinpointing elusive failures, our latest release introduces a mechanism to automatically collect tech support dumps for all devices whenever a test case encounters a failure. Access the tech support dump within the test report folder for the corresponding run.
Similar to tech support, collection of syslogs also helps in the root-cause analysis of the failures. Our latest release collects the syslogs for all DUTs for each test case. Like tech support dumps, the syslogs are also saved in the test report folder.
Likewise FTAS 2.2 has another useful enhancement to display DOM information to debug port issues. This data is captured in the test log whenever we encounter port issues e.g. if the port fails to activate in a test case. In some corner cases, these port issues may not be easily reproducible. Hence it is important to capture as much data as possible as and when the failure occurs during the run. The DOM data along with Syslog and Tech Support dump comes handy in narrowing down the underlying cause. This helps us release the SONiC image without corner case issues.
FAQs: Frequently Asked Questions About FTAs 2.2
How does FTAS 2.2 differ from the previous versions?
- Coverage of BGP Netops
- Increased dual stack scalability coverage
- Increased ECMP scalability coverage
- Auto collection of information to assist in debugging – techsupport bundle, syslog, DOM output
How does it help you achieve your goals and meet your expectations?
- Support for testing Day-2 NetOps functionality
- Support for testing Dual Stack & Graceful Restart scalability
- Support for testing ECMP scalability
- Easier and faster root cause analysis of the corner cases
Conclusion:
At Aviz, we’re dedicated to continually improving FTAS to empower your SONiC deployment process. Stay tuned for more updates on our journey to enhance SONiC assessment and deployment readiness.
We actively gather inputs and feedback from our customers regarding the extended coverage, incorporating this information into our approach to enhance product usability and debug capabilities
Schedule a call with our team to delve into FTAS. For comprehensive information before the scheduled call, visit our FTAS product page.
