5 reasons why SONiC must have neutral testing for prime time

 

Since its initial release in 2017, SONiC (Software for Open Networking in the Cloud) has emerged as a leading NOS (network operating system). The open source Linux-based NOS offers robust functionality — including support for BGP and RMDA — and has proven itself in production at several major cloud-service providers’ datacenters. Clearly, the potential of SONiC to abstract away the hardware layer and enable flexibility, extensibility, and standardization is huge.

 

However, there are several challenges that must be addressed before SONiC can cross the chasm and become the norm on enterprise networks. One of those challenges is the need for neutral, open, and standardized testing. Why? Because end-users need an authoritative source of truth for what they can expect when they deploy hardware running SONiC. While most hardware vendors today “support” SONiC, it’s not clear or easy to determine what features are and aren’t supported. Because of this ambiguity, using SONiC isn’t feasible for many IT project managers.

 

Of course, SONiC is tested extensively by contributors and members of the community today. So why isn’t the current testing paradigm sufficient? Here, we’ll answer that question and expand on why neutral SONiC testing to complement existing tests is a must-have before the “prime time” of mass-market adoption can occur.

Why SONiC Needs Neutral Testing Reason 1: Expand Depth and width of coverage

Today, much of the formal SONiC testing is performed using PTF (Packet Testing Framework) and  SPYTEST. While these tools offer contributors and experienced community members to run tests and validate a number of use cases, there are several issues with the current testing paradigm. Specifically:

 

  • Current PTF testing delivers wide, but not deep testing
  • Current SPYTEST tests go deep, but not wide
  • Testing options are abstracted away from the general population of network deployments that are vital to the growth of the SONiC platform


As more and more contributors enter the SONiC ecosystem, things will only get noisier and harder for enterprises to navigate. Case in point: suppose an enterprise needs an L3 switch that supports CoS (Class of Service) IEEE 802.1p, Layer 3 ACLs (Access Control Lists), VRF (Virtual Routing and Forwarding) Lite, and BGP with both IPv4 and IPv6 support. Today, mapping it’s easy enough to determine SONiC supports all those features, but confirming SONiC running on a given switch or ASIC has been validated can be difficult.

 A standardized vendor-neutral approach to SONiC testing — for example using SONiC TaaS (Testing as a Service) to validate ASICs and switches — can provide end-users, R&D teams, and corporate IT ops departments specific, reliable, and actionable test data they need to move forward with a SONiC-based deployment.

Why SONiC Needs Neutral Testing Reason 2: Support isn’t a simple YES or NO

“Does product X support SONiC?” isn’t a simple yes or no question. Validating support requires accounting for:

  • SAI (Switch Abstraction Interface) version tested
  • SONiC version tested
  • Type of ASIC tested
  • Type of switch tested

 

Today, the list of supported platforms on GitHub is useful but doesn’t go far enough. By adhering to a set of standards for running and recording test results, neutral testing can provide clear and authoritative answers to the “SONiC support” question.

Why SONiC Needs Neutral Testing Reason 3: Trust

There are many contributors doing testing in the SONiC community. Often, these contributors are vendors of ASICs, switches, network operating systems, or orchestration tools. While contributors have done and continue to do great work for the community, commercial vendors have a particular set of interests.

 

A testing body without ties to a specific vendor can do for SONiC what organizations like UL (Underwriter’s Laboratories) and ISO (International Standards Organization) have done for other hardware and software categories: provide a trusted source of information on the standards a given product meets.

Why SONiC Needs Neutral Testing Reason 4: Transparency

As you might expect, a great way to increase trust is to increase transparency, particularly in the open source world. However, an interesting paradox in the SONiC community today is: the source code is open, but the test results aren’t.

 

For SONiC to achieve the ambitious goal of making open networking the norm, that needs to change. By testing platforms to the same set of standards and sharing the results with the community, neutral testing can add a level of transparency

Why SONiC Needs Neutral Testing Reason 5: Standard and reliable test reporting is a must

Looking back at the list of supported SONiC platforms, there is no easy way to get data on exactly what was tested, by whom, or the extent of the tests run. Tests for system stress and day-2 operations are vital to understanding how SONiC switches will perform in the field.

 

A vendor-neutral testing solution can provide detailed test reports based on standardized tests. As a result, users interested in the SONiC platform will have access to actionable data that allows them to pick the right solution and further drive SONiC’s growth.

Next Steps: Working towards a Neutral Testing Solution for SONiC

Clearly, vendor-neutral testing is needed in addition to the testing SONiC community members are running today. To help SONiC address this challenge and enable more end-users to view open networking as a production-ready solution, we’ve partnered with Keysight to create the industry’s first independent SONiC verification lab.  From state-of-the-art Keysight  Labs in four different locations, we run comprehensive vendor-neutral to validate switches and ASICs running SONiC. To learn more about our SONiC Verification Lab (SVL), download the SVL brief here or at our partner Keysight or contact us today for any SONiC questions!!