Since the release of much awaited 3GPP Release-16 in June last year lot of vendors have proliferated their products and brought their 5G SA A.K.A Standalone products to market and with promises like support of Slicing , massive IoT , uRLLC and improved , Edge capability ,NPN and IAB backhauling it is just natural all big Telco’s in APAC and globally have already started their journey towards 5G Standalone core . However, most of the commercial deployments are based on vendor E2E stack which is a good way to start journey and offer services quickly however with the type of services and versatility of solution specially on the industry verticals required and expected from both 3GPP Release16 and SA Core it is just a matter of time when one vendor cannot fulfill all the solutions and that is when a dire need to build a Telco grade Cloud platform will become a necessity.
During the last two years we have done a lot of work and progress in both better understanding of what will be the Cloud platforms for 5G era , it is correct that as of now the 5G Core container platform from open cloud perspective is not fully ready but we are also not too far from making it happen . From community Anuket Kali that we are targeting in June is expecting to fulfill many gaps and our release cycle for XGVELA will try to close many gaps , so in a nutshell 2021 is the year where we expect a Production ready open cloud platforms avoiding all sorts of vendor lock ins .
Let’s try to understand top issues enlisted based on 5G SA deployments in Core and Edge Vendors are mostly leveraging existing NFVI to evolve to CaaS by using a middle layer shown Caas on Iaas , the biggest challenge is this interface is not open which means there are many out of box enhancements done by each vendor and this is one classic case of “When open became the closed “
The most enhancement done on the adaptors for container images are as follows
- Provides container orchestration, deployment, and scheduling capabilities.
- Provides container Telco enhancement capabilities: Hugepage memory, shared memory, DPDK, CPU core binding, and isolation
- Supports container network capabilities, SR-IOV+DPDK, and multiple network planes.
- Supports the IP SAN storage capability of the VM container.
- Migration path from Caas on IaaS towards BMCaaS is not smooth and it will involve complete service deployment, it is true with most operators investing heavily in last few years to productionize the NFVi no body is really considering to empty pockets again to build purely CaaS new and stand-alone platform however smooth migration must be considered
- We are still in early phase of 5G SA core and eMBB is only use case so still we have not tested the scaling of 5G Core with NFVi based platforms
- ETSI Specs for CISM are not as mature as expected and again there are lot of out of box customizations done by each vendor VNFM to cater this.
Now lets come to point where the open platforms are lacking and how intend to fix it
Experience #1: 5G Outgoing traffic from PoD
The traditional Kubernetes and CaaS Platforms today handles and scales well with ingress controller however 5G PoD’s and containers outgoing traffic is not well addressed as both N-S and E-W traffic follows same path and it becomes an issue of scaling finally.
We know some vendors like Ericsson who already bring products like ECFE and LB in their architecture to address these requirements.
Experience#2: Support for non-IP protocols
PoD is natively coming with IP and all external communication to be done by Cluster IP’s it means architecture is not designed for non-IP protocols like VLAN, L2TP, VLAN trunking
Experience#3: High performance workloads
Today all high data throughputs are supported CNI plugin’s which natively are like SR-IOV means totally passthrough, an Operator framework to enhance real time processing is required something we have done with DPDK in the open stack world
Experience#4: Integration of 5G SBI interfaces
The newly defined SBI interfaces became more like API compared to horizontal call flows, however today all http2/API integration is based on “Primary interfaces” .
It becomes a clear issue as secondary interfaces for inter functional module is not supported
Experience#5: Multihoming for SCTP and SI is not supported
For hybrid node connectivity at least towards egress and external networks still require a SCTP link and/or SIP endpoints which is not well supported
Experience#6: Secondary interfaces for CNF’s
Secondary interfaces raise concerns for both inter-operability, monitoring and O&M, secondary interfaces is very important concept in K8S and 5G CNF’s as it is needed during
- For all Telecom protocols e.g BGP
- Support for Operator frameworks (CRD’s)
- Performance scenarios like CNI’s for SR-IOV
today only viable solution is by NSM i.e service mesh that solves both management and monitoring issues
Experience#7: Platform Networking Issues in 5G
Today in commercial networks for internal networking most products are using Multus+VLAN while for internal based on Multus+VxLAN it requires separate planning for both underlay and overlay and that becomes an issue for large scale 5G SA Core Network
Similarly, top requirements for service in 5G Networks are
- Network separation on each logical interface e.g VRF and each physical sub interface
- Outgoing traffic from PoD
- NAT and reverse proxy
Experience#8: Service Networking Issues in 5G
For primary networks we are relying on Calico +IPIP while for secondary network we are relying ion Multus
Experience#9: ETSI specs specially for BM CaaS
Still I believe the ETSI specs for CNF’s are lacking compared to others like 3GPP and that is enough to make a open solution move to a closed through adaptors and plugin’s something we already experienced during SDN introduction in the cloud networks today a rigorous updates are expected on
- IFA038 which is container integration in MANO
- IFA011 which is VNFD with container support
- Sol-3 specs updated for the CIR (Container image registry) support
Experience#10: Duplication of features on NEF/NRM and Cloud platforms
In the 5G new API ecosystem operators look at their network as a platform opening it to application developers. API exposure is fundamental to 5G as it is built into the architecture natively where applications can talk back to the network, command the network to provide better experience in applications however the NEF and similarly NRF service registry are also functions available on platforms. Today it looks a way is required to share responsibility for such integrations to avoid duplicates
Reference Architectures for the Standard 5G Platform and Capabilities
Cap#1: Solving Data Integration issues
Real AI is the next most important thing for Telco’s as they evolve in their automation journey from conditional #automation to partial autonomy . However to make any fully functional use case will require first to solve #Data integration architecture as any real product to be successful with #AI in Telco will require to use Graph Databases and Process mining and both of it will based on assumption that all and valid data is there .
Cap#2: AI profiles for processing in Cloud Infra Hardware profiles
With 5G networks relying more on robust mechanisms to ingest and use data of AI , it is very important to agree on hardware profiles that are powerful enough to deliver AI use cases to deliver complete AI pipe lines all the way from flash base to tensor flow along with analytics .
Cap#3: OSS evolution that support data integration pipeline
To evolve to future ENI architecture for use of AI in Telco and ZSM architecture for the closed loop to be based on standard data integration pipeline like proposed in ENI-0017 (Data Integration mechanisms)
Cap#4: Network characteristics
A mature way to handle outgoing traffic and LB need to be included in Telco PaaS
Cap#5: Telco PaaS
Based on experience with NFV it is clear that IaaS is not the Telco service delivery model and hence use cases like NFVPaaS has been in consideration for the early time of NFV . With CNF introduction that will require a more robust release times it is imperative and not optional to build a stable Telco PaaS that meet Telco requirements. As of today, the direction is to divide platform between general PaaS that will be part of standard cloud platform over release iterations while for specific requirements will be part of Telco PaaS.
The beauty of this architecture is no ensure the multi-vendor component selection between them. The key characteristics to be addressed are
Paas#6: Telco PaaS Tools
The agreement on PaaS tools over the complete LCM , there is currently a survey running in the community to agree on this and this is an ongoing study
Paas#2: Telco PaaS Lawful interception
During recent integrations for NFV and CNF we still rely on Application layer LI characteristics as defined by ETSI and with open cloud layer ensuring the necessary LI requirements are available it is important that PaaS include this part through API’s
Paas#3: Telco PaaS Charging Characteristics
The resource consumption and reporting of real time resources is very important as with 5G and Edge we will evolve towards the Hybrid cloud
Paas#4: Telco PaaS Topology management and service discovery
A single API end point to expose both the topology and services towards Application is the key requirement of Telco PaaS
Paas#5: Telco PaaS Security Hardening
With 5G and critical services security hardening has become more and more important, use of tools like Falco and Service mesh is important in this platform
Paas#6: Telco PaaS Tracing and Logging
Although monitoring is quite mature in Kubernetes and its Distros the tracing and logging is still need to be addressed. Today with tools like Jaeger and Kafka /EFK needs to be include in the Telco PaaS
Paas#7: Telco PaaS E2E DevOps
For IT workloads already the DevOps capability is provided by PaaS in a mature manner through both cloud and application tools but with enhancements required by Telco workloads it is important the end-to-end capability of DevOps is ensured. Today tools like Argo need to be considered and it need to be integrated with both the general PaaS and Telco PaaS
Standard packages like VNFD which cover both Application and PaaS layer
Paas#8: Standardization of API’s
API standardization in ETSI fashion is the key requirement of NFV and Telco journey and it needs to be ensured in PaaS layer as well. For Telco PaaS it should cover VES , TMForum,3GPP , ETSI MANO etc . Community has made following workings to standardize this
- TMF 641/640
- 3GPP TS28.532 /531/ 541
- IFA029 containers in NFV
- ETSI FEAT17 which is Telco DevOps
- ETSI TST10 /13 for API testing and verification
Based on these features there is an ongoing effort with in the LFN XGVELA community and I hope more and more users, partners and vendors can join to define the Future Open 5G Platform