Key Industry Challenges and devising new models for Business enablement using End to End Network Slicing

Network Slice is a concept within 3GPP CT and its history can be traced back to R13/R14 with the introduction of static Slicing in LTE networks. It is being said that the real business case of Network Slicing will come with the arrival of 5G, although in Rel15 Stage3 release in Dec, 2017 still the complete definition with use case mapping  is missing but it is being said R16 Stage3 coming in Q2 2018 it will be available. 5G network must address the Network Slicing from dynamic slicing point of view which can be provisioned, managed and optimized through Orchestrator in real time for different use cases like Massive IoT, Ultra Reliable and enhanced MBB.
But what is the consistent definition of Network Slice? Because still I find the term means different to different teams and hence in this paper I will quest for a deep dive to come to a common definition and how it can be implemented in a consistent manner. In addition I want to answer one common question should we delay slicing now and wait for 5G Rel16 Stage 3 or we can start with enablement of simple slicing scnerio that can be upgradable to 5G as we move it.
Frankly speaking Slicing is not something only needed in 5G the prior networks do need and somehow support them due to one very reason which is business case like 4G EPC deployed in Oil industry, Corporate banks connectivity  etc. However in the 4G era there are some limitations like only support FUP RAN channels or not-shared RAN channels can be used. In a nutshell it means only a static slice can be provisioned mainly define well ahead of time mostly based on APN /IMSI. Since 5G is mostly about verticals and 3GPP CT3 is doing a fantastic job to define details of how a dynamic network slicing will be about. Please refer to 3GPP TR 28.801 just widening up the Options for runtime slice instance selection. IN 5G in addition to DNN (the 5G equivalent of an APN) we can use IMSI + NSSAI comprising up to 8 S-NSSAI for mapping the access session to a slice instance, also MANO will be evolved to support NSSF-Manager for handling all SLA/KPI and management part.
Hence we will try to analyze evolution of network slicing as we evolve future and software defined networks. The detailed understanding of concept requires understanding the whole concept involving the following dimensions.
1. Understanding Slice requirements from TR22.891
  • The operator shall be able to create and manage network slices that fulfil required criteria for different market scenarios.
  • The operator shall be able to operate different network slices in parallel with isolation that e.g. prevents data communication in one slice to negatively impact services in other slices.
  • The 3GPP System shall have the capability to conform to service-specific security assurance requirements in a single network slice, rather than the whole network.
  • The 3GPP System shall have the capability to provide a level of isolation between network slices which confines a potential cyber-attack to a single network slice.
  • The operator shall be able to authorize third parties to create, manage a network slice configuration (e.g. scale slices) via suitable APIs, within the limits set by the network operator.
  • The 3GPP system shall support elasticity of network slice in term of capacity with no impact on the services of this slice or other slices.
  • The 3GPP system shall be able to change the slices with minimal impact on the ongoing subscriber’s services served by other slices, i.e. new network slice addition, removal of existing network slice, or update of network slice functions or configuration.
  • The 3GPP System shall be able to support E2E (e.g. RAN, CN) resource management for a network slice.
Figure  1 Understanding Slice Requirements in NFV/SDN enabled Future Networks
It seems clear even if the Network slicing standard is not locked the existing NFV/SDN architecture can be used to enable it at least as over lay to provide resource isolation as Tenant level.
2. Understand Network Slicing from NFV/SDN Point of view
NFV/SDN is to enable agile delivery of service using multi-tenant provisioning in Common NFVI, one basic Slicing concept can be understood by provision multi VNF for different use cases like for Massive IoT we can have a light weight C-SGN combining both Control and User Plane while for uRLLC it can mean a distributed VNF’s using CUPS architecture to deliver real experience to the Edge. Even for eMBB it can mean end service can be delivered by avoiding vFW or other NE’s in order to build a big pipe to deliver video and live broadcast use cases. Some new additional function in Rel15/Rel16 can be used to deliver slice from Telco point of view like décor is considered to be an early enabler for the Slicing but obviously it do not come with end to end provision and monitor/management functions which can only be promised in 5G
Figure  2 Understanding Slicing context in NFV/SDN
The Above figure explains a good Reference to understand Network Slicing, for those familiar with NFV/SDN the key change is the addition of Network Slice layer. This layer may be part of NSD /VNFFGH or exist as an independent layer. Since one Service can comprise of multiple slices as needed so this layer is required even if 1:1 relation exist, it is abstraction layer to offer flexibility.
Services ⇒ It is a business service something which will be offered to end customer
Network Slice instance-NSI ⇒ a collection of resources from below layer s that defines a slice
Sub Network instance NSSI ⇒ consider it as a group of related VNF/PNF
Both the NSI and NSSI is delivered as part of NSD. Hence a Network Slice Instance (NSI) may be composed by none, one or more Network Slice Subnet Instance (NSSI), which may be shared by another NSI. Similarly, the NSSI is formed of a set of Network Functions, which can be either VNFs or PNFs.
Figure  3 Network Slice components
3. Network Slice enablement from 5G 
The real enablement of Dynamic Network slicing will come with 5G to ensure dedicated services to each customer for their customized use case or segment. The complete advantage of network slicing can only be achieved when whole Telco Network is virtualized including RAN and possibly RF modules also because the UE will request a slice based on specific ID that needs to be identified by RF, RAN to ensure it can be delivered end to end otherwise the whole concept of network slice is like a static overlay based on VPN/VRF as is implemented today. This is the main reason why all Tier1 CSP’s want to accelerate NFV/SDN use cases before 5G to assure the promise of Network slicing can delivered end to end. 5G CT RAN3 also investigating the implications of Network slice as 5G NR will open access to all access including 5G Wi-Fi, it is not well known how in such cases Network slice will be delivered specially with convergence with Wifi Networks but it is agreed in the planetary meeting that this function will be available for slicing. In 5G Networks the UE will keep track of all slices associated with it by connecting to unique AMF and hence one UE can be associated with 8 Unique Slice offering which is more than enough as per business requirements.
From RAN perspective idea of Network slicing involves that the Slice ID can be linked or transferred to correct NF in the Core Network. In the initial phase this slicing can be static but over long term this scnerio from UE till the DN must be automatic and dynamic using NFVO closed loop control methodology.
4. End to End Slice Management in 5G 
As we are well aware the key characteristics of Network slicing are two which are to sell common platform to end users and tenants using Naas <Network as a service> and to monitor /manage it. From end user perspective the Slice ID consists of Slice Type and Slice Differentiator to explain further use case with in a slice type <eMBB, IoT, uRLLC>. For some unique enterprise use cases it is also possible to provision unique nonstandard slice values. These values will be recognized by Radio network in 5G and UE will carry it during service use to assure the dedicated resources are delivered end to end. In order to manage the slice end to end each Telco VNF starting from RAN must have a new logical entity names NSSF (Network Slice selection function) to assure mapping of request to correct slice is done correctly and that consistency is delivered end to end. Below you will find how the Network view of Slice will come in NFV. As we can see the Os-Ma Sol5 is the key to integrate Slice management functions in the NFV.
Figure 4 E2E Slice management in 5G
From a resource management viewpoint, NSI can be mapped to an instance of a simple or composite NS or to concatenation of such NS instances. From a resource management viewpoint, different NSIs can use instances of the same type of NS (i.e. they are instantiated from the same NSD) with the same or different deployment flavors. 3GPP SA5 also considering exposure of Slice management to third party using Rest API which means an operator can become virtual and that slices can be provisioned, managed and optimized by 3rd party. This model is key to enable industry verticals in this domain.
In addition to that Transport Network being a multi service network must support slicing between 5G and non 5G Network services. Since all the services are placed in the VN so it is mandatory to isolate the traffic between different VN’s. Further  it is expected that the Slice manager will request the configuration of Managed Network Slice Subnet Instances (MNSSIs) to support the different 5G services (e.g., uRLLC, eMBB, etc.). A MNSSI will be supported by a VN. In the fronthaul network only one MNSSI is required since all services are carried between the RRU and DU in a common eCPRI encapsulation.
Figure 5 Slice Diversity in 5G
5. What is the Optimum way for NW slicing 
There are many SDO’s recently working on the NW slicing like NGMN, ETSI, NFV, 3GPP, etc. obviously because there is a lot of traction and appeal together to create a new business case to sell new products and solutions to the verticals and industry alike. As industry has witnessed In both pre and post era of Y2K of the industry and business shift from voice/SMS to MBB/Internet Era where CSP’s work as Pipes only the new 5G direction sets the new umbrella about the definition of broadband to include all verticals. Obviously to sell such dream the NW slicing need to cater dedicated slice and its E2E management and fulfillment by the tenant itself. This is the true model of SaaS something Facebook, Dropbox, Google have been doing so successfully in the past.
Figure 6 Network Slicing Logical view
However this is just the one sided view of picture the business case of 5G combined with business model requirements make it a worth of dime to consider possible definitions of a new standard in the ETSI NFV architecture itself.
Figure 7 Network Slicing Model in Hybrid Networks
However, the PNF would be managed outside of MANO, as well as sub-network parts composed of connection of PNF. This would require the Network Slice Life Cycle Management to also interface with a non-virtualized life cycle management & operation environment, as shown on Figure 4, with an open Nsl-PN interface, with PN standing for “Physical Network.”
6. Key Findings from ETSI GR NFV-EVE 012 standard on Network Slicing
Each tenant manages the slices that are operative in its administrative domain by means of its NFVO, logically placed in the tenant domain. Tenants rely on their NFVOs to perform resource scheduling functions in the tenant domain. As these resources may be provided by different Infrastructure Providers, the NFVO should need to orchestrate resources. Across different administrative domains in the infrastructure. Slicing requires the partitioning and assignment of a set of resources that can be used in an isolated, disjunctive or shared manner. A set of such dedicated resources can be called a slice instance. Defining a new network slice is primarily configuring a new set of policies, access control, monitoring/SLA rules, usage/charging consolidation rules and maybe new management/orchestration entity, when network is deployed with a given set of resources. In addition the ability to differentiate network slices through their availability and reliability and the ability for the network operator to define a priority for a network slice in case of scarce resource situations (e.g. disaster recovery) requires a strong coordination between NFV and SDN domain for slice management
In Point#2 I have already explained what it means to deploy a slice in Pure NFV environment so let us enlist Network slice as it apply to SDN below
A network slice in NFV context may belong to multiple Sites a compelling idea especially when VNF are split across different NFVi POPs
Within the context of NFV/SDN the Security is of prime importance for Slice management reason being that the Slice requires a view like a graph involving interconnecting VNF’s. Ideally it means for each slice to have a separate VNF and possibly PNF which is not optimal way hence it is assumed slice management and NFVO capable to ensure separation of data flows for each slice. In order to meet this requirement the VNF redesign may be necessary to incorporate the HMEE <Hardware mediated Execution Enclave> as discussed in ETSI NFV SEC 009.
Figure 8 What are the Slice End Points
A part from above the NSM <Network slice management> is very important as Slice involve NFV, SDN, Cloud and possibly PNF’s also and ability of Os-Ma SOL5 as well as the NFVO to manage hybrid environment is key.
In the nutshell Network slicing should enable proper network control and logical separation in terms of dedicated resources, operations isolation, feature isolation, reserved radio resource, separate policy control, SLA control, security and service reliability control. This is a unique differentiator of NFV/SDN/5G networks and opens new possibility for operators.
In this paper Author tried to explain new possibilities and evolution path for network slicing, it is well known before 5G SA complete standardization the Slicing will not be available end to end at least the dynamic slicing and also the framework of NSM is not locked however there are certain features and functions of slicing that can be exploited using NFV/SDN networks in today’s networks. It is very clear Operator need to start now for Slicing trying to implement use cases of static slicing using NFV/SDN and MANO functions as available today and that to upgrade it as the 5G becomes main stream. I hope audience have like this content and will help them details understand the details about important concept and that it will allow architects to best plan their 2020 Networks.
References
  • 3GPP TR22.891
  • 3GPP TS23.501
  • 3GPP TR 28.801
  • 5G_Americas_Network_Slicing_11.21_Final
  • NGMN 160113 Network Slicing v1 0
  • NFV-EVE 012v3.1.1 – GR – Network Slicing report
  • GSTR –TN5G Transport Network support  of IMT 2020/5G

Decoding the Symbols and Anecdotes of Open Innovation and Transparent Collaboration ~ Platform perspective

RedHat Summit is one of the technologist’s community events which over the years acclaimed reputation of open innovation and transparent knowledge sharing .Therefore it is natural to see techies from around the globe coming together with common vision of sharing , learning , growing together .One can consider it a one stop shop for all industries that leverage power of software to grow, from Lockheed martin F35 design, Exon Mobile deep-sea exploration, DT IT Digital platforms, Delta airline journey to open platforms and innovation, Banking and financial systems digitization or Telco’s pursuing to evolve through power of openness and innovation. Each of story has a lesson in its own and industry shared joint responsibility to obviate the issues of vendor lock in and the customers success stories from different industries support for benefit of all as a collective goal.

However, with so much influx of information and hearing perspectives from different markets and different mindsets it is probable to get diluted between a marketing theme and what is actually possible. It is therefore important to decode symbols and anecdotes as I grabbed them in this year’s RedHat summit held in Boston, U.S from May 7 to May 9  2019 . https://www.redhat.com/en/summit/2019 as follows .

 

Promote the Spirit of Open source is through collaboration and belief

Open source by its spirit stands on the ground that sum of collective effort will outweigh the sum of individual efforts. Infact Open source is nothing more than collaboration and belief of sharing that should benefit all.This collaboration opens the new channels and opportunities in each industry we operate by reaching a new possibility

What I liked most was a symbolic depiction from Apollo U.S Program which only made possible after we set high possibilities something which obviously was not possible with technology possibility of that day

No alt text provided for this image

 

Making money through transparent innovation

During last few years we have seen so many companies joining the band wagon of open source but still the industry has not able to benefit in full way from all of the promised achievements . I think this issue happened because still we need to collectively enforce idea that if you get you have to give and that embracing the fact that it’s all about dependency not independency which makes ecosystem as the ultimate winner of this transformation

Open source is not only about products but also partnerships

Bringing People and technology together

All benefits from points highlighted above can be achieved only if we have Platform and tools that will enable industries and teams to come together to explore opportunities together. Hence the platform must have following characteristics.

  • Accepted unanimously across all industries
  • Supports collaboration using open and shared principles between different teams
  • Open enough to steer new innovation
  • Free enough to experiment and grow, After all this is so important for dollarization of technology

Platforms and Tools are innate to materialize Digital Transformation

Team empowerment

Despite of all lucrative promises why many industries (Specifically Telco’s) have not really able to reap full benefits i think is because to realize truly immersive targets we must empower teams with tools and trust them. This is the only way to take us forward from an industry dominated by a few vendors to the open dominated by best ideas and best solutions.

Culture and Trust is useless with out capability and vice versa

No alt text provided for this image

Smooth Operations

Being innovative and reaching for the moon is one thing and running smooth operations is something else. Specially troubleshooting of opensource is difficult than one can even imagine. This is an area that require thoughtful analysis and support.

Alignment of operations will decide how quickly we can get rid of old infrastructure

Transfer technology equity to less innovative industries

This question is being asked to me by many teams, in summits, workshops, interviews every where. We all know specifically for Telco industry only the T1 CSP’s have pockets and muscle to rise to the real challenge. As industry we owe a collective responsibility to transfer the equity of what we learnt in the form of clear reference architectures and tools that are programmable, customizable and less costly.

To put in a perspective Edge and distributed cloud is a big opportunity that require public web scale cloud companies like AWS , Google , Azure and others to pass Innovation to less innovative industry which needs tooling for clear use cases like AI, ML, Deep learning , AR and VR etc to mention a few .

As industry we owe collective responsibility to build something that can be consumed by less innovative industries and clients

Annex:

Client Success Stories IBM, Delta, exon mobile, Lockheed martin

RedHat summit 2019

Ideas Exchange -The Cube

Report this

Published by

Saad Sheikh  ☁

A “Dev-Net-Ops” Framework for Digital Telcos (DSP’s) ~ Telco Application Playbooks

1

“Dev-Net-Ops” Framework for Digital Telco’s (DSP’s)

2

From vendor defined tools to Telco Applications CI/CD Playbooks

As the Business adoption to DevOps accelerates at Stupendous pace . In a recent working with Gartner we conclude that by 2022 expected 75% of such project will fail to deliver business value as we put DevOps idea to practice. It means for Comm-SPs really we need to put some realistic framework on ground to ensure we adopt DevOps in steps using realistic use cases that deliver real value to business. This paper is all about this how author thinks the industry should address this problem. This white paper describes key practices and how they can benefit the telco world, which has conventionally relied on separate development and operations to meet its stringent service quality requirements in a heavily regulated market.

I think once the Telco applications will be based on Cloud Native and micros services it will be easy to use many of repository and tools from IT but what about situation today. From where we shall start with technology as heart of transformation what is ground step-0 , I think it still lies on vision and approach to not go after fancy marketing stuff in a frenzy and define a clear use case , Lets say to introduce DevOps in Telco can we select a simple open source Application and cloud it with a tool chain to manage it through a Digital market place from inception to delivery across the life cycles .

 

For Comm-SPs what will be the right definition of DevOps for Network or Dev-Net-Ops , the answers can be many as most Telco’s want to introduce a pure IT concept to Telco’s , however the fact is that the Development pipe line of Telco’s today is totally controlled by Vendors not visible on Community Hubs or commercial ready repositories .So to introduce Dev-Net-Ops in Network Telco’s should prepare a customized framework for DevOps can help them automate the Networks , enable agile service delivery and most importantly is open enough to introduce nimble players in the Networks .

Then there is a commercial aspect of the same, when I started to build my career in IT a decade ago always we are trying to build a solution where product /licenses can be procured from vendors while the service is managed in house. However Telco’s operate in a more vendor dominated mode since decades and it is due to very reason the Service cost is eating huge part of their CAPEX. A systematic approach in to DevOps and scoping it in right direction will help solve the issues

Current NFV/SDN pain issues related to DevOps use cases

Starting NFV/SDN with high expectations with Multi-vendor solutions, half a decade on the difficult road and still find difficult to operationalize the network. The list of Comm-SP pain points for transformation can be huge and adding Operational complexity without automation and analytics using DevOps can make this issue further complex, however for the purpose of this paper I will like to demarcate framework list can be big but can be summarized along following three points.

3

(1) End to End system integration is still not a pure decoupled offering from vendors, which ultimately means in a Multi-vendor project an Operator thinks Service will be led by integrator but ultimately, they find them in the mid-way requiring service procured from all players and dealing with them all the way. We believe a focus on Test environment and Validation defined by Net-Dev-Ops can help solve both technical and commercial issues.

(2) VNF On boarding is still a silo scenario , normally for 1st time VNF certification and onboarding we can digest more cost/time but subsequent VNF’s of such type/scale must be a clone and should be accessible at Zero service cost , after this is what we want to achieve a Network offering as a code and programmable network . a Dev-Net-Ops can enable us for this.

(3) TaaS or Testing service need to be based on open source tools and automated in CI/CD, in other words Telco should be able to customize tool chain and use variety of tools from different vendors. A correct Dev-Net-Ops will help us solve this.

(4) On Top of everything DevOps with relying to breaking the code and aligned with automation and frequent builds seems contradicting with Telco world strict requirement for highly reliable and performance accelerated service.

From Automation to DevOps Pipe Line

We all agree the Telco Networks are still not automated unlike their IT and Cloud Counterparts and in fact require lot of manual tweaking whether it is for integration, deployment, testing and worst all for troubleshooting. So, one might be asking if Automation can fix those issues?

“Twitter user @ tweeted this perspective on code reviews: “10 lines of code = 10 issues. 500 lines of code = looks fine.” There simply are not enough people with enough time for manual processes to adequately oversee application delivery”

4

*Courtesy of Ramyram GitHub

The answer to such type of question is not definitive, because Telco industry Development pipe is still controlled and tested by Vendors. So for us as a Telco it makes a lot of sense if we address those issues of Automation that help us to use set of tools that enable agile integration, fast testing and nevertheless help solve issues in an automated manner. So what should be the characteristics of such a Tool Chain?

  • Simple to use
  • Simple to integrate with both Cloud and Legacy Network
  • Support a programmable interfaces that are Secure
  • Support native coherence with IT automation tools JIRA /JFROG/JENKINS/GITHUB
  • Put Orchestration as first way to achieve automation not focus much on Below layers

Why I think Automation will not solve issues for Telco Cloud Journey is due to two main reasons once is because most application are still Fat and secondly too much reliance on vendors specially on Dev part simply means complete automation is not possible and secondly if we achieve fake automation define by vendor it will simply not deliver desired results

I think at least of now CI /CD /CT that enable smart FCAPS is key to introduce in Telco Networks, this should be a first step and that as we will see reply strongly on developing, using many of the tools in Operator Test environments that enable them to smoothly roll out the Next Era services. . A Culture of Shared Responsibility, as defined in the Scaled Agile Framework (SAFe) supports the aim of DevOps to tear down silos and build shared tool-sets, vocabularies and communication structures focused on one goal: delivering value rapidly and safely.

Bringing Synergy with IT for Telco Net-Dev-Ops

“DevOps automation can be a complex and arduous task as toolchains emerge from the vast number of tools used to deliver applications in faster more agile ways. I&O leaders require a top down view to designing and implementing successful toolchains. Tool chains are often built from stand-alone disconnected tools that are available to I&O leaders in their organization” Gartner

5

*Courtesy of colleague Mateo Červar

With Mature DevOps in IT and NFV/SDN bringing a lot of Open source tools the value chain requires an organic growth means it can reuse existing tools and leverage it with new Tools. Secondly it supports Mapping of many tools let’s say from open source to package it in one tools in simple to use interface like Nokia Matrix. In other words the DevOps tool chain must be Orchestrated as well .

Why I not speak about the Culture

It’s a toil of exercise to know how a big Telco want to evolve to DevOps and the only sad part is too much reliance on Culture and Processes without solving the Technology Architecture is like pushing water from top of a tank that is broken form bottom, the more water you puts in the more will be passed out. So, the question of technology framework and tool chain has to be solved first.

6

Infact as we try to apply the traditional DevOps concepts In Comm-SPs , the main reasons can be building a DevOps environment that can solve Operational issues in a way that do not require a major change in existing Operational environments of the Operators. Then the rigidness of Resources in Telco’s is far more than those existing in IT . For example the H/W resource in IT is cheap making it possible to roll out PoD in short notice and to apply pure IaaS model.

Can a Comm-SP adopt the same ? The current Infrastructure selection triggered by application itself and the high cost of NFVI makes it more difficult to build an open DevOps solution. Obviously these are not areas strictly depending on DevOps but rather deciding how Dev-Net-Ops will finally roll out in the Networks. I think these points also define value chain of Orchestration vs. pure automation. In most cases pure Ansible is not something that can automate an NFV environment than to use an Open Orchestration solution.

Cost of Net-Dev-Ops

In Legacy world and still in NFV transition the question arises cost impacts for DevOps . For this first point is that once the tool chain is ready and working still, we need to optimize it continuously. So, a Comm-SP should carefully select the use case to make DevOps scale to business requirements and ensure we not invest in something we cannot quantify to deliver business results

7

DevOps Monitoring

This is a BOQ Cost item which obviously lies outside the DevOps umbrella but so vital for Comm-SP’s that simply DevOps can not seem deliver value if it is not addressed. I personally feel developing a DevOps Tool chain specially the CI/CD tool chain is simply what is Telco teams require. In our latest user story with Delivery teams we understand from new feature or new product development all items on line across pipeline with clear ownership and progress is vital, After all this is what we all have seen from the GitHub about open software’s. Today industry stands in two extremes Redmine and Jenkins that simply is control and process with no tool chain and vendors tool chain like Huawei NICS or Nokia Matrix which simply is not open control and process and in worst case not easily to optimize across iterations. This is an important area STC have been participating and influencing partners and industry and I hope building a framework will speed Net-Dev-Ops adoption across industry.

8

https://slack.engineering/moving-fast-and-securing-things-540e6c5ae58a

Role of vendors in Telco Dev-Net-Ops

Comm-SP’s as of today comprise of networks with many vendors and such situation will become more prevalent as procurement will follow principles of open source supply chain. Hence a need to build a framework and followed by reference architecture that is agreed among all stakeholders is key to success. This commitment to continuous planning, integration, testing and deployment will deliver rapid innovation through collaboration. Continuous delivery in a multi vendor environment requires the automated integration and testing of all service components to ensure high reliability of service. Complexity grows exponentially when the number of service components and their individual version increases. Synchronizing delivery from multiple vendors is essential. Portions of the operator’s infrastructure must also be open to allow testing and verification of new solutions. New channels for the automated collection of customized operational feedback on demand will also allow development teams to improve their components based on production data. This chapter of paper is based on author discussion with Nokia sales marketing team

Automated Testing

The most important for DevOps successful transformation is selecting the right use case and what can deliver best results than automate the testing , its advantages are many for example it will allow a Comm-SP not worry about Release management or whatever changes in the infrastructure .CI and API based based Tool chains will definitely help solve the issues .DT is one of the operator who has really focused on this case and certainly it helped early adoption of DevOps .

9

*Ericsson public document titled NFV DevOps Life cycle automation PoC

Dev-Net-OPS to offer Security as a Service

Web scale architectures and early adapters already see advantage of offering pipeline as a service. Disney offer DevOps as a service for developers but for Telco’s NFV so much restrained by security will benefit more if we have a DevOps use case of security as a service.

10

Why I think so because in NFV/SDN with so many integration points and thousands of API calls in a complex multi-vendor solution means ensuring and monitoring a costly initiative for a company. In our recent study we found cost of deploying and managing security solutions for Telco DC’s is almost same as of deploying a solution and I sincerely believe DevOps on this front with Red-Blue teams will solve this issue

DevOps for Comm-SP’s Open Source solutions and analytics

There are two more areas I want to address in this thought Paper, the first is a question to industry can we really rely on closed or Fat vendors to offer us OSS solutions? I will certainly not support it because they simply not good at that and not know how to control, manager and support this. I think all of those vendors want to package certain OSS versions from community under their tool that is fixed and a new release will require lot of changes, simply stating it means a big commercial and financial hit for a company expecting new release every two weeks. Can we use tools with JFROG, SALT STACK like frameworks solve above issue?

11

The second main question is how to deliver automation cases that is based on data analysis like cross layer cross relation, data mining and ML . Simply stating both of above cases must be solved as DevOps use case and if we adopt to vendor solutions, we see great risk of lock in. A detailed study of these models will certainly help Dev-Net-Ops help achieve real early wins for a comm-SP who is trying to evolve to target 2020+ digital architecture.

I sincerely hope through this thought paper we have shared our vision of what will be the DevOps for Telco DSP , all comments are welcome to let us understand the Telco issues together to refine and redefine Dev-Net-Ops 2.0 . I hope such a thought paper will follow a collaborative engagement , shared responsibility and standards alignment if ever such transformation will create real value chain for the industry as a whole .

References

  • Juniper Automation with JUNOS
  • Gartner How to Navigate Your DevOps Journey
  • GitHub (https://github.com/capitalone/Hygieia)
  • For a deeper dive into application release automation features and functionality, see the Forrester report “Vendor DevOps automation”
  • Landscape: Application Release Automation Tools”
  • Nokia DevOps for the Telco World
  • Ericsson DevOps in Verizon a public document
Report this

Published by

Saad Sheikh  ☁

Open Networking. So Who am I and What I Exactly Do ?

Tata1-419

 

I think our professional life should be a journey which will help us rise from a fountain to become lake and evolve to a stream that will finally find its way to the big sea .

This bigness and fullness can only be achieved if one believes on comradeship and sharing and this is why I have started and managing my blog post and sharing page here

So who am I ? obviously my aim here is no commercial because i do a nice job for living . However i do expect i contribute to community and learn in a two way manner . This is what I have strived for all my career .

In few words I can summarize my career as follows

I am Senior Architect with a passion to architect and deliver solutions focusing on business adoption of the Cloud and automation covering both Telco and IT markets. With 15+ years of diverse experience in ICT including (10Yrs in Telco) and 5 Years in ICT (Telco +  IT) I proved my capabilities as Solution Architect, System integrator and Transformation advisor. I am focused in defining NFV/SDN/Cloud solutions based on web-scale architectures, HA, reliability and horizontal scale with a passion to map business requirements to robust E2E technology architectures meeting client’s requirements in most cost-effective manner. Ability to solve complex problems through architecture homogeneity and converged solutions. My work in carrier Digital transformation focused on Platforms including Clouds both Open stack and container platforms, carrier grade NFV, MANO, DevOps CI/CD, 5G and Edge Platforms, TaaS platforms and journey towards unified clouds including transition strategy for successful migrations to the Cloud .

  • Telco Domain:  NFV/5G/Core /EPC /VAS /Wireless (GSM /UMTS /LTE )/IMS /VoLTE /VoWiFi/MPLS & IP
  • IT/Cloud and Orchestration Domain: Cloud, DC virtualization, IT Applications deployment, NFVI including Server, Storage, SAN, SDS , CI/CD DevOps , NFVO and Orchestration ,MEC and Edge Clouds including 3rd party App development
  • Networking: Infrastructure Networking and SDN

for freelancing or knowledge sharing you can reach me at saads83nfv@gmail.com

Current Work Profile

Currency my work affiliation is with STC as Chief Architect consultant for NFV,SDN,Cloud and 5G Program . Specially focused to build solutions that can serve for company evolution towards Kingdom’s 2030 vision focusing around Cloud and Networks Automation .

Capture

 

Current Consulting Profile

As Chief Architect and President Advisor , I lead the company consulting and services business in 5G , Cloud and Enterprise solutions offering together working with 25+ partners  contributing my part to build the future of  Telco , IT and Enterprise business in Middle East .

Capture

 

 

CMM modeling to build a True Telco Cloud for Carriers (Empirical vs Hypothetical)

CMM modeling to build a True Telco Cloud for Carriers (Empirical vs Hypothetical)

Saad Sheikh

Saad Sheikh

NFV|SDN|Telco Cloud|DevOps

 

Integrated Approach from model introspection to Architecture transformation of CSP’s

As organizations embrace the importance of Digitization to meet the mandatory requirements set by the Industry4.0 evolution there are certain targets which all business whether small or big want to achieve which are nimble, efficient, open, and scalar and HPC (High Performance). Since the word Cloud translates seems like a One stop nerd jerky solution to fulfill all above problems so whether we like or not every organization want to evolve to cloud, though every enterprise targets may vary like For a bank it can be centralized control and Operations agility, for a CSP it can vision2020 to compete with OTT’s and nimble players and making situation more complex  for vendors like Huawei whose future is driven by customer requirements to update product, services and solution offerings to meet those requirements.

Believe or not every business, every segment evolving this way and Cloud is central to it. As an industry reference we can look at RedHat official annual reports in recent years and blogs from James Whitehurst and explanation of Cloud companies rise in this segment, Mirantis is not too far from the same. On services side we can see a big turmoil still as for your information AWS in 2016-2017 in Australia market captures 75% revenue from niche consultancy services (internet source) , similarly Cloud companies one can safely say that are in a state of turmoil the Cloud company want to put an Telco vendor Hat and similarly Telco vendors want to put a hat of IT companies everybody wants to give one message to Customer CXO and Strategist they know better than others with clearly missing one notion how to best create a value of customers?

I think best way to answer without prejudice and balance is as important as developing a solution because if evolution partner is promoting its own solution no way It can support organization to reach its target Architecture and meet minimum criteria of minimum enterprise continuum as we used to build the future Network2020 for a CSP .

So in this paper I want to bring some dimensions of How to make a robust design of a Telco Cloud , how to smoothly integrate it in the Live environment , I will cover the domains of NFV , Telco Cloud , SDN its integration and Agile delivery based on Telco DevOps approach . This is my focus area for many years and I want to share my own idea as a network architect on this direction to the Network architects , the purpose is also to indirect propose an approach for any open System integrator to execute such transformation in the most open and transformation manner .

So first for an architect we must list down difference dimensions and how IT Cloud will be different than Telco Cloud to benchmark a transparent index to select ,evaluation and optimize !!!

1.  Solution Selection

How to select the best products for your Network, I think there is no single answer and most clear answer is driven by requirements , not only technical (SoC , Proposals , KPI ) but most importantly on commercial and commitment from partners to strive to meet customer requirements and fulfil smooth migrations . Not only delivery is important but the Support service SLA is importantly important. we all know the traditional RPO/RTO model as we used to see in the IT services can handle all CSP requirements, believe or not IT companies reliance on so many partners and total separation of H/W selection and application services is the biggest concern for a CSP and a matter of principle the partner who shall combine different layers will leverage maximum advantage for the Enterprise

2.  Maturity of Solution

In the Legacy world all vendors used to develop in one direction, lock the standard first in standard first, then do a pilot project for solution hardening then go for mass rollout. Only issue is that in NFV world with standards refreshing changing every 6months it’s not the technology but the speed with how we harness them. It means the main factor must be that when a Bug comes with so fast changing standard who will Fence it and block from CSP environment. Partner capability in eco system development, Open Labs, EANT Labs (my favorite initiative) and Plug test (4th one and we are straight participating and evolving through it ) will define the Value Chain to ensure Openness does not mean non- Standard and solves the one main issue with Innovation in all those years !

3.  Use Case development  

Rome was not built in a day as adage goes so goes in open world, the Solution must be Use case based and Cloud is not an exception. I think for CSP specially Tier1 the Key use case of NFV Telco Cloud are as follows

Lock the Interface specification specially Ve-VNFM-VF/EM

On boarding API’s and Image parameters standardization

Simulate a value chain such as done in OSM community like vCPE, vIMS, vBNG end to end through NFVO /Tacker

I think in IT Cloud too much work talks to the Code and as matter of principle cannot fulfil how Cloud must integrate with the Telco environment

So as we see it our target in Cloud Journey in 1-2 Years must be

                       Target1: any new VNF /Application coming from any vendor even in house can onboard on any Cloud in 1month Maximum

                       Target2: any copy or clone as we used to say must not take more than 2days, 1 day for onboarding, followed by another day for tuning and validation through tempest, Rally, Cloud Performance, Cloud Availability etc to mention the few

In the Re-Architected world the Capability continuum must sum at something like 60-90min for any application but this target need standardization of interfaces and obviously need Re-Architect process completion first.

4.  Represent a Telco Device as a code 

Rome was not built in a day as adage goes so goes in open world, the Solution must be Use case based and Cloud is not an exception. I think for CSP specially Tier1 the key is to know steps of Telco PNF modeling as a code, it seems simple bur involvement of so many community and standard body make it complex also. We need to refer Point#6 to answer this part.

5.  How to develop Networking Solution for a Telco Cloud

For an IT Cloud this seems like a standard simple shot to design the Underlay and to overlay the traffic using BGP /MPLS using an agile way. Contrail which is market leader gives a good overview of this in the contrail community updates. However as we move to the Telco Cloud things no longer remain the same and whole story must start from POD or which my transport friends call Fabric design . Sorry for mixing POD with Fabric but fact is this is an important idea of a Telco Cloud. For a Telco Cloud based on OpenStack, a POD should be self-contained with APIs (controllers), compute nodes, and storage as well as any network nodes including the infrastructure. So whole idea is that east/west and north/south is segmented and controlled in a way that E-W traffic must not leave the POD and . It also helps if they are setup in Availability zones, this way workloads that are east/west intensive can be assigned to PODs where they won’t have to leave the POD for east/west data. POD sizes should be small and there should be a lot of horizontal scaling. A hefty good paper from big switch explain this part http://go.bigswitch.com/rs/974-WXR-561/images/Core-and-Pod%20Overview.pdf

Moving up the next thing that come are bridges Br-int, Br-Ext and BR-Tunn, actually Bridge is deployed using ML2 Neutron drivers as a plug of OVS

To make design simple the Management zone entities like Mirantis MCP, RH Over cloud, Ceph can use Standard ML2 bridges with the OVS which is also a default configuration.

For the VNF the mature VNF vendors propose the idea to industry that bridge is not used because most HPC VNF’s will use Provider network and not the tenant networks for the workload. As this requires VNF directly access the Fabric network so no need vNIC presentation to the Bridge and this is the reason for the OVS-DPDK scenario no need to deploy a bridge on the Computes , for SR-IOV it will be more simple . Also as per community recommendations it is suggested not to deploy Standard OVS with the OVS+DPDK on same host for the reason that the Kernel OVS bridge which is based on ML2 Plugin will cause Br-Int as single transfer point and can impede the performance of a Telco Cloud . From OpenStack Pike the community can work really how to make standard configuration and architect design for whole Cloud ………..For now this issues is left to the community.

In my later blog I will talk how to optimally plan your NFV DC consider any VNF, any Cloud to decide on optimum Flavors, OVS and Bridges mode and as a matter you may come to architect a solution where for Customer-1 you may think deploy every VNF as SR-IOV will save you more NIC/CPU resource than another scenario-2 where the Deployment should follow the VNF principle like a OVS/EVS for IMS while a SR-IOV for BNG etc. This itself is a detailed topic and author believes need to shed light in detail and will be discussed separately.

For networking BGP will be the key, with SDN as well as without SDN. One of the most important architect component shall be to know the fact that the SDN and specially the contrail https://www.juniper.net/us/en/products-services/sdn/contrail/contrail-cloud/ consider BGP to integrate both future Cloud and non-Cloud Network part , the Open Stack consider BGPaaS so DC should consider this part .

 

6.  Telco DevOps is not IT DevOps

In the Year 2011 the webinar introduced me to this strange term of DevOps , over the years I have come to conclusion that since NetOps is NE focused while DevOps is IaaC focus , means IT only want code abstraction and automation so the two cannot be same . In my recent talks with leading Community architects and mentors I have come to following conclusion about this

a.   Linux scripting is the key and you cannot do DevOps unless a CSP cannot free their OPS from Infra OPS , must go scripting way

b.  The Cloud must support multi cloud. I means Telco DevOps must watch for the future of deployment of OpenStack + Container , using an open source tools like Spinnaker , Rally will be a nice idea to go

c.   Link to the community, the Crazy Mirantis MCP is a great example of this using SALT formulas, Meta model standardization and artifacts pulling through gerrit and Jenkins is a nice crazy way to automate you Telco Cloud.

d.  All are IT terms how about Telco, the Telco DevOps must consider TOSCA, HOT, YANG and YAMl scripts automation and obviously its abstraction. You can follow my little crazy profile on Jenkins and Code Club talk about this. But code is useless and logic is the key, so how to successfully model your Telco NE’s in software and script is the real challenge. Second will come how the Telco DevOps will join the hands of IT Cloud to automate the whole ecosystem through single point of access which is the orchestrator

e.   Finally to guide for transformation you need move from PMI-PMP to PMI-ACP as I did a year before , teams , process and tools to support it to develop an Agile Process support this , what is target for new requirement can meet in 2weeks maximum from idea to offering .However it seems like a long journey to reach it

7.  Automated testing

For a IT Cloud it’s a good idea to automate everything but for a Telco Cloud the nice idea is to start from test automation, if a NFV project can automate 50% test and on top for any Infrastructure change can validate all environment automatically it will be ideal to build a unified Telco Cloud, a nice bench mark to start automation journey in hill climb phase must be

  • New DC Clone (75min)
  • Cloud verification (65min)
  • VNF deploy (55min)
  • FR/NFR test (20min)
  •  Continuous monitor using AODH , Gnocchi , ELK , Nagios is a nice point of start Telco Cloud Journey

Finally the Cloud Transformation partner must have a Solution continuum to support such service like 50+scnerio and 1000+ cases which a project customer can select and optimize.

8.  Multi-Site Cloud Design

What makes the Telco Cloud different from IT Cloud is the way the service will be orchestrated and healed. For example cross DC , backup’s and network scaling to mention the few , as per Telco Cloud Design foundation Tricircle and its shared network design is a key to plan a Telco cloud , a nice new community Trio20 which talks about Nova ,Cinder extension across sites is another nice idea . The one important concept will be SAN or Ceph as it is the data container of all Instances, and for Solution selection any SDS will work as long it can handle for the structured and un-structured data

9.  SLA Multi-Site Cloud Design

It is very common to say Telco Cloud as 5 9’s vs IT Cloud as 3 9’s but how it should realize is the key to successful realization , below guide line is a nice idea to start with

a.   In OpenStack use HAproxy , mem cache , redis to offer HA

b.  VNF blue print to support HA by domain , by service

c.   Service pool or load sharing design to control HA by Telco Service

d.  VM and Aggregate design

 

10.                  HPC Cloud  

HPC or High Performance Clouds are the way NFV should build , the NUMA design is the key on NFVI , will NUMA will be crossed , how to adequately match the CPU cores , PCI Pass-through support like for VNF1 use DPDK , VNF2 use the SR-IOV , similarly dynamic Huge pages design is a key . Although it is a good idea to plan a uniform huge pages but as per our experience this will result in huge compromise in optimum resource selection and hence an architect must consider.

11.                  Integration tool set   

The IT application as a principle are standalone applications and only Vn-NF interface is enough to make them work while for the NFV Cloud it are 10+ interfaces like NFVI , Ve-VNFM , Or-Vi , Or-VNFM , Os-Ma , Se-Ma etc

How to standardized these interfaces and API’s for smooth onboarding and resource orchestration and developing the corresponding tool chain is the key for transformation

12.                  IT Cloud Migration vs Telco Cloud Migration

In a typical IT Cloud migration you must normalize and study only the DB replication and instance states but for Telco Migration the list can be exhaustive including meeting Customer KQI etc , PNF to VNF migration and managing operations evolution , customer experience and Smooth evolution is the key to success

13.                  Multiple Hypervisor Support

In an IT Cloud we primarily talk about XEN while for NFV Cloud the ETSI talk about KVM in their Reference Architecture but VMware Stake is as important and design must consider how to incorporate KVM with VMware. The driver testing validation and pooling of both is key to expand the cloud. In this phase of the industry may be it is difficulty but for Industry4.0 this will be target reference architecture.

14.                  Open Platform is the real problem in Telco Cloud

IT cloud is disruptive and very open, it is Open based on API’s, while a Telco Cloud requires lot of standardization such as ETSI RA, Plug test, EANT, Community etc

15.                  EPA and building a High performance Clouds

I remember the inclusion of EPA in OpenStack and its use case for 4K video the famous one in community to prepare different hardware for different use case, like make best use of memory, Disk, RAM, smart NIC etc. A company named mellanox have done a hell of job for this work. To sum up in the future converged Cloud following will be key to deploy a true NFV cloud

a.   NUMA and the CPU affinity need focus how to assure workloads placed in right NUMA , you may need make design align VNF , NUMA , OVS/Bridge and CPU/NIC

b.  TLB buffer size and associated Huge pages normalization for a target Cloud

c.   CPU pinning main focus must be the PMD threads , their dimension criteria and allocation across different VM’s /vNIC

I personally think there seems like an inherent Pandora box in Community specially for OVS workloads that needs to be addressed for SR-IOV it seems OK as NIC pass through can be used to achieve the Line speed

16.                  Architecture transformation

For the Architecture you must consider migration as ultimate target is move some service from As is to To be Architecture. The key principle for IT Cloud is App focused as all customization is to be done on App not the Cloud. This makes a Public Cloud very simple to manage and automate , conversely the Telco Cloud is a different world , the PNF migration will need understand the detailed analysis of service requirements and the to customize the Cloud like EPA , HPC , THP etc to meet the Telco Service needs .This is very important point for the Architecture .

Conclusion for the CXO Office and Chief Architects 

Finally I want to wrap this paper with the summary that Telco Cloud is not the IT Cloud, as Network architect you must quantify the real requirements and meet with both the Solution maturity and the service capability of the partner. To summarize the dimensions which make Telco Cloud different from IT Cloud are MVI IoT , KPI , Performance , Elasticity , Security , HA , Service SLA /KQI , NFV Assurance part and smooth operations . For Tier1 which want to transform there are some more areas to look like how to transform through DevOps, Process, Tools and Skills. This al need to evaluate from Telco service point of view. I think till now why despite many commercial Telco Clouds the Migration has not happened is not technical, it is because the Architecture not consider the fact that this change is not the technology but the Enterprise architecture and Operations will be the biggest impact for this .This is the reason a strong partner with IT skills and Telco mind is the key to remove the impediments and achieve quick wins for the business.

Closing the discussion ,well what we infer from this paper is that IT Cloud concepts no longer linearly applies to the Telco Cloud specially if you architecting a cloud that will future host the vAPP seamlessly on a common DC . Similarly concepts of Python, Net flow no longer applies to a case where you need Service abstraction and automation from business and CRM perspective. Specially consider future OSS the Business use case need model to Service catalog and one click instantiation , so you can see the concept is not the same at well . Similarly scalar VNF is not the same idea as we know Scaling in IT Cloud which is a mere VM and resource expansion. Those who are new on this part may need to learn statistics first especially how LMA and DLS algorithms really support for the Network automation. Seeing it altogether NFV is a system while Cloud is a Platform so concepts of E2E QOS, HA, SLA, Security, FCAPS and building a flatter architecture will be key objective of CXO office.

Nevertheless no architecture is complete unless Security is built in. We all know  Cloud took longer acceptability time from end users due to this and a matter of principle security need design in instead of design out because Open interfaces also means that everyone can know the language of how to talk to a component it means the advantage can be exposed . Inherently the API security through HAProxy, image compression can address high level issues but control of tenants will be key especially for the future converged clouds, the biggest challenge as CSP will open the Cloud will be IP related because Self-service case means many malicious attacks from Unknowns. Hypervisor security and future separating the Host OS per domain security in Kubernetes will be the main concern for the CSP’s. The Network is just opening imagine the case where the future S/W developer from a university will be invited to access your cloud to write an application for you. How the Chief Security architect will accept it. It is not acceptable now but will be acceptable as we evolve to build architecture moving along for this the key point Cyber teams need to remember is division of domains and duplication of analytics . Confused I mean IPtables + Security groups +ACL but it will impact performance for now so to support such architecture NFVI need evolve , Pike standard is just addressing 80% of such scenarios and we do write to community to ramp up this Part . May be you know now in the community this talks is given most important and infact you can join this work also.

The Market has just opened there are many partners along , even we know cases where having Cloud expertise can be shown by company a sign they know the evolution part , this is a trap knowing service and its assurance is the key , to best to survive is who is more open . For me Open means Tireless effort to bring value to customers and make community feel. Hey Guys we are together, we are Open and we will solve together!!!!

Finally every business has right to separate wheat from Chaff and we should embrace all that glitter is not gold. Best of luck and see you later in the new edition

The paper cannot be considered complete unless I thanks following

  1. Ben Silverman Principal Architect OnX Cloud , a teacher , a friend , my mentor
  2. Jaakko Vuorensola from Redhat a longtime friend and influencer
  3. Ajay simha RedHat Chief architect for the crazy work for the reference architectures
  4. OPNFV ,ETSI , OSM are obviously bible for all this work , how to solve issues in Open Source way is obviously a nice idea to have while evolving from NetOps
  5. Customers and partners , your questions and problems is what define my writings

And obviously the crazy consulting team in Huawei, together we believe understanding customer real requirements and to build a solution is the best way to transform business and to bring long time value to customers, partners and industry.

Sheikh is Huawei Middle East Senior Architect for NFV, SDN, and Telco Cloud with focus on ICT Service delivery through Telco DevOps. Focused to define the Roads for future 5G Core Network. Always interested in those disruptive technology driving the industry transformation, Author hails from Telco CSP background and since 2013 working on Telco Cloud domain including Amazon, Huawei, Mirantis, VMware, RedHat etc . The comments in my writings are my own and shall not be considered as any relation/binding with those of my employer .

Adapting the Elasticity use case for Scalar Telco Clouds

xx

Sheikh is Huawei Middle East Senior Architect for NFV , Telco Cloud ,SDN with focus on ICT Service delivery through Telco DevOps . Focussed to define the Roads for future 5G Core Network . Always interested in those disruptive technology driving the industry transformation

What are Telco Applications in software means ? . A modular ,segmented code with defined API’s that can be customized for different functions and requirements .For Telco applications with varying requirements including uneven usage, bandwidth ,spikes during periods, the requirement for having built in elasticity and scalability is crucial for larger POD’s environments .However for such cases the applications should be designed to detect variations in the real-time demand for resources including but not limited to bandwidth, storage, compute , SLA and KPI to add for Telco use case. However, in legacy world applications had been developed to run on a single machine and require re-coding to adapt for both the scalability and elasticity that the Telco cloud provides.

The scenario that Telco Applications are persistent in nature make the scnerio more intricate to achieve

There are two main concepts to understand between Elasticity and Scalability in Telco Cloud.

  1. Scaling Horizontally or Scale Out /Scale In:

Those guys who have delivered a NFV based Telco Cloud knows the Scaling is supported currently , what it actually means is that EMS will notify current load to VNFM in real time or Polling way and it can make decision to add/remove a new VM and launch new instances of VM for the application . This do require an algorithmic policing to balance the load of vAPP on all new and old VM’s in a controlled fashion. Normally this is practice used in NFV Projects now

  1. Scaling vertically or Scale Up /Scale down

If you are not following industry may be you will not be well aware what this will mean for NFV? This is concept normally coming from AWS use case of Netflix for Persistent workloads a use case which AWS supports for large Enterprises. It will mean that moving a vAPP real time to a bigger /smaller VM or resizing the VM.

  1. The myth of Scaling

As we can see as long as the vertical scaling in Telco environment can have a Telco DevOps as I use to call the process to verify Build, I&V, Test the service in somehow automatic way .It will make a perfect case to avoid trap of load balancing and validation through long exercise of manual testing and MOP following by front line.

Incorporating this model through OPNFV test projects like Functest and Yard stick will mean this model can help for fast scaling and complete DC expansion more quickly. But obviously Industry need to consider how to assure no impact on VNF performance, SLA, KPI and that such scaling must be real time on persistent workloads?

Currently this case is not supported???? may be one alternative is use Telco service capabilities to isolate certain VNF first and then apply this model, but for sure it will have challenges in case VNF is split across sites in a federated network?

This model will be the key to deliver agility on big Telco DC , the model 1 as used now in NFV world will be obsolete in coming 2-3years after all Telco’s also want to become the Netflix of industry ………………………….:)

It will mean NFV world will move one step ahead from Being Scalar to being elastic for the workloads.

  1. Key Challenges

However to evolve to this model requires Industry to agree and solve some key issues

  • How to solve FCAPS and concerns from operations customer
  •  How to assure KPI before and after
  •  100% automated test after expansion
  •  100% automated performance validation
  •  How to introduce service for workloads? Consider there will be running traffic before expansion
  •  How to address customer experience for such cases
  1. Watch the Road Ahead

Finally to wrap us we must also think how soon Openstack the unified standard for Telco NFV will support evolution from PNF to VNF and subsequently Kubernetes can deploy on it to support Scaling through containers . It is sure finally this will solve issues but till the Telco world key concerns as higlhighted above are not adressed . The road to reach there seems gloomy . Do watch below video to see whats happening latest in this domain as a take away from Boston summit .

I will like to know your thoughts on this matter , because for persistent workloads the scaling have more challenges than ephemeral workloads and journey is challenging but interesting .

This will be the key to success !!!

 

2017 Road Ahead for the Maintenance Solutions in Converged Clouds and Hyper scale networks

 

We are in 2017 the 2016 was a bumpy year for the NFV/SDN industry realization .Together we have come far to enter from Concept phase to commercial realization phase, it seems the Clouds are mature now and All large Opco’s and small enterprises alike are happy to achieve the real advantages from cloud which is Scalar, High performance and Modular but the story is not over.

The 2017 business model of all large Tier1 Operators is circumvented by two main concepts, (1) The first one is the converged clouds and (2) The later one is the Co-Cloud.

This is why for their Clouds they are looking for FCAPS and Maintenance solutions that are scalar in a vertical agnostic fashion

To draft a business solution for FCAPS and Unified Maintenance in this scenario is very complex because traditionally each of the solution requires a tenant based analysis, processing and output which can be further customized based on RBAC functions .The task becomes more complex as we solve the myth and find that Industry leading cloud vendors have given less and not enough importance to this complex task and demarcate FCAPS boundary as the Open Stack Boundary itself which is not reasonable after all the industry is evolving from PNF to VNF and lot of points need to integrate well to replicate same or better maintenance functions of IT and Telco converged Network

There are two schools of thoughts prevailing in industry till date as per the OpenStack latest standing as per Newton Release and SDN Maruno release .

School 1: Uses the Silows and offer separate solution on Physical and Cloud/Virtualization layer (Huawei esight and Fusion Manager together make it place in this slot)

School 2: Uses the Cloud OpenStack services to unify the relation and offer complete aggregated FCAPS from single Pane of services of Telemetry like Ceilometers .The idea is good but practically till date it is a naïve idea gained less attention in the industry. In fact till now the clear mapping between two layers no IT vendor can clearly explain. In fact within the community there have been discussion on this point with no clear solution to help IT identify and solve physical alarms easily using Cloud services (Huawei System Integration Services is key to customize solution in this part)

Customer in converged environments require real time Push solutions based on API’s for key metric like Current Status of DC ,Resource usage , Resource Utilization , Tenant based metrics , Log analysis

NFV industry Defined as “monitoring as a service,” Monasca in a large framework that incorporates all aspects of monitoring, including alarms, statistics, measurements and more, for all OpenStack components. Monasca largely focuses on allowing tenants to define what should be measured, what statistics should be collected, what should trigger an alarm, and the best notification method. The relationship between Monasca and Ceilometers is currently being defined.

However still Monasca is a very vague idea , since last MWC Barcelona the Community is making many efforts in the Doctor Project and most of this effort try to replicate the existing deployed tools in the IT to the Converged Datacenters like Zabbix , Nagios , Zenoss however the fact is that they can not offer the complete solutions , this is why the industry now have a consensus to unify the Tools upstream like all of above three can be aggregated in a Open Source Project like Sensu , Similarly for Perforamcne Collectd and for logs Fluentd and the list will go on like this . https://wiki.openstack.org/wiki/Operations/Tools

The point is that in the current wave of NFV and SDN Projects the CIO sees to reduce so many tools to list of 3-5 to control the support, bug fixes and other issues that seem to be really big issues in a commercial company. I think community is doing best but still without the summation of these tools it seems difficult.

The Next wave as I see is the Unification , last month I have met couple of senior IT executives and they all have a believe with Organizations following a digital transformation path , they think there is a way to collaborate and unify many of the information in a Single Pane form operations perspective . It will open a Funnel pipe for accurate management of the cloud in terms of correlation and root cause of operational nature and obviously can build the Big IT intelligence mechanisms for a future programmable way to do things.

We need to a consensus a seamless integration and unification is a key for programmable network otherwise micro pieces even programmed in an excellent fashion cannot deliver the promise of NFV and SDN.

Just to give an idea below picture is from https://www.sdxcentral.com/ blog on SDN main components with the inclusion of more and more open source components it still looks very curious how the issues will be consolidated and solved

???????????????????

At Huawei I believe we are almost there close to community, industry, customers and market alike to build Tribalizer for Wave1 and now we need to be more open and cooperative to form alliances with other vendors and customers alike to find the Open Solutions for industry issues of how to actually achieve the Wave2?

https://www.linkedin.com/pulse/api/edit/embed?embed=%257B%2522type%2522%253A%2522link%2522%252C%2522title%2522%253A%257B%2522localized%2522%253A%257B%2522en_US%2522%253A%2522The%2520components%2520of%2520SDN%252C%2520NFV%252C%2520MANO%2520%2526%2520OSS%2522%257D%257D%252C%2522description%2522%253A%257B%2522localized%2522%253A%257B%2522en_US%2522%253A%2522When%2520you%2520curate%2520a%2520website%2520%252F%2520blog%2520called%2520PassionateAboutOSS%252C%2520it%2527s%2520pretty%2520obvious%2520that%2520you%2520would%2520see%2520the%2520world%2520through%2520OSS-tinted%2520glasses.%2520The%2520tech-world%2520at%2520least.%2520And%2520so%2520it%2520has%2520been%2520for%2520me%2520when%2520evaluating%2520new%2520comms%2520network%2520%252F%2520service%2520concepts%2520like%2520SDN%2520%252F%2520NFV.%2522%257D%257D%252C%2522author%2522%253A%257B%2522name%2522%253A%2522Ryan%2522%257D%252C%2522provider%2522%253A%257B%2522name%2522%253A%2522Passionate%2520About%2520OSS%2520and%2520BSS%2522%252C%2522display%2522%253A%2522Passionate%2520About%2520OSS%2520and%2520BSS%2522%252C%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%2522%257D%252C%2522request%2522%253A%257B%2522originalUrl%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fthe-components-of-sdn-nfv-mano-oss%252F%2522%252C%2522finalUrl%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fthe-components-of-sdn-nfv-mano-oss%252F%2522%257D%252C%2522images%2522%253A%255B%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fwp-content%252Fuploads%252F2014%252F06%252FSDN_NFV_Drawing_v2.png%2522%252C%2522width%2522%253A1096%252C%2522height%2522%253A632%257D%252C%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fwp-content%252Fuploads%252F2017%252F01%252FECOMP.png%2522%252C%2522width%2522%253A602%252C%2522height%2522%253A538%257D%252C%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fwp-content%252Fuploads%252F2014%252F05%252FMasteringYourOSS_CoverImage.jpg%2522%252C%2522width%2522%253A400%252C%2522height%2522%253A500%257D%252C%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fwp-content%252Fuploads%252F2015%252F06%252FOSS_LandscapeTechSolnOrg_June2015.jpg%2522%252C%2522width%2522%253A170%252C%2522height%2522%253A220%257D%252C%257B%2522url%2522%253A%2522http%253A%252F%252F1.gravatar.com%252Favatar%252Fa564e905c3c15fc7268bd86149e6626b%253Fs%253D68%2526d%253Dmm%2526r%253Dpg%2522%252C%2522width%2522%253A68%252C%2522height%2522%253A68%257D%255D%252C%2522data%2522%253A%257B%2522com.linkedin.treasury.Link%2522%253A%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fthe-components-of-sdn-nfv-mano-oss%252F%2522%252C%2522html%2522%253A%2522When%2520you%2520curate%2520a%2520website%2520%252F%2520blog%2520called%2520PassionateAboutOSS%252C%2520it%2527s%2520pretty%2520obvious%2520that%2520you%2520would%2520see%2520the%2520world%2520through%2520OSS-tinted%2520glasses.%2520The%2520tech-world%2520at%2520least.%2520And%2520so%2520it%2520has%2520been%2520for%2520me%2520when%2520evaluating%2520new%2520comms%2520network%2520%252F%2520service%2520concepts%2520like%2520SDN%2520%252F%2520NFV.%2522%252C%2522width%2522%253A-1%252C%2522height%2522%253A-1%257D%257D%257D&signature=AabES296V41K11DCMgJo9iqC1zcB

The Technical task is daunting but the direction is clear. Openness, alliances and sharing is key to success and together with our partners we want to build the Future Network of customers

References:

https://wiki.openstack.org/wiki/Operations/Tools

https://sensuapp.org/docs/0.26/overview/architecture.html

http://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_aggregation

http://www.fluentd.org/

https://github.com/centos-opstools/opstools-ansible

How to plan the Best Server Solution for a Converged Telco Cloud Data center

During a recent workshop one question popping up which server make is the best to be deployed in a Telco Data center ?

As a Solution Advisory i know that such a question has no fixed answer and answer depend on solution , its basis and long term business and network strategy of customers but still for such questions customers require explain in terms of experience in similar projects and how to actually make such a comparison ? Specially for converted IT and Telco Data Centers .

To start with for this scenario in carriers business we have two solutions viz. Rack Servers and Blade Servers to fill in the blank ; the first thing we need to look at is the industry of NFV and Enterprise till date .What we know is that in NFV normally we are using 42U racks and Blade servers like HP7000, C7000 Gen8/Gen9 Servers, and Huawei E9000 all are blade servers and also Dell M630 also a blade server.

Technically we all know both server types are COTS X86 machines so must be OK to use any of them then why industry has come to such thinking of adopting the Blade servers? After detailed research and compared the benchmarks I can summarize the selection based on following items

1.Performance: in a Telco Cloud, performance is the killer decision because customer will only adopt the solution if it is same or better than the legacy one. We all know that blade servers processing in terms of V3 CPU Core, Disk and Obviously DRAM is much faster than Rack. Although the advocates of Rack servers proclaim with new technology the expansion in upgrade in rack servers is easy but it is balanced when we know the Space /Power /Maintenance of cabling issues in Rack server deployments

2. Cost: There is no doubt the Rack servers main intention was to support mass deployment in data centers which require less performance and quality, For example Google Cloud, Amazon Web hosting, Face book .In fact for these data centers it is more pragmatic to make expansion in bay and for complete racks to have cost advantage, this is one area where Rack servers are certainly doing better than blade servers

3.Scaling: a Telco Cloud must be hyper scalar .It means we can increase by server, by module without complexity. In fact for both type of servers the Compute node expansion is possible by scalar way but the complexity for O&M and installation for the two is not the same. We all know Rack servers need more cabling, more artifacts to get the job done .So for me it lacks promise of scaling

4.Energy: In a big data center it matters most to save power. Comparing the different solutions from different vendors we found that the Blades are more compact. I knew there is a term famous to data center guys about Power usage like power provided to application specific nodes to total power input. Sorry I fail to recall the ratio name but it is far better in Blades in Telco Special case

5.Infrastructure cost: The blades are best from Form factor point of view with Shared power, switches, disk and is a reason for less cabling needs compared to Rack

6 Solution unification: I think all Solution Architects know in a complex ICT world all the big carriers are trying to consolidate both IT and Telco domains to come to a unified solutions Finally .the initial High CAPEX is OK if it promise large OPEX reductions in the TCO cycle. Since Blade servers promise certain optional features and functions which Racks do not have so it is reasonable to deploy with Blade servers or to select multiple type of rack server’s .To explain this one can see the open Stack EPA (Enhance platform awareness) case what we know is that the ideas of Solution unification fits well for NFV when we use Blade servers

I hope these bench marks will enable you to plan your data centers but please note I am enthusiastic writer and blogger. So the comments and analysis in this paper are Sheikh’s own as per industry reach and self understanding, they do not represent any official understanding from my Employer Huawei.

Secondly there are many white box vendors and market of both Rack and Blades are huge. You may come to a situation where the Blade in analysis is worse than the counterpart so the reviews are more relevant if you are planning a Telco Cloud because this is my main focus area…

Adoption challenges in Open source and opportunities for niche System Integrator

In 2019 the IT and Telecom industry has really come to synergy point through the power of open Source solutions. I still remember days in September 2012 when we are firstly looking the Open Source solutions .It was really difficult that time to even think the Open Source can integrate the so called real Isle Industries of IT and Telecom. However the fact is that big Companies already doing production ready work for consolidation of IT and Telecom Datacenters.

What is most exciting for me is that I started 2019 with a CXO workshop asking the way forward to include the Banking Application on the same Data Center. Obviously at this time the idea is very vague because the Banking as we all know requires highly secure and reliable. Many of the ideas of traditional IT and Telecom cannot be directly applied there .However the message is clear the future IT services will cover whole Ecosystem and infact each industry is Just like Application appliance .

The Cloud Based infrastructure is an idea which is so important to develop in multi tenant environment because each vertical will come with its own set of requirements. It is clear that there is huge market demand for this and actually no vendor can eat all this market though many will try to do this.

This type of Cross industry and Multi tenant solutions will strongly depend on the power of Open Source solutions to couple the solutions form many vendors as a module. The concept which is known in industry as a micro service .However the integration of all these Open Source is very complex especially when it comes to adaption and matching CIO requirements. It is because although the solution can be Open but customer requirements can be customized. It is because no customer want to live in a house build on the wish of someone .The Personalization of each customer solution is very important.

Obviously all customer wish to transform to an agile infrastructure and reduce OPEX but really what must be there first steps like how to select the Components from the Open Source Solution? What must be there Cloud, VIM, MANO, VNF, Repository etc. This type of question many CXO already put to vendors and the answer of each vendor depend on its business model like what is there Ecosystem without delving to details whether it can satisfy customer real needs

I think this strategy is damaging the NFV adoption on a wider scale, because vendors want to push a solution while the best strategy for open Source solution must be a Pull solution which matches customer needs. Without this it is difficult to increase the real demand of NFV and Open Source Projects

After selecting the software the next daunting task must be how to make it functional.  The product view of this scenario is to develop all applications as a VNF Appliance through image file. It means any application can be available and only usage is licenses. However the story is not so simple. Infact it is in stark contrast to the Idea of App Store as prevailing in IT industry.

The part of such endeavors must be how to efficiently use these images to develop a service, something customer really need, way to auto test and auto deploy the service and obviously how to apply Puppet, Ansible and Chef Play books schemes to achieve it. I mean to say just the Appliance and their abstraction is not enough how to model the service and make it interwork is key to success for this realization

This type of open Solution ecosystem will support the wide range of adoption across the industry; maybe we will see the NFV going live in both Private and Public Clouds and the application will be as freely distributed as the Windows of today

However with more and more open solutions the dilemma to integrate them will become more and more important question by all CIO’s , not only this we will come to a situation where customer will question the reliability and performance of open source solutions in commercial deployments ? Can SI can use all open Source tools and offer something a single Pane glass operations modeled around Open API’s and programming abstraction .

This is the real value of System integrator, the Hercules the IT and Telco industry waiting for years.

Can 2019 be the year of that SI’s making an impact? The story is open and the best team will make it happen. System integration team has to listen to customers listening to the industry, partners and customer alike to solve the issues of Ecosystem.

Collaboration is the key and sharing of experiences to solve industry issues is critical to be the System integrator of choice for the industry

https://www.linkedin.com/pulse/nascent-open-source-solutions-success-looking-si-saad-sheikh-be-pe