How Open Orchestration enhances  Enterprise, 5G , Edge and Containerized applications in Production

Picture1

Source: ETSI <www.etsi.org>

 

How Open Orchestration (OSM Release-7) enhances  Enterprise, 5G , Edge and Containerized applications in Production

An architect’s perspective from ETSI® the Standards People

 

As highlighted in the Heavy reading latest End-to-End Service Management for SDN & NFV all the major T1 Telco’s are currently refining their Transformation journey to bring standard Orchestration and Service modeling in their networks , one of such standard approach is promised by ETSI OSM a seed project from ETS® the standards people .

Recently in Q4 2019 ETSI OSM release the Release7 which address surmount challenges of brings CNF and Containerized applications to the production ETSI OPEN SOURCE MANO UNVEILS RELEASE SEVEN, ENABLES MORE THAN 20,000 CLOUD-NATIVE APPLICATIONS FOR NFV ENVIRONMENTS

This capability of ETSI® OSM is specifically important considering the ushering of 5G SA architecture and solutions which already find its way to the market thanks to early work from CNCF and specifically CNTT K8S specs . OSM brings value to the picture as it will allow to design, model, deploy and manage CNF’s (As ETSI NFV call is a containerized VNF) without any translation or modeling. It also lets operators experience early commercial use case of integration Helm2.0 in their production environments. On top of it will allow a NS (Network Service) to combine CNF’s with existing VNF’s or legacy PNF’s to deliver complex services in an easy to deploy and manageable manner.

In the following part of this paper I will try to share my understanding on OSM release7 and sum up results from ETSI OSM webinar on this subject held on JAN 16th 2020 . For details you may need to refer to webinar content itself and can be found https://www.brighttalk.com/webcast/12761/380670  

Why Kubernetes is so important for Telco and Enterprise

Telco industry has experienced lot of pain points the way NFV journey has steered with focus on migrating existing PNF’s to the Cloud. K8S offers opportunity for all Platform providers, application vendors, assurance partners to build something on modern principles of micro services, DevOps and Open API’s driven. This is something that already made its way to Telco’s in OSS and IT systems as an example mycom OSI UPM , OSM and  infact ONAP all are already based on Kubernetes , the arrival of 5G SA and uCPE branches has driven almost all operators adopt networks to use Kubernetes . Further it is principally agreed as CSP’s move to Edge the K8S will be the platform of choice.

Foundation for K8S Clusters

Kubernetes made it simple for the applications and CNF’s to use API’s in a standard fashion using K8S Clusters which are deployed either in an open source manner or via Distros. The early adoption of CNF’s in Telco largely supports the consumption model of vendor Distros like RedHat OpenShift, Vmware PKS, Ericsson CCD to mention the most important ones.

Since containers are like a floating VM’s so networking architecture specially the one promised by L3 CNI plugin and Flannel is important direction to be supported in Platforms as it is supported in OSM .

The reusability of API makes it simple for application to craft unique application in form a build configuration files using artifacts of PoD, services, cluster, config map and persistent volumes which are defined in a very standard manner in K8S by which I mean deploy all artifacts through a single file.

ETSI® OSM can be deployed using both HELM2.0 as well as Juju charmed bundles

Picture2

Foundation for Helm

Helm gives teams the tools they need to collaborate when creating, installing, and managing applications inside of Kubernetes. With Helm, you can… Find prepackaged software (charts) to install and use Easily create and host your own packages , Install packages into any Kubernetes cluster Query the cluster to see what packages are installed and running Update, delete, rollback, or view the history of installed packages Helm makes it easy to run applications inside Kubernetes. For details please refer to details HELM packages on https://helm.sh/blog/helm-3-released/

In a nut shell all day1 and day2 tasks required for the CNF’s are made possible using Helm and its artifacts known as Helm charts including application primitives, network connectivity and configuration capabilities.

Key Features of OSM Release7

OSM Release 7 is a carrier grade and below are its key features as per wiki

  • Improved VNF Configuration interface (One stop shop) for all Day0/1/2 operations
  • Improved Grafana dashboard
  • VNFD and NSD testing
  • Python3 support
  • CNF’s support in both options where OSM creates the Cluster or rely on OEM tools to provision it
  • Workload placement and optimization (Something very important for Edge and Remote clouds)
  • Enhancement in both Multi VIM and Multi SDN support
  • Support for Public Clouds

How OSM handles deployment of CNF’s

For most Telco guys this is most important question e.g how VNF package will be standardized with arrival of CNF’s , Will it mean a totally new Package or enhancement of existing.

Fortunately, OSM approach on this is modeling of Application in a standard fashion which means same package can be enhanced to reflect containerized deployment. On a NS level it can flexibly interwork with VNF/PNF as well, the deployment unit used to model CNF specific parameters is called KDU’s (Kubernetes Deployment Unit) other major change is K8S cluster under resources. It is important as it explains most important piece the Networking and related CNI interfaces.

OSM can deploy the K8S cluster using API integration or rely on 3rd party tools like Openshift® or PKS deploy it on instructions of OSM

Picture7Changes to NFVO interfaces

Just like Or-Vi is used for infrastructure integration with Orchestration the Helm2.0 (Will support 3.0 in near future) is used for infrastructure integration with K8S applications. Since the NBI supports mapping of KDU’s in same NSD it means only changes from orchestration point of view is on the south side only.

Workload Placement

As per latest industry standing and experience sharing in Kubecon and Cloud Native  summit Americas  there is a growing consensus that Container is the platform of choice for the Edge primarily due to its robustness , operational model and lighter foot print . As per our experience of containers here in STC a 40% reduction in both CAPEX and Foot print will be realized on DC’s if deployed Edge using Containers.

However, definition of business definition of Edge raise number of queries the most important of it are work load identification, placement and migration specially consider the fact the Edge is a lighter foot print that in future will host carrier mission critical applications.

Optimization of Edge from CSP perspective has to address following  Cost of compute in NFVI PoP’s , Cost of connectivity and VNFDFG something implemented by SFC’s and Constraints on service like SLA, KPI and Slicing

Picture3

The issues with the Upgrades and How OSM addresses

Compared to early release the OSM ns action primitives allow the CNF to be upgrades to the latest release and execute both dryrun and  Juju tests to ensure the application performance bench mark is same like before  .Although this works best for small applications like LDAP the same is difficult to achieve with more complex CNF’s like 5G SA . Through liaison with LFN OVP program I am sure soon the issue will be addressed. We as operator have a plan to validate it on a 5G SA nodes.

Picture4

My final thoughts on this that  Container journey for CSP is already a reality and coming very shortly in 2020+ and OSM ecosystem supports the commercialization of CNF’s through early use cases of 5G SA , Enterprise branch uCPE and most important Edge including MEC for which OSM seems to reach maturity  For details and how to participate and use do get involved in upcoming OSM Hackfest IN MARCH OSM-MR8 Hackfest

Many thanks to colleague , mentor and industry collaborator Jose Miguel Guzman , Francisco Javier Ramón Salguero  Gerardo García and Andy Reid for OSM growth in recent years … See you in Madrid

Picture5

References:

ETSI

Linux Foundation

OVP

Delivering 5 9’s Security for Mission Critical 5G Systems

1

“Can an Open Cloud Based System be more secure for Mission Critical Applications”

2

So finally the Frenzy of 5G Networks and how they will bridge the gaps between different industries and societies seems finally come to materialization .As most of the Tier1 Operators are working to build the Use cases that will support for early launch and market capture catalyst for early movers in the area still the area of 5G security seems gloomy with still lacking much detailed standards being output by ETSI and other SDO’s compared to 5G technology itself.

There are many questions in the air need to address both from architecture point of view and from End to End working solution perspective. For example

1.     Is 5G security same or conflicting with NV/SDN security?

2.     How operators will develop a unified solution that can meet requirements from all industries

3.     If a standard solution exist will it scale? Or finally in 2-3 Years down the road we need to live with lot of customized solution difficult to assure?

4.     What about solution relevance in Open source networks with many players around

5.     Finally how to imbue Cyber security dilemmas in the 5G Telco Networks.

6.     Will End user privacy will be a killer decision in 5G

I think this list gives author enough challenges faced by 5G and verticals and in this paper I shall try to build a high level model to address them in a unified UML model.

  In a world where computing is ubiquitous, where a mist of data and devices diffuses into our lives, where that mist becomes inseparable— indistinguishable—from reality, trustworthy computing is but axiomatic. ( David James Marcos /NSA)

Before dig deep to formulate the security architecture we should know at a high level the 5G system security will no longer be like 4G networks because of reason no single domain like traditionally Core/UE can promise complete security solution . The enigma of 5G security is huge involving devices like Malware , MitM, low cost devices , Air interface jamming , frequency scan , Back haul DDOS , packet sniffing , NFV and virtualization vulnerabilities , API issues , NW security , VNF application ,platform and IP vulnerabilities and hence we should analyze 5G system in depth from whole system aspect and need look in following important dimensions

1.     Decentralized Architecture: The biggest problem that lies ahead is that the Telco Networks are programmed to work not the way around. It actually means they do not predict and obviously do not interpolate to the scale of issues 5G will go to face. This is an architecture issue because like in 3G/4G source of security seems like in Core Network, in NFV/SDN it seem to imbue in the platform but for 5G planning a single control unit to handle and process all data seems impossible. But if we decentralize how to control it. We cannot decentralize without control it and how to control a device we do not trust? I think 5G must model a concept like Block Chain in Banking sector to share security but in a trusted manner and in addition not point of failure due to compromise in a unit or layer

The understanding of 5G System architecture and how it will influence the present Telco Services migration along with how it can make a thriving eco system is key area of interest for the architect. There are different dimensions like first we need to understand 5G is based on a SBA architecture which requires whole network separated from Infrastructure which makes NFV/SDN almost an inevitable enabler for it . It will allow the deployment of network a slice to support each use case separately. Currently how to model one solution and can it be applicable to customize it for each offering is key area of discussion in ETSI.

3

 

  2.     Resource demarcation: This is a scary topic because IMT2000 already divided network in three domains as per latency use case requirement. The dilemma is that it require different RF resource need to map to a different NFV/SDN DC resource in the Cloud is biggest problem that lies ahead is that the Telco Networks are programmed to work not the way around. It actually means they do not predict and obviously do not interpolate to the scale of issues 5G will go to face. This is an architecture issue because like in 3G/4G source of security seems like in Core Network, in NFV/SDN it seem to imbue in the platform but for 5G planning , so in a broad sense multi RAT for each slice may not be the right approach

3.     5G Network Threat Model extension: This host VNF’s which are source or sunk of user workload like DNS , AAA ,IPAM is east use case but introducing middle Box VNF like AS , Control plan and Media boxes means we need to introduce Telco Concepts like multi homing , A/S architectures , CSLB and on top of it complex dependence on IT Network redundancy like Bonds ,bridges and it makes the Security a big issue of concern . Obviously introducing a disparate solution means security threat boundary will extend than it is originally supposed to be

4

4.     5G Security Frame work for 5G SA System: Well I will not go in to the details here because an expert buddy has just done it perfectly watch Hitchhikers guide here https://www.linkedin.com/pulse/hitchhikers-guide-5g-security-special-edition-junny-song/

However I do want summarize a bit as follows the 5G Rel15 specifications consider EN-DC (E-UTRAN New Radio Dual Connectivity) as the defacto standard for 5G security at least in 2018 or let’s say till H1 2019 reason is obvious because the final Standalone Security specification TS33.501 will freeze in Dec ,2018 http://www.tech-invite.com/3m33/tinv-3gpp-33-501.html#toc . Why EN-DC security is important but same time not very difficult to embrace is that The EN-DC security is based on the existing LTE security specification, TS 33.401 with EN-DC enhancement as shown below

5

http://www.3gpp.org

The Good news about EN-DC is that it works almost the same way the LTE-DC runs the concepts of Key Generation, Key Management, Ciphering and Integrity Protection are re-used from LTE –DC concept TS23.501 while the DRB <Data Radio Bearer Security> context is added with regard to 5G Core Network. For EN-DC security, new X2 Information Elements, “SgNB security Key” and “UE Security Capabilities” is newly defined.

6

Here shows EN-DC bearers and PDCP termination points from Network side. MN is the master eNB and SN is the secondary gNB. If the PDCP/NR-PDCP is terminated in the MN, LTE security works, on the other hand, if the NR-PDCP is terminated in the SgNB, NR security covers. EEA is redefined as NEA, EIA is also now called NIA. As you can guess NEA, NIA stands for NR Encryption Algorithm and NR Integrity Algorithm

A good analysis of 5G security protocol can be seen in below https://www.ethz.ch/content/dam/ethz/special-interest/infk/inst-infsec/information-security-group-dam/research/software/5G_lanzenberger.pdf

7

•       In 2018 implement EN-DC architecture almost same as LTE DC

•       Use existing USIM but program USIM/UICC it need USIM vendor support

•       5G Success depend on e-SIM trial special for IoT

5.     Assuring NFV/SDN security for 5G: 5G Network is not about a network but about a system. It involves a plethora of NFV, SDN and Network automation in context of Enablers for 5G to support the future SBA based architecture. These days biggest question we have been talking in the ETSI ISG Security group and in TMforum is actually do Network automation a bliss or curse for security assurance.

6.     Scalable Security solution : Historically the Telco companies and 3GPP must be credited of building a robust security architecture , it can be reflected in 2G/3G/4G and same is expected in 5G with only problem that scale of 5G devices is billions not millions and a solution to expand only Core network and related Authentication servers is not enough . It require inclusion of distributed security architectures and above all IAM solutions which best use network API exposure to guarantee security. It means in future Security as a service can be possible and that an operator can open the network to guarantee whole system security using best offering from the third party. Anyways it will not change 5G Security Frame work for 5G SA System as I explained in Point5 of this paper.

The scalable solution also means that security can be provisioned for each use case in an orchestrated manner something very similar like VNF OLM management where security policy, test criteria all can be customizable as per required use case and SLA.

7.     Security assessment and Verification: The 5G system is complex and include plethora of many technologies. The security context of IT , Cyber , Information security all are added along with the Telco security but till now even ETSI SA3 have not finalized the detailed scenario

The 5G System is big and complex , the 3GPP SA3 is doing a remarkable work to get the standard readiness and proto type before Rel-16 Stage-3 specs are output in June this year . The main focus of this year SA3 key targets are 1. Key hierarchy 2. Key derivation 3. Mobility 4. Access Stratum security 5. Non-Access Stratum security 6. Security context 7. Visibility and Configuration 8. Primary authentication 9. Secondary authentication 10. Inter working 11. non-3GPP access 12. Network Domain Security 13. Service based architecture 14. Privacy . I hope to refresh the material for whole 5G security by the time i got more visibility based on SA3 work and till the time got more inputs from vendors of exactly how they will be approaching this critical but important point in 5G .

References

National Security Agency review of Emerging Technologies

3GPP TR.501

3GPP TS28.891

3GPP TS 23.799

3GPP TS28.531

3GPP TS38.300

NFV EVE 011

NFV SOL03 ,04

 

Addressing Solvency of Open source production and adoption models for TMT and Telco industry

1.png

Addressing Solvency of  Open source production and adoption models for TMT and Telco industry

An architect’s perspective

2.png

Source: http://www.cncf.io

Technology, Media and Telecom industry known as TMT by commoners is going through major transformation programs globally. One of the prime recipes of this revolution are their vigor and participation in Open source. However, over years of experience in Open source revealed some key points to us, For example most of industry believes opensource is

  • Open (without defining what is Open)
  • Cheaper (with our a valid Business case and TCO working)
  • Simple to deploy and use (with out analyzing ecosystem and interworking).

On one hand I believe Opensource is a bandwagon every one wants to sit but is also a wagon no one want to drive at least in a commercial and production environment. Therefore, it is very important to share my views on this .

Understanding the Story of TMT Industry at a Glance 

Pictures are always an easy way to summarize and let me share some very useful insights from lumina networks  in recent #ONS Days Melbourne showing Opensource will address solve business issues around . Thanks for IldikoVancs for sharing this summary .

  • Automate any network with any form factor
  • Resource optimization and utilization
  • Offer NaaS and Slicing to Verticals most important 

Attracting business to Opensource is the most important challenge for Architects , Will building a best of breed staff fulfills TCO and CXO transformation objectives ? 

2Capture

Source: @lunminanetworks

https://twitter.com/IldikoVancsa/status/1193684616869113856/

Let us see now results of applying story to practice through Impacts of Open source Glory in TMT industry

Use of open source in Telco has definitely a long-term value but because still as a community we have failed to transfer Open source frameworks to MVP of vendor products, some thing that can drive and navigate our Sourcing, RFX and vendor selection process. I think this was something that was supposed to be done at the outset of this journey but what really happened. I think some smart marketing crooks sold dreams to Telco’s that they will become future Google and hyper scales which was never the purpose and direction of a service industry like ours.

The long term repercussions of this was that still down the road of almost a decade we found ourselves surrounded by vendors that come with a proprietary and engineered solutions  that they say is aligned with Opensource like ONAP , OPENSTACK,K8S etc but the sad story reveals its just the concept or in SDK terms the front end that is something like Open while all backend and software forking is proprietary .

So despite I lead my company in many of these initiatives I am still circumspect of the approach we have taken or at least we should agree that we need to really now think about value quantification to put business metrics in perspective and to ponder how to really make these initiatives work.

Grow adoption through power of synergy

3

Openstack is one of the most successful example of Opensource adoption and summing our experiences in Cloud may support to build the best prospective in other domains, may it be technical issues , licensing models or operators and TMT’s transformation journeys . The wide adoption of Openstack in IT , application, Telco and verticals proved that opensource frameworks that can serve a wide use cases not limited to particular industry is the best way to proceed. It has really enabled to adjoin industries which were traditional on a minimum of an Isle’s distance.

However balancing an opensource adoption in a balanced way with out skewness towards a particular segment is still a big challenge, it comes a life line when we consider each industry requires unique characteristics and building something for commoners cannot fulfill business needs.  This is what leads industry to collaboration or which in business or RFX we name as partnership and ecosystems.

Sharing and not swallowing

As Johanne Mayer, Director MayerConsult and TM Forum Distinguished Fellow at the Layer123 Network Transformation Congress in San Jose the Telco’s in 5G era can not find a solution that comes from a single vendor. It simply is not possible as more and more verticals will come in to the picture. The success of such initiatives will depend more on working together then to get community outputs and build end to end solutions and products that simply are not aligned with open standards and concept of mixing best of the breed approach.

Integrating the Open source solutions

4

Open source is a journey of software and idea that future the solution will come from a hefty number of vendors. However, it is still a reality that at least till the decade Telco’s will run the hybrid networks including Cloud , PNF , VNF etc . So we need to solve these issues on API , standards and Fitness for use at the outset .

Also we need to see use of open source in customer X can be totally different than the use in vendor Y because the end product and business case will require some solution tailoring. The Personalization of each customer solution is very important.

RFX of Open source solutions

One of the bigger pain points in Open source solution adoption is to do with sourcing and RFX process. It is easier part for Telco to give requirements of an open solution but fact is which vendor will take end to end responsibility of third party in such multi-vendor offering.

Then how an Operator can be sure the vendor is not pushing its solution whereas ideal solution must be the Pull solution where based on requirements each component should be selected and then integrator to work on modalities of integrating them all . It is easier said than done however failure to adopt this direction means we find an integrator that can only work with certain vendors or at least can only take project if certain vendors on certain later are there. This is a frightening situation that will impede to slow adoption and commercialization of open source technology. If we address these points not only on technology level but also process and sourcing model then I am sure adoption will be as freely distributed as the Windows of today.

Its Services not the solutions

Some time ago I was reading a blog from NetApp  with a sentence that caught me in the moment

“You Can’t Change Your DNA – EMC Thinks Mainframe; Dell Thinks PC; NetApp Thinks Open Systems”

That is so true for the TMT and specifically the Telco industry , we must not be lured by story of openness and innovation that we forget our purpose. I think key to our survival in new era of digital transformation is fail fast and to deliver services .  Focusing on services and not on solution is so vital for Long term sustainability.

5

Source: https://www.dreamstime.com/

 When ROI of Open source solutions will realize

Although painful but reality is that no CSP unless the giants with big R&D arms have achieved real ROI or a fair visibility into it at least till now. One practical result of this is slow down strategy by many CSP’s and Telco adopted , however slowdown simply solves issue but delays it only .

Lately I have seen many Telco’s in EMEA to solve issue by breaking stack from the top or North side then to fix it from ground (NFVI upwards) however it a child’s cry unless we  at least agree and define clearly what is the definition of automation

The risk of industry failure to solve this issue is all major Telco’s are considering proprietary solutions of orchestration and automation to have a feel of something practical that can do something realistic on automation.

Then for real ROI we must agree on something which will finally scale , as an example spending X years while building two Data centers NFV stack we find once we will scale to lets say 10 the advantages of scale is simply not there. The issue of Open source introduction at the Edge will make this discussion more prevalent as simply an Operator can not afford to go to Edge with high setup cost, had open source solution not fix this side will obviously result in the slowdown of Edge as a whole.

Finally, just solving OSM and ONAP architectures with out ensuring vendors MVP align with it and that finally automation stacks can be built by combing best of breed from different vendors is and will remain a dream unless definition , reference architecture , reference implementation and testing/validation is not agreed and vendor’s are not enforced to apply this across their products . Although latest work on hackathons (LFN developer and testing forum)  have tried to address the same concern however still author believes it is not production ready . This is how LFN defines it and I think this is where standardization and SDO’s need catch vendors to align their MVP with architecture frameworks and standard .

“Co-hosted with GSMA, this LFN Developer & Testing Forum brings together developers across ONAP, CNTT, and OPNFV, with a special focus on VNF compliance and verification testing. As the principal event for LFN open source projects, the technical gathering provides a platform for community members to collaborate, plan future releases, conduct a plugfest, and perform interoperability, compliance, and integration testing across the full network stack, including NFVI, MANO, and VNFs”

 

 Future lies at the intersection of our path

6

With so many things happening in the industry around both open source development and testing and its commercialization, it is not surprising that many carriers especially those outside U.S and EU to follow a more conservative strategy

However my takeaway to CXO will be to ask technical team focus on bigger picture to know how and when the proprietary automation or Open source solution will converge to real open solution opening choices and offerings for the service providers , had this not addressed the Open spruce journey will remain a glory which every body want to talk about but a road no business wish to take finally .

 

My final thoughts on this topic is actually best described by a session on O-RAN by Orange in ONS Europe 2019 which is

“The Price to pay for Open source greater flexibility, innovation and openness is the complexity of test and integration” .

So finally, all will come to one line how to define MVP products, benchmark the solutions and about all develop a common test and integration model. It is clear that open source bring value but if we do not know how to deploy and fix the issue it becomes a nightmare nobody wish to keep. This is where SDO’s and Operator community needs to focus in 2020 . One such direction is the CNTT (Common NFVI Task Force) and CNCF TUG (Telecom User Group) which are expected to solve very issues highlighted above .

Currently we are working on the CNTT R2 which is expected to be a GA by JUNE 2020 . Similarly R2 of CNCF will combine the testing and validation of open source solutions to adress the issues that had been faced by TMT clients .

Capture

https://wiki.lfnetworking.org/display/LN/Common+NFVI+Telco+Task+Force+-+CNTT

2017 Road Ahead for the Maintenance Solutions in Converged Clouds and Hyper scale networks

 

We are in 2017 the 2016 was a bumpy year for the NFV/SDN industry realization .Together we have come far to enter from Concept phase to commercial realization phase, it seems the Clouds are mature now and All large Opco’s and small enterprises alike are happy to achieve the real advantages from cloud which is Scalar, High performance and Modular but the story is not over.

The 2017 business model of all large Tier1 Operators is circumvented by two main concepts, (1) The first one is the converged clouds and (2) The later one is the Co-Cloud.

This is why for their Clouds they are looking for FCAPS and Maintenance solutions that are scalar in a vertical agnostic fashion

To draft a business solution for FCAPS and Unified Maintenance in this scenario is very complex because traditionally each of the solution requires a tenant based analysis, processing and output which can be further customized based on RBAC functions .The task becomes more complex as we solve the myth and find that Industry leading cloud vendors have given less and not enough importance to this complex task and demarcate FCAPS boundary as the Open Stack Boundary itself which is not reasonable after all the industry is evolving from PNF to VNF and lot of points need to integrate well to replicate same or better maintenance functions of IT and Telco converged Network

There are two schools of thoughts prevailing in industry till date as per the OpenStack latest standing as per Newton Release and SDN Maruno release .

School 1: Uses the Silows and offer separate solution on Physical and Cloud/Virtualization layer (Huawei esight and Fusion Manager together make it place in this slot)

School 2: Uses the Cloud OpenStack services to unify the relation and offer complete aggregated FCAPS from single Pane of services of Telemetry like Ceilometers .The idea is good but practically till date it is a naïve idea gained less attention in the industry. In fact till now the clear mapping between two layers no IT vendor can clearly explain. In fact within the community there have been discussion on this point with no clear solution to help IT identify and solve physical alarms easily using Cloud services (Huawei System Integration Services is key to customize solution in this part)

Customer in converged environments require real time Push solutions based on API’s for key metric like Current Status of DC ,Resource usage , Resource Utilization , Tenant based metrics , Log analysis

NFV industry Defined as “monitoring as a service,” Monasca in a large framework that incorporates all aspects of monitoring, including alarms, statistics, measurements and more, for all OpenStack components. Monasca largely focuses on allowing tenants to define what should be measured, what statistics should be collected, what should trigger an alarm, and the best notification method. The relationship between Monasca and Ceilometers is currently being defined.

However still Monasca is a very vague idea , since last MWC Barcelona the Community is making many efforts in the Doctor Project and most of this effort try to replicate the existing deployed tools in the IT to the Converged Datacenters like Zabbix , Nagios , Zenoss however the fact is that they can not offer the complete solutions , this is why the industry now have a consensus to unify the Tools upstream like all of above three can be aggregated in a Open Source Project like Sensu , Similarly for Perforamcne Collectd and for logs Fluentd and the list will go on like this . https://wiki.openstack.org/wiki/Operations/Tools

The point is that in the current wave of NFV and SDN Projects the CIO sees to reduce so many tools to list of 3-5 to control the support, bug fixes and other issues that seem to be really big issues in a commercial company. I think community is doing best but still without the summation of these tools it seems difficult.

The Next wave as I see is the Unification , last month I have met couple of senior IT executives and they all have a believe with Organizations following a digital transformation path , they think there is a way to collaborate and unify many of the information in a Single Pane form operations perspective . It will open a Funnel pipe for accurate management of the cloud in terms of correlation and root cause of operational nature and obviously can build the Big IT intelligence mechanisms for a future programmable way to do things.

We need to a consensus a seamless integration and unification is a key for programmable network otherwise micro pieces even programmed in an excellent fashion cannot deliver the promise of NFV and SDN.

Just to give an idea below picture is from https://www.sdxcentral.com/ blog on SDN main components with the inclusion of more and more open source components it still looks very curious how the issues will be consolidated and solved

???????????????????

At Huawei I believe we are almost there close to community, industry, customers and market alike to build Tribalizer for Wave1 and now we need to be more open and cooperative to form alliances with other vendors and customers alike to find the Open Solutions for industry issues of how to actually achieve the Wave2?

https://www.linkedin.com/pulse/api/edit/embed?embed=%257B%2522type%2522%253A%2522link%2522%252C%2522title%2522%253A%257B%2522localized%2522%253A%257B%2522en_US%2522%253A%2522The%2520components%2520of%2520SDN%252C%2520NFV%252C%2520MANO%2520%2526%2520OSS%2522%257D%257D%252C%2522description%2522%253A%257B%2522localized%2522%253A%257B%2522en_US%2522%253A%2522When%2520you%2520curate%2520a%2520website%2520%252F%2520blog%2520called%2520PassionateAboutOSS%252C%2520it%2527s%2520pretty%2520obvious%2520that%2520you%2520would%2520see%2520the%2520world%2520through%2520OSS-tinted%2520glasses.%2520The%2520tech-world%2520at%2520least.%2520And%2520so%2520it%2520has%2520been%2520for%2520me%2520when%2520evaluating%2520new%2520comms%2520network%2520%252F%2520service%2520concepts%2520like%2520SDN%2520%252F%2520NFV.%2522%257D%257D%252C%2522author%2522%253A%257B%2522name%2522%253A%2522Ryan%2522%257D%252C%2522provider%2522%253A%257B%2522name%2522%253A%2522Passionate%2520About%2520OSS%2520and%2520BSS%2522%252C%2522display%2522%253A%2522Passionate%2520About%2520OSS%2520and%2520BSS%2522%252C%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%2522%257D%252C%2522request%2522%253A%257B%2522originalUrl%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fthe-components-of-sdn-nfv-mano-oss%252F%2522%252C%2522finalUrl%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fthe-components-of-sdn-nfv-mano-oss%252F%2522%257D%252C%2522images%2522%253A%255B%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fwp-content%252Fuploads%252F2014%252F06%252FSDN_NFV_Drawing_v2.png%2522%252C%2522width%2522%253A1096%252C%2522height%2522%253A632%257D%252C%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fwp-content%252Fuploads%252F2017%252F01%252FECOMP.png%2522%252C%2522width%2522%253A602%252C%2522height%2522%253A538%257D%252C%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fwp-content%252Fuploads%252F2014%252F05%252FMasteringYourOSS_CoverImage.jpg%2522%252C%2522width%2522%253A400%252C%2522height%2522%253A500%257D%252C%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fwp-content%252Fuploads%252F2015%252F06%252FOSS_LandscapeTechSolnOrg_June2015.jpg%2522%252C%2522width%2522%253A170%252C%2522height%2522%253A220%257D%252C%257B%2522url%2522%253A%2522http%253A%252F%252F1.gravatar.com%252Favatar%252Fa564e905c3c15fc7268bd86149e6626b%253Fs%253D68%2526d%253Dmm%2526r%253Dpg%2522%252C%2522width%2522%253A68%252C%2522height%2522%253A68%257D%255D%252C%2522data%2522%253A%257B%2522com.linkedin.treasury.Link%2522%253A%257B%2522url%2522%253A%2522http%253A%252F%252Fpassionateaboutoss.com%252Fthe-components-of-sdn-nfv-mano-oss%252F%2522%252C%2522html%2522%253A%2522When%2520you%2520curate%2520a%2520website%2520%252F%2520blog%2520called%2520PassionateAboutOSS%252C%2520it%2527s%2520pretty%2520obvious%2520that%2520you%2520would%2520see%2520the%2520world%2520through%2520OSS-tinted%2520glasses.%2520The%2520tech-world%2520at%2520least.%2520And%2520so%2520it%2520has%2520been%2520for%2520me%2520when%2520evaluating%2520new%2520comms%2520network%2520%252F%2520service%2520concepts%2520like%2520SDN%2520%252F%2520NFV.%2522%252C%2522width%2522%253A-1%252C%2522height%2522%253A-1%257D%257D%257D&signature=AabES296V41K11DCMgJo9iqC1zcB

The Technical task is daunting but the direction is clear. Openness, alliances and sharing is key to success and together with our partners we want to build the Future Network of customers

References:

https://wiki.openstack.org/wiki/Operations/Tools

https://sensuapp.org/docs/0.26/overview/architecture.html

http://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_aggregation

http://www.fluentd.org/

https://github.com/centos-opstools/opstools-ansible