OpenYurt Self-assessment

This assessment was created by community members as part of the Security Pals process and is currently pending changes from the maintainer team.

Authors: Zeyu Zhao (@vie-serendipity), Lu Chen(@luc99hen)

Contributors/Reviewers: Linbo He (@rambohe-ch), Bingchang Tang (@zyjhtangtang)

The Self-assessment is the initial document for projects to begin thinking about the security of the project, determining gaps in their security, and preparing any security documentation for their users.

Table of contents

• Metadata
  • Security Links
• Overview
• Background
• Actors
• Actions
• Goals
• Non-goals
• Self-assessment use
• Security functions and features
• Project compliance
• Secure development practices
• Security issue resolution
• Appendix

Metadata

Security Links

Overview

OpenYurt is an open-source initiative built upon Kubernetes, dedicated to fostering seamless collaboration between cloud and edge computing paradigms. It tackles distinctive hurdles in cloud-edge orchestration within the Kubernetes ecosystem, encompassing unreliable or intermittent cloud-edge networking connectivity, edge autonomy, edge device management, region-aware deployment, and various related intricacies.

Background

In the contemporary landscape, a significant portion of computational power is decentralized, dispersed across various edge computing platforms and IoT devices in diverse forms. These distributed, heterogeneous resources often incur substantial human capital expenditure.

Kubernetes is an open-source system for automating the deployment, scaling, and management of containerized applications, which obscures underlying heterogeneous computing resources and dramatically mitigates management complexity. Nevertheless, due to the distinctive challenges inherent in cloud-edge scenarios, such as unstable cloud-edge connectivity, disparities in multi-region application deployment, and inaccessible cloud-edge operational management, direct utilization of Kubernetes in edge environments may result in unstable edge-side applications and difficulties in troubleshooting.

By making non-intrusive enhancements, OpenYurt empowers customers to manage large-scale edge computing workloads in different architectures (e.g., ARM and X86) in a native Kubernetes manner.

Actors

YurtManager

The yurt-manager consists of various controller and webhook processes to provide desired functionality in a cloud-edge collaboration scenario. The yurt-manager is deployed in kubernetes control plane, usually consists of two instances, one leader and one backup.

YurtHub

YurtHub runs as a systemd service on the node, mainly responsible for proxying requests from pods and kubelet on the node, enabling node autonomy, multiplexing traffic on the node, and managing network topology.

Raven

Raven enhances network capabilities. It provides L3 and L7 network connectivity among pods in different physical regions. The Raven server, functioning as a controller within the YurtManager, is deployed in the cloud, while Raven agents are distributed as DaemonSets across edge nodes.

YurtAdm

YurtAdm is CLI mainly to provide the ability of joining edge nodes to an OpenYurt cluster.

YurtIotDock

Yurt-iot-dock enables seamless integration of EdgeX Foundry into cloud-native architecture for non-intrusive fusion in edge IoT solutions.

Actions

Edge Autonomy

1. Users utilize YurtAdm to join a node to an existing cluster, initiating a CSR request using a bootstrap token.
2. YurtManager validates and subsequently approves the CSR.
3. The YurtHub acquires the certificate from the CSR, successfully joining the cluster.
4. YurtHub functions as a proxy for system components on the node, facilitating communication with the apiserver while caching responses locally.
5. Upon cloud-edge network disconnection, YurtHub utilizes its local cache to serve as a server, responding to requests from kubelet and other system components.

Cloud-Edge Operations

1. When users use kubectl exec or logs commands, requests are relayed from the apiserver to the Raven service.
2. The Raven server establishes a secure tunnel with the Raven agent residing on the elected gateway node at the edge. This tunnel employs a dual-layer encryption approach: the L7 utilizes gRPC with mutual TLS, while L3 is safeguarded through IPsec protocols.
3. The Raven agent on the node invokes kubelet to retrieve and return the requested information.

IoT device management

1. When users manage edge devices using Yurt-IoT-Dock , they create or update a custom resource called Device in the OpenYurt cluster.
2. After Yurt-IoT-Dock lists/watches changes to the Device, it generates corresponding EdgeX commands to control the device’s status.
3. Yurt-IoT-Dock retrieves the device status in real-time from EdgeX and synchronizes this status with the corresponding Device status.
4. Users can obtain the actual device status by querying the Device in the OpenYurt cluster.

Goals

General

• Edge Autonomy: Ensures stable application operation on edge nodes during unreliable cloud-edge network conditions.
• Cloud-Edge Operations: Provides operational channels for managing edge nodes efficiently.
• IoT Device Management: Integrates EdgeX to facilitate comprehensive management of IoT devices.
• Region-aware application deployment: customized deployment of multi-region applications.
• Traffic multiplexing: reuse traffic at both node and node-pool levels facilitates the integration of large-scale edge nodes.

Security

• OpenYurt components should be protected and robust against tampering
• Authenticating and authorizing access OpenYurt to control plane components
• Protect the OpenYurt control plane from being compromised

Non-goals

Self-assessment use

This self-assessment is created by the OpenYurt team to perform an internal analysis of the project’s security. It is not intended to provide a security audit of OpenYurt, or function as an independent assessment or attestation of OpenYurt’s security health.

This document serves to provide OpenYurt users with an initial understanding of OpenYurt’s security, where to find existing security documentation, OpenYurt plans for security, and general overview of OpenYurt security practices, both for development of OpenYurt as well as security of OpenYurt.

This document provides the CNCF TAG-Security with an initial understanding of OpenYurt to assist in a joint-assessment, necessary for projects under incubation. Taken together, this document and the joint-assessment serve as a cornerstone for if and when OpenYurt seeks graduation and is preparing for a security audit.

Security functions and features

Critical

• In cloud-edge orchestration scenarios, edge node is vulnerable to security attacks. OpenYurt’s design allows each node to use separate certificates and limited permissions, reducing the attack area.
• YurtManager enforces the separation of RBAC for different controllers and webhooks. Following the principle of least authority, one controller or webhook in YurtManager will not be granted permissions it will not use.
• Raven uses TLS and VPN protocols to secure network connections across domains. All cloud-edge and edge-edge container network communications are encrypted through two-way TLS certificates and VPN.
• Security scanning of code and image artifacts is integrated into the CI process to ensure that they are free of known vulnerabilities.

Security Relevant

• YurtAdm supports passing bootstrap file instead of join token for node bootstrapping. In this way, join token will not be exposed to the user environment.

Project compliance

OpenYurt does not document meeting particular compliance standards.

Secure development practices

• The OpenYurt project has clear contributing guidelines
• Anyone is encouraged to submit an issue, code, or documentation change
• Proposals should be submitted before making a significant change
• Decisions are made based on consensus between OpenYurt community . Proposals and ideas can either be submitted for agreement via a Github issue or PR.

Development Pipeline

The Contributing document contains details about development pipeline. The main points are summarized below.

• Contributions are made via GitHub pull requests
• Pull request will trigger a Github workflow including tests below
  • Type check and lint ci.
  • Trivy scan is adopted to scan vulnerabilities for every image.
  • Unit tests and e2e tests.
  • Automatic code coverage using codecov.io is generated in the PR for each submitted
• Code Review
  • Changes must be reviewed and merged by the project maintainers .
• Release
  • The release process of a new version of OpenYurt(involving changelog, documents, helm charts) is detailed in the release process document .

Communication Channels

Internal

• Team members use Github, Ding Talk , and OpenYurt Community Slack to communicate and discuss PRs.
• Community meetings are held biweekly on Wednesdays for team members to discuss releases, feature proposals, and user issues. The minutes and recording for each meeting are openly available.

Inbound

• Users contact with the OpenYurt team through the OpenYurt Community Slack and Ding Talk .

• Users can post bug reports, feature requests, and support requests on Github issues .

Outbound

• The team now uses OpenYurt Community Slack and Ding Talk to send community meeting reminders, meeting agendas, and project announcements to the community.

• Team members communicate with users through the emailing list .

Security issue resolution

OpenYurt has a responsible disclosure process for reporting security vulnerabilities. This process is designed to ensure that vulnerabilities are handled in a timely and effective manner. The process can be found here: https://github.com/openyurtio/openyurt/security/policy

• Security researchers can report vulnerabilities confidentially by emailing security@mail.openyurt.io .
• Security-related issues can be reported through GitHub issues at https://github.com/openyurtio/openyurt/issues
• The maintainers will triage the vulnerability and determine the appropriate remediation.
• Reporters can expect a response from OpenYurt project maintainers within 2 business days acknowledging receipt. It is the maintainers’ responsibility to triage the severity of issues and determine remediation plans.
• Remediation: OpenYurt commits to supporting the n-2 version minor version of the current major release; as well as the last minor version of the previous major release.
• Disclosures: OpenYurt encourages the community to assist in identifying security breaches; in the event of a confirmed breach, reporters will receive full credit and have the option to stay informed and kept in the loop.

Communication

GitHub Security Advisory will be used to communicate during the identification, fixing, and shipping of vulnerability mitigations.

The advisory becomes public only when the patched version is released to inform the community about the breach and its potential security impact.

Appendix

• Known Issues Over Time

OpenYurt doesn’t have any security vulnerabilities pointed out as of the tools and frameworks that it uses (for eg. Golang vulnerabilities).

• CII Best Practices The OpenYurt project has got the passing badge in OpenSSF best practices in PR #2208 .

• Case Studies Many organizations have adopted OpenYurt and are using our project

  • Alibaba Cloud: Using OpenYurt as base framework and integrated with other cloud services (like SLB etc.) to provide hosted edge kubernetes service.
  • Sangfor Technologies Inc: A company that focuses on providing security services, and uses OpenYurt for edge autonomy and operation and maintenance communication.
  • China Telecom: Using OpenYurt for managing edge nodes across different IDC region and network
  • Sony Group Corporation: Working on internal PoC for cloud and edge container orchestration.
  • Lixiang Auto Inc: A company that designs and produces new energy vehicles, and uses OpenYurt for managing edge nodes, and deploying edge applications.
  • Shanghai Cue Co., Ltd: Use OpenYurt to manage their retail edge AIBoxes.

• Related Projects / Vendors