Skip to content
This repository has been archived by the owner on Nov 19, 2024. It is now read-only.

Latest commit

 

History

History
214 lines (176 loc) · 6.84 KB

vm-poweroff.md

File metadata and controls

214 lines (176 loc) · 6.84 KB
id title sidebar_label
vm-poweroff
VM PowerOff Experiment Details
VM PowerOff

Experiment Metadata

Type Description Tested K8s Platform
VMWare PowerOff the VMWare VM for a certain chaos duration Kubeadm cluster on VMware

Prerequisites

  • Ensure that Kubernetes Version > 1.16
  • Ensure that Vcenter Version is 6.X (Support for 7.X will be added soon)
  • Ensure that the Litmus Chaos Operator is running by executing kubectl get pods in operator namespace (typically, litmus). If not, install from here
  • Ensure that the vm-poweroff experiment resource is available in the cluster by executing kubectl get chaosexperiments in the desired namespace If not, install from here
  • Ensure that you have sufficient Vcenter access to stop and start the vm.
  • (Optional) Ensure to create a Kubernetes secret having the Vcenter credentials in the CHAOS_NAMESPACE. A sample secret file looks like:
apiVersion: v1
kind: Secret
metadata:
  name: vcenter-secret
  namespace: litmus
type: Opaque
stringData:
    VCENTERSERVER: XXXXXXXXXXX
    VCENTERUSER: XXXXXXXXXXXXX
    VCENTERPASS: XXXXXXXXXXXXX

Note: You can pass the VM credentials as secrets or as an chaosengine ENV variable.

Entry-Criteria

  • VM instance is healthy before chaos injection.

Exit-Criteria

  • VM instance is healthy post chaos injection.

Details

  • Causes Stops/PowerOff a VM before bringing it back to running state after a specified chaos duration
  • Experiment uses vmware api's to start/stop the target vm.
  • It helps to check the performance of the application/process running on the vmware server.

Integrations

  • VMPoweroff can be effected using the chaos library: litmus, which makes use of api calls to start/stop the vm instance.
  • The desired chaoslib can be selected by setting the above options as value for the env variable LIB

Steps to Execute the Chaos Experiment

  • This Chaos Experiment can be triggered by creating a ChaosEngine resource on the cluster. To understand the values to provide in a ChaosEngine specification, refer Getting Started

  • Follow the steps in the sections below to create the chaosServiceAccount, prepare the ChaosEngine & execute the experiment.

Prepare chaosServiceAccount

  • Use this sample RBAC manifest to create a chaosServiceAccount in the desired (app) namespace. This example consists of the minimum necessary role permissions to execute the experiment.

Sample Rbac Manifest

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: vm-poweroff-sa
  namespace: default
  labels:
    name: vm-poweroff-sa
    app.kubernetes.io/part-of: litmus
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: vm-poweroff-sa
  labels:
    name: vm-poweroff-sa
    app.kubernetes.io/part-of: litmus
rules:
- apiGroups: [""]
  resources: ["pods","events","secrets"]
  verbs: ["create","list","get","patch","update","delete","deletecollection"]
- apiGroups: [""]
  resources: ["pods/exec","pods/log"]
  verbs: ["create","list","get"]
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["create","list","get","delete","deletecollection"]
- apiGroups: ["litmuschaos.io"]
  resources: ["chaosengines","chaosexperiments","chaosresults"]
  verbs: ["create","list","get","patch","update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: vm-poweroff-sa
  labels:
    name: vm-poweroff-sa
    app.kubernetes.io/part-of: litmus
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: vm-poweroff-sa
subjects:
- kind: ServiceAccount
  name: vm-poweroff-sa
  namespace: default

Prepare ChaosEngine

  • Provide the application info in spec.appinfo. It is an optional parameter for infra level experiment.
  • Provide the auxiliary applications info (ns & labels) in spec.auxiliaryAppInfo
  • Override the experiment tunables if desired in experiments.spec.components.env
  • To understand the values to provided in a ChaosEngine specification, refer ChaosEngine Concepts

Supported Experiment Tunables

Variables Description Specify In ChaosEngine Notes How to get
APP_VM_MOID Moid of the vmware instance Required Once you open VM in vCenter WebClient, you can find MOID in address field (VirtualMachine:vm-5365). Eg: vm-5365
TOTAL_CHAOS_DURATION The time duration for chaos insertion (sec) Optional Defaults to 30s
INSTANCE_ID A user-defined string that holds metadata/info about current run/instance of chaos. Ex: 04-05-2020-9-00. This string is appended as suffix in the chaosresult CR name. Optional Ensure that the overall length of the chaosresult CR is still < 64 characters

Sample ChaosEngine Manifest

apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: nginx-chaos
  namespace: default
spec:
  # It can be active/stop
  engineState: 'active'
  #ex. values: ns1:name=percona,ns2:run=nginx
  auxiliaryAppInfo: ''
  chaosServiceAccount: vm-poweroff-sa
  experiments:
    - name: vm-poweroff
      spec:
        components:
          env:
            # set chaos duration (in sec) as desired
            - name: TOTAL_CHAOS_DURATION
              value: '30'

            # provide vm moid
            - name: APP_VM_MOID
              value: ''

Create the ChaosEngine Resource

  • Create the ChaosEngine manifest prepared in the previous step to trigger the Chaos.

    kubectl apply -f chaosengine.yml

  • If the chaos experiment is not executed, refer to the troubleshooting section to identify the root cause and fix the issues.

Watch Chaos progress

  • You can use vcenter console to keep a watch over the instance state.

Check Chaos Experiment Result

  • Check whether the application is resilient to the vm-poweroff, once the experiment (job) is completed. The ChaosResult resource name is derived like this: <ChaosEngine-Name>-<ChaosExperiment-Name>.

    kubectl describe chaosresult nginx-chaos-vm-poweroff -n <application-namespace>