Paul Bastide

Tag: etcd

  • etcdctl hacks

    If you are running etcd, and need to check a few thing / see the status of your cluster, use the included hacks.

    Check the Endpoint Status and DB Size

    If you want to see some key details for your cluster, you can run the etcdctl:

    $ etcdctl -w table endpoint status
+----------------------+------------------+-------+---------+-------+-------+---+---------+-----+--------+
|        ENDPOINT      |        ID        |VERSION| DB SIZE |LEADER |LEARNER|RT |RAFTINDEX|RAFT APPLIED INDEX | ERRORS |
+----------------------+------------------+-------+---------+-------+-------+---+---------+-----+--------+
| https://1.2.3.3:2379 | e97ca8fed9268702 | 3.5.3 |  128 MB | false | false | 8 | 2766616 | 2766616 |   |
| https://1.2.3.2:2379 | 82c75b78b63b558b | 3.5.3 |  127 MB |  true | false | 8 | 2766616 | 2766616 |   |
| https://1.2.3.1:2379 | afa5e0b54513b116 | 3.5.3 |  134 MB | false | false | 8 | 2766616 | 2766616 |   |
+----------------------+------------------+-------+---------+-------+-------+---+---------+-----+--------+

    Check the Revision and Count for All the Keys

    If you need to see how many keys you have, you can execute the following command, and you get 5061 keys.

     $ etcdctl get / --prefix --count-only=true --write-out=fields
"ClusterID" : 1232296676125618033
"MemberID" : 9423601319307597195
"Revision" : 2712993
"RaftTerm" : 8
"More" : false
"Count" : 5061

    Check the 5 Highest ModRevisions for a Key/Value

    If you need to find the Highest utilized (Updated), keys you can use this hack:

    $ for KEY in $(etcdctl get / --prefix --keys-only=true | grep -v leases)
do 
    if [ ! -z "${KEY}" ]
    then 
        COUNT=$(etcdctl get ${KEY} --prefix --write-out=fields | grep \"ModRevision\" | awk '{print $NF}')
        echo "${COUNT} ${KEY}"
    fi
done | sort -nr | head -n 5

2732087 /kubernetes.io/validatingwebhookconfigurations/performance-addon-operator
2731785 /kubernetes.io/resourcequotas/openshift-host-network/host-network-namespace-quotas
2731753 /kubernetes.io/validatingwebhookconfigurations/multus.openshift.io
2731549 /kubernetes.io/network.openshift.io/clusternetworks/default
2731478 /kubernetes.io/configmaps/openshift-service-ca/service-ca-controller-lock

    Calculating the Theoretical Memory Pressure

    Per the website, you can calculate memory pressure as:

    The theoretical memory consumption of watch can be approximated with the formula: memory = c1 * number_of_conn + c2 * avg_number_of_stream_per_conn + c3 * avg_number_of_watch_stream

    etcd benchmark site

    Command to be added in the future

    References