The configuration of the Cassandra image has been delegated to two init-containers that are executed prior to the Cassandra container when the pod is started.
There is a specific bootstrap image that is build from the docker directory and contains all required files or scripts to work with CassKop.
The init container is responsible for the following actions :
- copy the default Cassandra configuration from user's provided Cassandra image.
- The init-config container is by default the baseImage Cassandra image
- The default command executed by the init-container is:
This container is responsible for the following actions :
- Generate all the cassandra configuration (cassandra.yaml, java. etc..) from the different config entries in the CassandraCluster object level (spec, datacenters and racks)
It uses cass-config-builder which is a tool developed by Datastax.
The bootstrap Container :
- Adding files and additional jar from the bootstrap image to the default configuration
- Applying a configmap on top of the default configuration
- Modifying the configuration to be suitable to run with CassKop:
- Updating the cluster name
- Setting the seedlist
- Adding a Cassandra exporter and Jolokia agent
We provide the bootstrap image, but you can change it using
Spec.bootstrapImage but you need to comply with the
required actions, see Bootstrap.
One of the requirements for CassKop is to always keep the same number of nodes in each of its racks per Cassandra DCs. The number of nodes used for the Cassandra Cluster is configured using the
If you have not specify a Cluster Topology, then you'll have a default datacenter named
dc1and a default rack named
CassKop will keep the same number of nodes in each Cassandra rack as it is a good practice for Cassandra.
You can define different numbers of replicas for racks in different Cassandra DataCenters using the
nodesPerRacks property at the datacenter level. (in
specified on the datacenter level, this parameter takes priority over the global
Example: example to scale up the nodesPerRacks in DC2 :
The number of Cassandra nodes will be the multiplication of the number of racks * the nodesPerRacks value.
The configuration of the different components of Cassandra can be changed in the object and be overriden at different levels. Here is an example showing how to update some Java and Cassdnra parameters as well and overwrite them at the datacenter and rack level:
CassKop allows you to customize the configuration of Apache Cassandra nodes by specifying a dedicated
name in the
CassandraCluster.spec.configMapName containing configuration files to be overwritten above the default
configuration files of the dedicated Docker Image.
We have a specific Cassandra Docker image startup script that will overwrite each file in the directory
/etc/cassandra, from the one specified in the configMap if they exist.
You can surcharge any files in the docker image
/etc/cassandrawith the ConfigMap.
Typical overwriting files may be :
- specifying a pre_run.sh and/or post_run.sh script
See the example below:
You can create the ConfigMap above from the repo:
Now you can add the
configMapName: cassandra-configmap-v1 to the Spec section of your CassandraCluster definition
If you edit the ConfigMap it won't be detected neither by CassKop nor by the statefulsets/pods (unless you reboot the pods). It is recommended for configuration changes, to version the configmap and to create apply new configmap in the CRD, this will trigger a rollingRestart of the whole cluster applying the new configuration.
each time you specify a new configMap CassKop will start a
rollingUpdate of all nodes
in the cluster. more info on UpdateConfigMap
At this time CassKop won't allow you to specify only excerpt of the configurations files, your ConfigMap MUST contain valid and complete configuration files
In case you need to make some specific actions on a particular node, such as make use of the CASSANDRA_REPLACE_NODE
variable, you can use the pre_run.sh script in the ConfigMap. If present, the cassandra docker will execute this script
prior to the
run.sh script from the docker image.
Example of a configMap with the pre_run.sh script :
In the case you use the configmap for one-time specific action, don't forget to edit again to remove the specific treatment once it is no more needed.
Apache Cassandra is running inside a Java Virtual Machine (JVM). JVM has many configuration options to optimize the performance for different platforms and architectures.
CassKop allows configuring these values by adding a
jvm.options in the user
The default value used for
-Xmx depends on whether there is a memory request configured for the container :
If there is a memory request, the JVM's maximum memory must be set to a value corresponding to the limit.
When there is no memory request, CassKop will limit it to "2048M".
set the memory request and the memory limit to the same value, so that the pod is in guarantee mode
CassKop will automatically compute the initial and max heap size from 1/4 of the available defined resources.
CassKop uses Jolokia from the cassandra-image to communicate. We can add authentication on Jolokia by defining a secret :
and in the CRD you will define
CassKop will propagate the secrets in Cassandra so that it can configure Jolokia and use it to connect.