First steps with Apache Spark on K8s (part #3): GCP Spark Operator and auto scaling

#1.helm repo add spark-operator https://googlecloudplatform.github.io/spark-on-k8s-operatorhelm install gcp-spark-operator spark-operator/spark-operator
#2.kubectl get deployments
kubectl get po
#3.git clone https://github.com/domasjautakis/spark-operator-gcp
#4.kubectl apply -f spark-operator-gcp/1.rbac.yaml
kubectl apply -f spark-operator-gcp/2.spark-app-gcpoperator.yaml

Horizontal auto scaling

#5.more spark-operator-gcp/3.spark-app-gcpoperator-scaled.yaml
#6.kubectl apply -f spark-operator-gcp/3.spark-app-gcpoperator-scaled.yaml
kubectl get po
#7.kubectl apply -f spark-operator-gcp/4.spark-app-gcpoperator-scaled.yaml
kubectl get po

Vertical auto scaling

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store