StarRocks Helm Chart 1.6.1 running on Red Hat OpenShift

[alberttwong]

This tutorial describes how you can get StarRocks running on Red Hat OpenShift or OKD. Thanks for contributions from @dengliu from Celonis and @piggyvenus from Red Hat.

Prerequisites

For this tutorial you need to:

• Have Red Hat OpenShift or OKD
• StarRocks Helm Chart

Login into Red Hat OpenShift or OKD

Logan into your OpenShift environment. For this tutorial, I'll be going through the UI to get access to "oc".

Once you access the "copy login command", you'll get the "oc login" command with token. An example is below.

oc login --token=sha256~3IDANu3xl0F2bYTUULQqaT5Tliq7Qlx2O1bZOFML4ks --server=https://api.cluster-c5plz.c5plz.sandbox506.opentlc.com:6443

Results will looks like

atwong@Alberts-MBP ~ % oc login --token=sha256~3IDANu3xl0F2bYTUULQqaT5Tliq7Qlx2O1bZOFML4ks --server=https://api.cluster-c5plz.c5plz.sandbox506.opentlc.com:6443
The server uses a certificate signed by an unknown authority.
You can bypass the certificate check, but any data you send to the server could be intercepted by others.
Use insecure connections? (y/n): yes

Logged into "https://api.cluster-c5plz.c5plz.sandbox506.opentlc.com:6443" as "opentlc-mgr" using the token provided.

You have access to 65 projects, the list has been suppressed. You can list all projects with 'oc projects'

Using project "default".

Deploy the StarRocks Helm Chart

We will be following the instructions listed at https://docs.starrocks.io/en-us/latest/deployment/helm#deploy-starrocks-with-helm

Note: StarRocks Operator does not currently work with OpenShift. See StarRocks/starrocks-kubernetes-operator#108

helm repo add starrocks-community https://starrocks.github.io/helm-charts
helm repo update

Results will look like

atwong@Alberts-MBP-2 sandbox % oc version
Client Version: 4.11.37
Kustomize Version: v4.5.4
Server Version: 4.10.36
Kubernetes Version: v1.23.5+8471591
atwong@Alberts-MBP-2 sandbox % helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "starrocks-community" chart repository
...Successfully got an update from the "stable" chart repository
...Successfully got an update from the "ververica" chart repository
Update Complete. ⎈Happy Helming!⎈
atwong@Alberts-MBP-2 sandbox % helm search repo starrocks-community
NAME                              	CHART VERSION	APP VERSION	DESCRIPTION
starrocks-community/kube-starrocks	1.6.1        	2.5.4      	kube-starrocks collects Kubernetes manifests, s...

Next we'll create the starrocks-sa service account and also apply "super user" permissions to the starrocks-sa service account. Note: Normally, you would not apply "super user" permissions. That is against K8S SCC (security context constraints) polices and K8S security best practices. There is an outstanding RFE: #22768 to fix this.

oc create sa starrocks-sa
oc adm policy add-scc-to-user anyuid -z starrocks-sa

Results will look like

Note: You'll see that you can interchange "oc" and "kubectl". "oc" is just an enhanced form of "kubectl" with additional commands/features. For basic "kubectl" commands, you can use either command to execute against OpenShift.

atwong@Alberts-MBP-2 sandbox % oc create sa starrocks-sa
serviceaccount/starrocks-sa created
atwong@Alberts-MBP-2 sandbox % kubectl get sa
NAME           SECRETS   AGE
builder        2         108m
default        2         115m
deployer       2         108m
starrocks-sa   2         13s
atwong@Alberts-MBP-2 sandbox % oc adm policy add-scc-to-user anyuid -z starrocks-sa
clusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "starrocks-sa"

Deploy the StarRocks cluster

Next we need to modify the values.yaml to use the correct service account. Download the default values.yaml and rename it to my-values.yaml. You'll need to modify 2 values in 3 different sections (for FE, BE, CN)

fsGroup: 1000
serviceAccount: "starrocks-sa"

Here's how my FE section looks:

Finally we deploy the Helm Chart

helm install -f my-values.yaml starrocks starrocks-community/kube-starrocks

Here's more output

atwong@Alberts-MBP-2 sandbox % helm install -f my-values.yaml starrocks starrocks-community/kube-starrocks
NAME: starrocks
LAST DEPLOYED: Fri Jun 30 16:11:11 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:

kube-starrocks has been installed, Check its status by running:
  kubectl --namespace default get starrockscluster -l "cluster=kube-starrocks"

Visit https://github.com/starrocks for instructions on how to create & configure.
atwong@Alberts-MBP-2 sandbox % kubectl get starrocksclusters.starrocks.com
NAME             FESTATUS   CNSTATUS   BESTATUS
kube-starrocks   running               running

Connect to the StarRocks Cluster from your workstation

There are two different ways to connect to the StarRocks Cluster from your workstation

Method 1: Use k8s port-forward
Method 2: Launch the StarRocks Toolkit in the same k8s namespace as your cluster

Each method has pros/cons. Method 1 allows you to use your own tools and you just connect remotely. Method 2 works for disconnected networks or secure networks (doesn't allow you to port forward) and/or you just don't want to deal with network issues. Since the tool box is just another pod and on the same namespace network, tools "just works".

K8S Port Forward

Use the port-forward command to connect your workstation into the k8s namespace.

kubectl port-forward svc/kube-starrocks-fe-service 9030:9030

Output

atwong@Alberts-MBP-2 sandbox % oc get svc
NAME                        TYPE           CLUSTER-IP       EXTERNAL-IP                            PORT(S)                               AGE
kube-starrocks-be-search    ClusterIP      None             <none>                                 9050/TCP                              9m39s
kube-starrocks-be-service   ClusterIP      172.30.248.230   <none>                                 9060/TCP,8040/TCP,9050/TCP,8060/TCP   9m39s
kube-starrocks-fe-search    ClusterIP      None             <none>                                 9030/TCP                              10m
kube-starrocks-fe-service   ClusterIP      172.30.121.157   <none>                                 8030/TCP,9020/TCP,9030/TCP,9010/TCP   10m
kubernetes                  ClusterIP      172.30.0.1       <none>                                 443/TCP                               128m
openshift                   ExternalName   <none>           kubernetes.default.svc.cluster.local   <none>                                125m
starrocks-tool-box          ClusterIP      172.30.230.0     <none>                                 8080/TCP                              5m16s
atwong@Alberts-MBP-2 sandbox % kubectl port-forward svc/kube-starrocks-fe-service 9030:9030
Forwarding from 127.0.0.1:9030 -> 9030
Forwarding from [::1]:9030 -> 9030
Handling connection for 9030

Launch another terminal session and run the mysql client

mysql -P9030 -h127.0.0.1 -uroot --prompt="StarRocks > "

Output

atwong@Alberts-MBP-2 ~ % mysql -P9030 -h127.0.0.1 -uroot --prompt="StarRocks > "

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 124
Server version: 5.1.0 StarRocks version 2.5.5

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

StarRocks > select current_version();
+-------------------+
| current_version() |
+-------------------+
| 2.5.5 24c1eca40f  |
+-------------------+
1 row in set (0.11 sec)

StarRocks > show backends;
+-----------+------------------------------------------------------------------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------+------------------+--------------------------------------------------------+-------------------+-------------+----------+-------------------+------------+------------+
| BackendId | IP                                                                     | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime       | LastHeartbeat       | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | ErrMsg | Version          | Status                                                 | DataTotalCapacity | DataUsedPct | CpuCores | NumRunningQueries | MemUsedPct | CpuUsedPct |
+-----------+------------------------------------------------------------------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------+------------------+--------------------------------------------------------+-------------------+-------------+----------+-------------------+------------+------------+
| 10002     | kube-starrocks-be-0.kube-starrocks-be-search.default.svc.cluster.local | 9050          | 9060   | 8040     | 8060     | 2023-06-30 23:13:32 | 2023-06-30 23:23:08 | true  | false                | false                 | 30        | 0.000            | 86.712 GB     | 99.489 GB     | 12.84 % | 12.84 %        |        | 2.5.5-24c1eca40f | {"lastSuccessReportTabletsTime":"2023-06-30 23:22:33"} | 86.712 GB         | 0.00 %      | 8        | 0                 | 1.86 %     | 0.0 %      |
+-----------+------------------------------------------------------------------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------+------------------+--------------------------------------------------------+-------------------+-------------+----------+-------------------+------------+------------+
1 row in set (0.08 sec)

StarRocks >

Using the StarRocks Tool Kit

Get the container at https://hub.docker.com/r/atwong/starrocks-tool-box and then execute a docker run.

Start the container

atwong@Alberts-MacBook-Pro-3 ~ % oc new-app --image=atwong/starrocks-tool-box
--> Found container image 8174163 (2 weeks old) from Docker Hub for "atwong/starrocks-tool-box"

    Red Hat Universal Base Image 9
    ------------------------------
    The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.

    Tags: base rhel9

    * An image stream tag will be created as "starrocks-tool-box:latest" that will track this image

--> Creating resources ...
    imagestream.image.openshift.io "starrocks-tool-box" created
    deployment.apps "starrocks-tool-box" created
--> Success
    Run 'oc status' to view your app.

Get the name of the tool-box pod and the IP of the FE.

atwong@Alberts-MacBook-Pro-3 ~ % oc get pods
NAME                                      READY   STATUS    RESTARTS   AGE
kube-starrocks-be-0                       1/1     Running   0          94m
kube-starrocks-cn-0                       1/1     Running   0          94m
kube-starrocks-fe-0                       1/1     Running   0          103m
kube-starrocks-operator-f97d6b85d-9b52h   1/1     Running   0          104m
starrocks-tool-box-5549cbcbd9-t7zz6       1/1     Running   0          67s
atwong@Alberts-MacBook-Pro-3 ~ % oc get svc
NAME                        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
kube-starrocks-be-search    ClusterIP   None            <none>        9050/TCP                              103m
kube-starrocks-be-service   ClusterIP   172.30.11.241   <none>        9060/TCP,8040/TCP,9050/TCP,8060/TCP   103m
kube-starrocks-cn-search    ClusterIP   None            <none>        9050/TCP                              103m
kube-starrocks-cn-service   ClusterIP   172.30.179.0    <none>        9060/TCP,8040/TCP,9050/TCP,8060/TCP   103m
kube-starrocks-fe-search    ClusterIP   None            <none>        9030/TCP                              176m
kube-starrocks-fe-service   ClusterIP   172.30.34.79    <none>        8030/TCP,9020/TCP,9030/TCP,9010/TCP   176m

Shell into the tool box pod

oc exec -it starrocks-tool-box-5549cbcbd9-t7zz6 -- /bin/bash

Execute the mysql client and connect to the FE.

bash-5.1$ mysql -P9030 -h172.30.34.79 -uroot --prompt="StarRocks > "
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 34
Server version: 5.1.0 3.1.0-rc01-64ca37e

Copyright (c) 2000, 2023, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

StarRocks >