Installation#

Pre-requisites#

SkyCluster Operator runs as a Kubernetes operator and requires a Kubernetes cluster to run on. You can use any Kubernetes cluster, including a local cluster created using kind.

To install and run SkyCluster Operator, you need to ensure the following requirements are met on your machine:

  • Kubectl

  • Helm

  • Kind

  • Docker (for local cluster)

  • Crossplane (for managing underlying cloud resources)

  • Tailscale

  • Public IP Address: The cluster in your local machine is used to act as a broker between other gateways across different cloud providers, and hence it requires a public IP address to be reachable from the internet.

  • You need to open the following ports on your firewall to allow communication cross-domain:

    • 4500/UDP: Required for inter-cluster communication

    • 8000/TCP: Required for SkyCluster dashboard

    • 8080/TCP: Required for headscale (overlay setup)

    • 3478/UDP: Required for STUN Protocol (overlay setup)

    • 41641/UDP: Required for Tailscale (overlay setup)

Please make sure you have installed all tools before proceeding. We utilize kind to create a local cluster to run SkyCluster operator. Please ensure you can use kubectl without sudo before proceeding (refer to the docker post-installation guide).

Create a Local Cluster#

A local cluster is required to run the skycluster-operator and act as the point of contact for submitting your application. You can create a local management Kubernetes cluster using kind with the following command for testing purposes. If your machine has a public IP address you can bound the cluster to it by using the --advertise-address flag. If you plan to use the cluster for production purposes, you should consider using a more robust solution such as kubeadm or Rancher.

kind create cluster --name skycluster --config skycluster-kind.yaml

and the skycluster-kind.yaml file should contain the following content:

 1kind: Cluster
 2apiVersion: kind.x-k8s.io/v1alpha4
 3networking:
 4  podSubnet: 10.0.0.0/19
 5  serviceSubnet: 172.31.0.0/16
 6  apiServerAddress: 0.0.0.0
 7  apiServerPort: 6443
 8kubeadmConfigPatches:
 9  - |
10    kind: ClusterConfiguration
11    apiServer:
12      certSANs:
13        - 127.0.0.1
14        - 0.0.0.0
15        - skycluster.local
16        - a.b.c.d    # Replace with your cluster internal IP
17        - e.f.g.h    # Replace with your cluster public IP
18nodes:
19  - role: control-plane
20    extraPortMappings:
21    # Required for inter-cluster communication
22    - containerPort: 4500
23      hostPort: 4500
24      protocol: UDP
25    # Required for overlay setup
26    - containerPort: 30080
27      hostPort: 8080
28      protocol: TCP
29  - role: worker

The cluster is used used to act as a broker between other gateways across different cloud providers, and hence it requires a public IP address to be reachable from the internet. Once installed replace the 0.0.0.0 with the actual public IP address of your machine in the ~/.kube/config file:

sed -i "s/0\.0\.0\.0/$(curl -s ifconfig.io)/g" ~/.kube/config

At least one node in your cluster should be labeled as a gateway node. You can label the control plane node as follows:

kubectl label node skycluster-control-plane \
  skycluster.io/node-type=gateway \
  submariner.io/gateway=true

Warning

Ensure that a node is labeled as a gateway node and that you can access the cluster using the public IP address before proceeding to the next step.

Install Crossplane#

To manage the underlying cloud resources, you need to install Crossplane in your cluster. You can do this using the following command:

helm install crossplane \
  --namespace crossplane-system \
  --create-namespace crossplane-stable/crossplane \
  --version 1.20.0

Note

Ensure that Crossplane is installed successfully and all pods are running before proceeding to the next step.

kubectl get pods -n crossplane-system

Install SkyCluster#

SkyCluster Main Chart#

SkyCluster Manager supports AWS, GCP and Azure as well as on-premises infrastructure powered by OpenStack. Install the skycluster using helm chart as follows. All settings are deployed to the fixed namespac skycluster-system.

helm repo add skycluster https://skycluster.io/charts
helm repo update

helm install skycluster skycluster/skycluster

Note

The installation may take a few minutes to complete depending on your internet connection. You should wait till all providers listed below are installed and healthy before proceeding to the next step.

kubectl get providers.pkg

Ensure that all pods have INSTALLED and HEALTHY states equal to True.

Once you have all providers listed above all ready, you can proceed to the next step:

SkyCluster CRDs#

Warning

WIP: The following charts are not yet available for installation.

helm install skycluster-crds skycluster/skycluster

SkyCluster CA#

SkyCluster uses a self-signed CA to sign the certificates for its components. The CA is automatically generated during the installation of the SkyCluster operator. You need to install the CA in your cluster to enable secure communication between the SkyCluster components.

You can run the following command to install the CA in your cluster:

curl -s https://skycluster.io/configs/install-ca.sh | bash

The above script performs the following steps:

install-ca.sh

1CA_CERT=$(kubectl get secret skycluster-self-ca \
2  -n skycluster-system -o jsonpath='{.data.ca\.crt}')
3
4# Ensure the CA_CERT is not empty then:
5
6echo "$CA_CERT" | base64 -d | \
7  sudo tee /usr/local/share/ca-certificates/skycluster.crt > /dev/null
8
9sudo update-ca-certificates --fresh

SkyCluster Secret#

You need to create a secret containing a public key and a private key for the skycluster to authenticate itself with its components. The secret should be created in the skycluster-system namespace.

First export your public and private keys, assuming your private and public keys are named id_rsa and id_rsa.pub or adjust the paths to your keys:

export PUBLIC_KEY=$(cat ~/.ssh/id_rsa.pub)
export PRIVATE_KEY=$(cat ~/.ssh/id_rsa | base64 -w0)

And then run the following command to generate the secret:

curl -s https://skycluster.io/configs/skysecret-cfg.sh | bash

Alternatively, you can create a secret using a YAML file below:

skysecret-example.yaml

 1apiVersion: v1
 2kind: Secret
 3metadata:
 4  namespace: skycluster-system
 5  name: public-private-key
 6  labels:
 7    skycluster.io/managed-by: skycluster
 8    skycluster.io/secret-type: default-keypair
 9type: Opaque
10stringData:
11  config: |
12    {
13      "publicKey": "ssh-rsa AAAAB3NzaC1yc...fKEgCExt6YjE= ubuntu@cluster-dev1",
14      "privateKey": "LS0tLS1CRUdJTiBPUEVOU1..gS0VZLS0tLS0K"
15    }

Providers’ Configuration:

Once all providers are installed, you need to provide form of authentication to enable using hyperscalers such as AWS and GCP. Please follow the instructions in provider configuration page to apply required configurations.

Setting up Regions and Locations:

To enable skycluster-manager to deploy services across different providers you need to setup each regions within each prvoider that you configured in the previouse step.