Kubernetes
==========

Kubernetes_ is a popular system for deploying distributed applications on clusters,
particularly in the cloud.  You can use Kubernetes to launch Dask clusters in the
following ways:

Dask Kubernetes Operator
------------------------

The Dask Kubernetes Operator is a set of Custom Resource Definitions (CRDs) and a controller that
allows you to create and manage your Dask clusters as native Kubernetes resources.

Creating clusters can either be done via the Kubernetes API with ``kubectl`` or the
Python API with ``KubeCluster``.

.. code-block:: bash

   helm install --repo https://helm.dask.org --create-namespace -n dask-operator --generate-name dask-kubernetes-operator

.. code-block:: bash

   # Create a cluster with kubectl
   kubectl apply -f - <<EOF
   apiVersion: kubernetes.dask.org/v1
   kind: DaskCluster
   metadata:
     name: my-dask-cluster
   spec:
     ...
   EOF

.. code-block:: python

   # Create a cluster in Python
   from dask_kubernetes.operator import KubeCluster
   cluster = KubeCluster(name="my-dask-cluster", image='ghcr.io/dask/dask:latest')
   cluster.scale(10)

This is a good choice if you want to do the following:

1. Have a Kubernetes native experience.
2. Manage Dask clusters via the Kubernetes API and tools like ``kubectl``.
3. Integrate Dask with other tools and workloads running on Kubernetes.
4. Compose Dask clusters as part of a larger Kubernetes application.

Learn more at `kubernetes.dask.org <https://kubernetes.dask.org/en/latest/operator.html>`_.


Dask Gateway
------------

Dask Gateway provides a secure, multi-tenant server for managing Dask clusters.
It allows users to launch and use Dask clusters in a shared, centrally managed cluster environment,
without requiring users to have direct access to the underlying cluster backend
(e.g. Kubernetes, Hadoop/YARN, HPC Job queues, etc…).

.. code-block:: bash

   helm install --repo https://helm.dask.org --create-namespace -n dask-gateway --generate-name dask-gateway

.. code-block:: python

   from dask_gateway import Gateway
   gateway = Gateway("<gateway service address>")
   cluster = gateway.new_cluster()

This is a good choice if you want to do the following:

1. Abstract users away from Kubernetes.
2. Provide a consistent Dask user experience across Kubernetes/Hadoop/HPC.

Learn more at `gateway.dask.org <https://gateway.dask.org/install-kube.html>`_.

DaskHub
^^^^^^^

You can also deploy Dask Gateway alongside `JupyterHub <https://jupyter.org/hub>`_ using the DaskHub helm chart.

.. code-block:: bash

   helm install --repo https://helm.dask.org --create-namespace -n daskhub --generate-name daskhub

Learn more at the `artifacthub.io DaskHub page <https://artifacthub.io/packages/helm/dask/daskhub>`_.


Single Cluster Helm Chart
-------------------------

You can deploy a single Dask cluster and (optionally) Jupyter on Kubernetes
easily using Helm_

.. code-block:: bash

   helm install --repo https://helm.dask.org my-dask dask

This is a good choice if you want to do the following:

1. Try out Dask for the first time on a cloud-based system
   like Amazon, Google, or Microsoft Azure where you already have
   a Kubernetes cluster. If you don't already have Kubernetes deployed,
   see our :doc:`Cloud documentation <deploying-cloud>`.

You can also use the ``HelmCluster`` cluster manager from dask-kubernetes to manage your
Helm Dask cluster from within your Python session.

.. code-block:: python

   from dask_kubernetes import HelmCluster

   cluster = HelmCluster(release_name="myrelease")
   cluster.scale(10)

Learn more at the `artifacthub.io Dask page <https://artifacthub.io/packages/helm/dask/dask>`_.

Further Reading
---------------

You may also want to see the documentation on using
:doc:`Dask with Docker containers <deploying-docker>`
to help you manage your software environments on Kubernetes.

.. _Kubernetes: https://kubernetes.io/
.. _Dask-Kubernetes: https://kubernetes.dask.org/
.. _Helm: https://helm.sh/