The Dask distributed scheduler provides live feedback in two forms:
- An interactive dashboard containing many plots and tables with live information
- A progress bar suitable for interactive use in consoles or notebooks
If Bokeh is installed then the dashboard will start up automatically whenever the scheduler is created. For local use this happens when you create a client with no arguments:
from dask.distributed import Client client = Client() # start distributed scheduler locally. Launch dashboard
It is typically served at http://localhost:8787/status ,
but may be served elsewhere if this port is taken.
The address of the dashboard will be displayed if you are in a Jupyter Notebook,
or can be queriesd from
There are numerous pages with information about task runtimes, communication, statistical profiling, load balancing, memory use, and much more. For more information we recommend the video guide above.
||Connect to and submit computation to a Dask cluster|
dask.distributed progress bar differs from the
ProgressBar used for
progress function takes a Dask object that is executing in the background:
# Single machine progress bar from dask.diagnostics import ProgressBar with ProgressBar(): x.compute() # Distributed scheduler ProgressBar from dask.distributed import Client, progress client = Client() # use dask.distributed by default x = x.persist() # start computation in the background progress(x) # watch progress x.compute() # convert to final result when done if desired
More in-depth technical documentation about Dask’s distributed scheduler is available at https://distributed.dask.org/en/latest