DataFrame.explain(stage: Literal['logical', 'simplified-logical', 'tuned-logical', 'physical', 'simplified-physical', 'fused'] = 'fused', format: str | None = None)

Create a graph representation of the Expression.

explain runs the optimizer and creates a graph of the optimized expression with graphviz. No computation is triggered.

stage: {“logical”, “simplified-logical”, “tuned-logical”, “physical”, “simplified-physical”, “fused”}

The optimizer stage that is returned. Default is “fused”.

  • logical: outputs the expression as is

  • simplified-logical: simplifies the expression which includes predicate pushdown and column projection.

  • tuned-logical: applies additional optimizations like partition squashing

  • physical: outputs the physical expression; this expression can actually be computed

  • simplified-physical: runs another simplification after the physical plan is generated

  • fused: fuses the physical expression to reduce the nodes in thr graph.


The optimizer stages are subject to change.

format: str, default None

The format of the output. Default is “png”.

None, but opens a new window with the graph visualization and outputs
a file with the graph representation.