Changelog
Contents
Changelog¶
2023.11.0¶
Released on November 10, 2023
Highlights¶
Zero-copy P2P Array Rechunking¶
Users should see significant performance improvements when using in-memory P2P array rechunking. This is due to no longer copying underlying data buffers.
Below shows a simple example where we compare performance of different rechunking methods.
shape = (30_000, 6_000, 150) # 201.17 GiB
input_chunks = (60, -1, -1) # 411.99 MiB
output_chunks = (-1, 6, -1) # 205.99 MiB
arr = da.random.random(size, chunks=input_chunks)
with dask.config.set({
"array.rechunk.method": "p2p",
"distributed.p2p.disk": True,
}):
(
da.random.random(size, chunks=input_chunks)
.rechunk(output_chunks)
.sum()
.compute()
)

See GH#8282, GH#8318, GH#8321 from crusaderky and (GH#8322) from Hendrik Makait for details.
Deprecating PyArrow <14.0.1¶
pyarrow<14.0.1
usage is deprecated starting in this release. It’s recommended for all users to upgrade their
version of pyarrow
or install pyarrow-hotfix
. See this CVE
for full details.
See GH#10622 from Florian Jetter for details.
Improved PyArrow filesystem for Parquet¶
Using filesystem="arrow"
when reading Parquet datasets now properly inferrs the correct cloud region
when accessing remote, cloud-hosted data.
See GH#10590 from Richard (Rick) Zamora for details.
Improve Type Reconciliation in P2P Shuffling¶
See GH#8332 from Hendrik Makait for details.
Additional changes
Fix sporadic failure of
test_dataframe::test_quantile
(GH#10625) MilesBump minimum
click
to>=8.1
(GH#10623) Jacob TomlinsonAvoid
PerformanceWarning
for fragmented DataFrame (GH#10621) Patrick HoeflerGeneralize computation of
NEW_*_VER
in GPU CI updating workflow (GH#10610) Charles Blackmon-LucaSwitch to newer GPU CI images (GH#10608) Charles Blackmon-Luca
Remove double slash in
fsspec
tests (GH#10605) Mario ŠaškoReenable
test_ucx_config_w_env_var
(GH#8272) Peter Andreas EntschevDon’t share
host_array
when receiving from network (GH#8308) crusaderkyGeneralize computation of
NEW_*_VER
in GPU CI updating workflow (GH#8319) Charles Blackmon-LucaSwitch to newer GPU CI images (GH#8316) Charles Blackmon-Luca
Minor updates to shuffle dashboard (GH#8315) Matthew Rocklin
Don’t use
bytearray().join
(GH#8312) crusaderkyReuse identical shuffles in P2P hash join (GH#8306) Hendrik Makait
2023.10.1¶
Released on October 27, 2023
Highlights¶
Python 3.12¶
This release adds official support for Python 3.12.
See GH#10544 and GH#8223 from Thomas Grainger for details.
Additional changes
Avoid splitting parquet files to row groups as aggressively (GH#10600) Matthew Rocklin
Speed up
normalize_chunks
for common case (GH#10579) Martin DurantUse Python 3.11 for upstream and doctests CI build (GH#10596) Thomas Grainger
Bump
actions/checkout
from 4.1.0 to 4.1.1 (GH#10592)Switch to PyTables
HEAD
(GH#10580) Thomas GraingerRemove
numpy.core
warning filter, link to issue onpyarrow
causedBlockManager
warning (GH#10571) Thomas GraingerUnignore and fix deprecated freq aliases (GH#10577) Thomas Grainger
Move
register_assert_rewrite
earlier inconftest
to fix warnings (GH#10578) Thomas GraingerUpgrade
versioneer
to 0.29 (GH#10575) Thomas Graingerchange
test_concat_categorical
to be non-strict (GH#10574) Thomas GraingerEnable SciPy tests with NumPy 2.0 Thomas Grainger
Enable tests for scikit-image with NumPy 2.0 (GH#10569) Thomas Grainger
Fix upstream build (GH#10549) Thomas Grainger
Add optimized code paths for
drop_duplicates
(GH#10542) Richard (Rick) ZamoraSupport
cudf
backend indd.DataFrame.sort_values
(GH#10551) Richard (Rick) ZamoraRename “GIL Contention” to just GIL in chart labels (GH#8305) Matthew Rocklin
Bump
actions/checkout
from 4.1.0 to 4.1.1 (GH#8299)Fix dashboard (GH#8293) Hendrik Makait
@log_errors
for async tasks (GH#8294) crusaderkyAnnotations and better tests for serialize_bytes (GH#8300) crusaderky
Temporarily xfail
test_decide_worker_coschedule_order_neighbors
to unblock CI (GH#8298) James BourbeauSkip
xdist
andmatplotlib
in code samples (GH#8290) Matthew RocklinUse
numpy._core
onnumpy>=2.dev0
(GH#8291) Thomas GraingerFix calculation of
MemoryShardsBuffer.bytes_read
(GH#8289) crusaderkyAllow P2P to store data in-memory (GH#8279) Hendrik Makait
Upgrade
versioneer
to 0.29 (GH#8288) Thomas GraingerAllow
ResourceLimiter
to be unlimited (GH#8276) Hendrik MakaitRun
pre-commit
autoupdate (GH#8281) Thomas GraingerAnnotate instance variables for P2P layers (GH#8280) Hendrik Makait
Remove worker gracefully should not mark tasks as suspicious (GH#8234) Thomas Grainger
Add signal handling to
dask spec
(GH#8261) Thomas GraingerAdd typing for
sync
(GH#8275) Hendrik MakaitBetter annotations for shuffle offload (GH#8277) crusaderky
Test minimum versions for p2p shuffle (GH#8270) crusaderky
Run coverage on test failures (GH#8269) crusaderky
Use
aiohttp
with extensions (GH#8274) Thomas Grainger
2023.10.0¶
Released on October 13, 2023
Highlights¶
Reduced memory pressure for multi array reductions¶
This release contains major updates to Dask’s task graph scheduling logic. The updates here significantly reduce memory pressure on array reductions. We anticipate this will have a strong impact on the array computing community.
See GH#10535 from Florian Jetter for details.
Improved P2P shuffling robustness¶
There are several updates (listed below) that make P2P shuffling much more robust and less likely to fail.
See GH#8262, GH#8264, GH#8242, GH#8244, and GH#8235 from Hendrik Makait and GH#8124 from Charles Blackmon-Luca for details.
Reduced scheduler CPU load for large graphs¶
Users should see reduced CPU load on their scheduler when computing large task graphs.
See GH#8238 and GH#10547 from Florian Jetter and GH#8240 from crusaderky for details.
Additional changes
Dispatch the
partd.Encode
class used for disk-based shuffling (GH#10552) Richard (Rick) ZamoraAdd documentation for hive partitioning (GH#10454) Richard (Rick) Zamora
Add typing to
dask.order
(GH#10553) Florian JetterAllow passing
index_col=False
indd.read_csv
(GH#9961) Michael LeslieTighten
HighLevelGraph
annotations (GH#10524) crusaderkySupport for latest
ipykernel
/ipywidgets
(GH#8253) crusaderkyCheck minimal
pyarrow
version for P2P merge (GH#8266) Hendrik MakaitSupport for Python 3.12 (GH#8223) Thomas Grainger
Use
memoryview.nbytes
when warning on large graph send (GH#8268) crusaderkyRun tests without
gilknocker
(GH#8263) crusaderkyDisable ipv6 on MacOS CI (GH#8254) crusaderky
Clean up redundant minimum versions (GH#8251) crusaderky
Clean up use of
BARRIER_PREFIX
in scheduler plugin (GH#8252) crusaderkyImprove shuffle run handling in P2P’s worker plugin (GH#8245) Hendrik Makait
Explicitly set
charset=utf-8
(GH#8250) crusaderkyTyping tweaks to GH#8239 (GH#8247) crusaderky
Simplify scheduler assertion (GH#8246) crusaderky
Improve typing (GH#8239) Hendrik Makait
Respect cgroups v2 “low” memory limit (GH#8243) Samantha Hughes
Fix
PackageInstall
by making it a scheduler plugin (GH#8142) Hendrik MakaitXfail
test_ucx_config_w_env_var
(GH#8241) crusaderkySpecCluster
resilience to broken workers (GH#8233) crusaderkySuppress
SpillBuffer
stack traces for cancelled tasks (GH#8232) crusaderkyUpdate annotations after stringification changes (GH#8195) crusaderky
Reduce max recursion depth of profile (GH#8224) crusaderky
Offload deeply nested objects (GH#8214) crusaderky
Fix flaky
test_close_connections
(GH#8231) crusaderkyFix flaky
test_popen_timeout
(GH#8229) crusaderkyFix flaky
test_adapt_then_manual
(GH#8228) crusaderkyPrevent collisions in
SpillBuffer
(GH#8226) crusaderkyAllow
retire_workers
to run concurrently (GH#8056) Florian JetterFix HTML repr for
TaskState
objects (GH#8188) Florian JetterFix
AttributeError
forbuiltin_function_or_method
inprofile.py
(GH#8181) Florian JetterFix flaky
test_spans
(v2) (GH#8222) crusaderky
2023.9.3¶
Released on September 29, 2023
Highlights¶
Restore previous configuration override behavior¶
The 2023.9.2 release introduced an unintentional breaking change in
how configuration options are overriden in dask.config.get
with
the override_with=
keyword (see GH#10519).
This release restores the previous behavior.
See GH#10521 from crusaderky for details.
Complex dtypes in Dask Array reductions¶
This release includes improved support for using common reductions
in Dask Array (e.g. var
, std
, moment
) with complex dtypes.
See GH#10009 from wkrasnicki for details.
Additional changes
Bump
actions/checkout
from 4.0.0 to 4.1.0 (GH#10532)Match
pandas
revertingapply
deprecation (GH#10531) James BourbeauUpdate gpuCI
RAPIDS_VER
to23.12
(GH#10526)Temporarily skip failing tests with
fsspec==2023.9.1
(GH#10520) James Bourbeau
2023.9.2¶
Released on September 15, 2023
Highlights¶
P2P shuffling now raises when outdated PyArrow is installed¶
Previously the default shuffling method would silently fallback from P2P
to task-based shuffling if an older version of pyarrow
was installed.
Now we raise an informative error with the minimum required pyarrow
version for P2P instead of silently falling back.
See GH#10496 from Hendrik Makait for details.
Deprecation cycle for admin.traceback.shorten¶
The 2023.9.0 release modified the admin.traceback.shorten
configuration option
without introducing a deprecation cycle. This resulted in failures to create Dask
clusters in some cases. This release introduces a deprecation cycle for this configuration
change.
See GH#10509 from crusaderky for details.
Additional changes
Avoid materializing all iterators in
delayed
tasks (GH#10498) James BourbeauOverhaul deprecations system in
dask.config
(GH#10499) crusaderkyRemove unnecessary check in
timeseries
(GH#10447) Patrick HoeflerUse
register_plugin
in tests (GH#10503) James BourbeauMake
preserve_index
explicit inpyarrow_schema_dispatch
(GH#10501) Hendrik MakaitAdd
**kwargs
support forpyarrow_schema_dispatch
(GH#10500) Hendrik MakaitCentralize and type
no_default
(GH#10495) crusaderky
2023.9.1¶
Released on September 6, 2023
Note
This is a hotfix release that fixes a P2P shuffling bug introduced in the 2023.9.0 release (see GH#10493).
Enhancements¶
Stricter data type for dask keys (GH#10485) crusaderky
Special handling for
None
inDASK_
environment variables (GH#10487) crusaderky
Bug Fixes¶
Fix
_partitions
dtype
inmeta
forDataFrame.set_index
andDataFrame.sort_values
(GH#10493) Hendrik MakaitHandle
cached_property
decorators inderived_from
(GH#10490) Lawrence Mitchell
Maintenance¶
Bump
actions/checkout
from 3.6.0 to 4.0.0 (GH#10492)Simplify some tests that
import distributed
(GH#10484) crusaderky
2023.9.0¶
Released on September 1, 2023
Bug Fixes¶
Remove support for
np.int64
in keys (GH#10483) crusaderkyFix
_partitions
dtype
inmeta
for shuffling (GH#10462) Hendrik MakaitDon’t use exception hooks to shorten tracebacks (GH#10456) crusaderky
Documentation¶
Add
p2p
shuffle option to DataFrame docs (GH#10477) Patrick Hoefler
Maintenance¶
Skip failing tests for
pandas=2.1.0
(GH#10488) Patrick HoeflerUpdate tests for
pandas=2.1.0
(GH#10439) Patrick HoeflerEnable
pytest-timeout
(GH#10482) crusaderkyBump
actions/checkout
from 3.5.3 to 3.6.0 (GH#10470)
2023.8.1¶
Released on August 18, 2023
Enhancements¶
Adding support for cgroup v2 to
cpu_count
(GH#10419) Johan OlssonSupport multi-column
groupby
withsort=True
andsplit_out>1
(GH#10425) Richard (Rick) ZamoraAdd
DataFrame.enforce_runtime_divisions
method (GH#10404) Richard (Rick) ZamoraEnable file
mode="x"
with asingle_file=True
for Dask DataFrameto_csv
(GH#10443) Genevieve Buckley
Bug Fixes¶
Maintenance¶
Add default
types_mapper
tofrom_pyarrow_table_dispatch
forpandas
(GH#10446) Richard (Rick) Zamora
2023.8.0¶
Released on August 4, 2023
Enhancements¶
Fix for
make_timeseries
performance regression (GH#10428) Irina Truong
Documentation¶
Add
distributed.print
to debugging docs (GH#10435) James BourbeauDocumenting compatibility of NumPy functions with Dask functions (GH#9941) Chiara Marmo
Maintenance¶
Use SPDX in
license
metadata (GH#10437) John A KirkhamRequire
dask[array]
indask[dataframe]
(GH#10357) John A KirkhamUpdate gpuCI
RAPIDS_VER
to23.10
(GH#10427)Simplify compatibility code (GH#10426) Hendrik Makait
Fix compatibility variable naming (GH#10424) Hendrik Makait
Fix a few errors with upstream
pandas
andpyarrow
(GH#10412) Irina Truong
2023.7.1¶
Released on July 20, 2023
Note
This release updates Dask DataFrame to automatically convert
text data using object
data types to string[pyarrow]
if pandas>=2
and pyarrow>=12
are installed.
This should result in significantly reduced memory consumption and increased computation performance in many workflows that deal with text data.
You can disable this change by setting the dataframe.convert-string
configuration value to False
with
dask.config.set({"dataframe.convert-string": False})
Enhancements¶
Convert to
pyarrow
strings if proper dependencies are installed (GH#10400) James BourbeauAvoid
repartition
beforeshuffle
forp2p
(GH#10421) Patrick HoeflerAPI to generate random Dask DataFrames (GH#10392) Irina Truong
Speed up
dask.bag.Bag.random_sample
(GH#10356) crusaderkyRaise helpful
ValueError
for invalid time units (GH#10408) Nat TabrisMake
repartition
a no-op when divisions match (divisions provided as a list) (GH#10395) Nicolas Grandemange
Bug Fixes¶
Use
dataframe.convert-string
inread_parquet
token (GH#10411) James BourbeauCategory
dtype
is lost when concatenatingMultiIndex
(GH#10407) Irina TruongFix
FutureWarning: The provided callable...
(GH#10405) Irina TruongEnable non-categorical hive-partition columns in
read_parquet
(GH#10353) Richard (Rick) Zamoraconcat
ignoringDataFrame
withouth columns (GH#10359) Patrick Hoefler
2023.7.0¶
Released on July 7, 2023
Enhancements¶
Catch exceptions when attempting to load CLI entry points (GH#10380) Jacob Tomlinson
Bug Fixes¶
Fix typo in
_clean_ipython_traceback
(GH#10385) Alexander ClausenEnsure that
df
is immutable afterfrom_pandas
(GH#10383) Patrick HoeflerWarn consistently for
inplace
inSeries.rename
(GH#10313) Patrick Hoefler
Documentation¶
Add clarification about output shape and reshaping in rechunk documentation (GH#10377) Swayam Patil
Maintenance¶
Simplify
astype
implementation (GH#10393) Patrick HoeflerFix
test_first_and_last
to accommodate deprecatedlast
(GH#10373) James BourbeauAdd
level
tocreate_merge_tree
(GH#10391) Patrick HoeflerDo not derive from
scipy.stats.chisquare
docstring (GH#10382) Doug Davis
2023.6.1¶
Released on June 26, 2023
Enhancements¶
Remove no longer supported
clip_lower
andclip_upper
(GH#10371) Patrick HoeflerSupport
DataFrame.set_index(..., sort=False)
(GH#10342) MilesCleanup remote tracebacks (GH#10354) Irina Truong
Add dispatching mechanisms for
pyarrow.Table
conversion (GH#10312) Richard (Rick) ZamoraChoose P2P even if fusion is enabled (GH#10344) Hendrik Makait
Validate that rechunking is possible earlier in graph generation (GH#10336) Hendrik Makait
Bug Fixes¶
Fix issue with
header
passed toread_csv
(GH#10355) GALI PREM SAGARRespect
dropna
andobserved
inGroupBy.var
andGroupBy.std
(GH#10350) Patrick HoeflerFix
H5FD_lock
error when writing to hdf with distributed client (GH#10309) Irina TruongFix for
total_mem_usage
ofbag.map()
(GH#10341) Irina Truong
Deprecations¶
Deprecate
DataFrame.fillna
/Series.fillna
withmethod
(GH#10349) Irina TruongDeprecate
DataFrame.first
andSeries.first
(GH#10352) Irina Truong
Maintenance¶
Deprecate
numpy.compat
(GH#10370) Irina TruongFix annotations and spans leaking between threads (GH#10367) Irina Truong
Use general kwargs in
pyarrow_table_dispatch
functions (GH#10364) Richard (Rick) ZamoraRemove unnecessary
try
/except
inisna
(GH#10363) Patrick Hoeflermypy
support for numpy 1.25 (GH#10362) crusaderkyBump
actions/checkout
from 3.5.2 to 3.5.3 (GH#10348)Restore
numba
inupstream
build (GH#10330) James BourbeauUpdate nightly wheel index for
pandas
/numpy
/scipy
(GH#10346) Matthew RoeschkeAdd rechunk config values to yaml (GH#10343) Hendrik Makait
2023.6.0¶
Released on June 9, 2023
Enhancements¶
Add missing
not in
predicate support toread_parquet
(GH#10320) Richard (Rick) Zamora
Bug Fixes¶
Fix for incorrect
value_counts
(GH#10323) Irina TruongUpdate empty
describe
top and freq values (GH#10319) James Bourbeau
Documentation¶
Fix hetzner typo (GH#10332) Sarah Charlotte Johnson
Maintenance¶
Test with
numba
andsparse
on Python 3.11 (GH#10329) Thomas GraingerRemove
numpy.find_common_type
warning ignore (GH#10311) James BourbeauUpdate gpuCI
RAPIDS_VER
to23.08
(GH#10310)
2023.5.1¶
Released on May 26, 2023
Note
This release drops support for Python 3.8. As of this release Dask supports Python 3.9, 3.10, and 3.11. See this community issue for more details.
Enhancements¶
Drop Python 3.8 support (GH#10295) Thomas Grainger
Change Dask Bag partitioning scheme to improve cluster saturation (GH#10294) Jacob Tomlinson
Generalize
dd.to_datetime
for GPU-backed collections, introduceget_meta_library
utility (GH#9881) Charles Blackmon-LucaAdd
na_action
toDataFrame.map
(GH#10305) Patrick HoeflerRaise
TypeError
inDataFrame.nsmallest
andDataFrame.nlargest
whencolumns
is not given (GH#10301) Patrick HoeflerImprove
sizeof
forpd.MultiIndex
(GH#10230) Patrick HoeflerSupport duplicated columns in a bunch of
DataFrame
methods (GH#10261) Patrick HoeflerAdd
numeric_only
support toDataFrame.idxmin
andDataFrame.idxmax
(GH#10253) Patrick HoeflerImplement
numeric_only
support forDataFrame.quantile
(GH#10259) Patrick HoeflerAdd support for
numeric_only=False
inDataFrame.std
(GH#10251) Patrick HoeflerImplement
numeric_only=False
forGroupBy.cumprod
andGroupBy.cumsum
(GH#10262) Patrick HoeflerImplement
numeric_only
forskew
andkurtosis
(GH#10258) Patrick Hoeflermask
andwhere
should accept acallable
(GH#10289) Irina TruongFix conversion from
Categorical
topa.dictionary
inread_parquet
(GH#10285) Patrick Hoefler
Bug Fixes¶
Spurious config on nested annotations (GH#10318) crusaderky
Fix rechunking behavior for dimensions with known and unknown chunk sizes (GH#10157) Hendrik Makait
Enable
drop
to support mismatched partitions (GH#10300) James BourbeauFix
divisions
construction forto_timestamp
(GH#10304) Patrick Hoeflerpandas
ExtensionDtype
raising inSeries
reduction operations (GH#10149) Patrick HoeflerFix regression in
da.random
interface (GH#10247) Eray Aslanda.coarsen
doesn’t trim an empty chunk in meta (GH#10281) Irina TruongFix dtype inference for
engine="pyarrow"
inread_csv
(GH#10280) Patrick Hoefler
Documentation¶
Add
meta_from_array
to API docs (GH#10306) Ruth ComerUpdate Coiled links (GH#10296) Sarah Charlotte Johnson
Add docs for demo day (GH#10288) Matthew Rocklin
Maintenance¶
Explicitly install
anaconda-client
from conda-forge when uploading conda nightlies (GH#10316) Charles Blackmon-LucaConfigure
isort
to addfrom __future__ import annotations
(GH#10314) Thomas GraingerAvoid
pandas
Series.__getitem__
deprecation in tests (GH#10308) James BourbeauIgnore
numpy.find_common_type
warning frompandas
(GH#10307) James BourbeauAdd test to check that
DataFrame.__setitem__
does not modifydf
inplace (GH#10223) Patrick HoeflerClean up default value of
dropna
invalue_counts
(GH#10299) Patrick HoeflerAdd
pytest-cov
totest
extra (GH#10271) James Bourbeau
2023.5.0¶
Released on May 12, 2023
Enhancements¶
Implement
numeric_only=False
forGroupBy.corr
andGroupBy.cov
(GH#10264) Patrick HoeflerAdd support for
numeric_only=False
inDataFrame.var
(GH#10250) Patrick HoeflerAdd
numeric_only
support toDataFrame.mode
(GH#10257) Patrick HoeflerAdd
DataFrame.map
todask.DataFrame
API (GH#10246) Patrick HoeflerAdjust for
DataFrame.applymap
deprecation and allNA
concat
behaviour change (GH#10245) Patrick HoeflerEnable
numeric_only=False
forDataFrame.count
(GH#10234) Patrick HoeflerDisallow array input in mask/where (GH#10163) Irina Truong
Support
numeric_only=True
inGroupBy.corr
andGroupBy.cov
(GH#10227) Patrick HoeflerAdd
numeric_only
support toGroupBy.median
(GH#10236) Patrick HoeflerSupport
mimesis=9
indask.datasets
(GH#10241) James BourbeauAdd
numeric_only
support tomin
,max
andprod
(GH#10219) Patrick HoeflerAdd
numeric_only=True
support forGroupBy.cumsum
andGroupBy.cumprod
(GH#10224) Patrick HoeflerAdd helper to unpack
numeric_only
keyword (GH#10228) Patrick Hoefler
Bug Fixes¶
Fix
clone
+from_array
failure (GH#10211) crusaderkyFix dataframe reductions for ea dtypes (GH#10150) Patrick Hoefler
Avoid scalar conversion deprecation warning in
numpy=1.25
(GH#10248) James BourbeauMake sure transform output has the same index as input (GH#10184) Irina Truong
Fix
corr
andcov
on a single-row partition (GH#9756) Irina TruongFix
test_groupby_numeric_only_supported
andtest_groupby_aggregate_categorical_observed
upstream errors (GH#10243) Irina Truong
Documentation¶
Clean up futures docs (GH#10266) Matthew Rocklin
Maintenance¶
Warn when meta is passed to
apply
(GH#10256) Patrick HoeflerRemove
imageio
version restriction in CI (GH#10260) Patrick HoeflerRemove unused
DataFrame
variance methods (GH#10252) Patrick HoeflerUn-
xfail
test_categories
withpyarrow
strings andpyarrow>=12
(GH#10244) Irina TruongBump gpuCI
PYTHON_VER
3.8->3.9 (GH#10233) Charles Blackmon-Luca
2023.4.1¶
Released on April 28, 2023
Enhancements¶
Implement
numeric_only
support forDataFrame.sum
(GH#10194) Patrick HoeflerAdd support for
numeric_only=True
inGroupBy
operations (GH#10222) Patrick HoeflerAvoid deep copy in
DataFrame.__setitem__
forpandas
1.4 and up (GH#10221) Patrick HoeflerAvoid calling
Series.apply
with_meta_nonempty
(GH#10212) Patrick HoeflerUnpin
sqlalchemy
and fix compatibility issues (GH#10140) Patrick Hoefler
Bug Fixes¶
Partially revert default client discovery (GH#10225) Florian Jetter
Support arrow dtypes in
Index
meta creation (GH#10170) Patrick HoeflerRepartitioning raises with extension dtype when truncating floats (GH#10169) Patrick Hoefler
Adjust empty
Index
fromfastparquet
toobject
dtype (GH#10179) Patrick Hoefler
Documentation¶
Update Kubernetes docs (GH#10232) Jacob Tomlinson
Add
DataFrame.reduction
to API docs (GH#10229) James BourbeauAdd
DataFrame.persist
to docs and fix links (GH#10231) Patrick HoeflerAdd documentation for
GroupBy.transform
(GH#10185) Irina TruongFix formatting in random number generation docs (GH#10189) Eray Aslan
Maintenance¶
Pin imageio to
<2.28
(GH#10216) Patrick HoeflerAdd note about
importlib_metadata
backport (GH#10207) James BourbeauAdd
xarray
back to Python 3.11 CI builds (GH#10200) James BourbeauAdd
mindeps
build with all optional dependencies (GH#10161) Charles Blackmon-LucaProvide proper
like
value forarray_safe
inpercentiles_summary
(GH#10156) Charles Blackmon-LucaAvoid re-opening hdf file multiple times in
read_hdf
(GH#10205) Thomas GraingerAdd merge tests on nullable columns (GH#10071) Charles Blackmon-Luca
Fix coverage configuration (GH#10203) Thomas Grainger
Remove
is_period_dtype
andis_sparse_dtype
(GH#10197) Patrick HoeflerBump
actions/checkout
from 3.5.0 to 3.5.2 (GH#10201)Avoid deprecated
is_categorical_dtype
from pandas (GH#10180) Patrick HoeflerAdjust for deprecated
is_interval_dtype
andis_datetime64tz_dtype
(GH#10188) Patrick Hoefler
2023.4.0¶
Released on April 14, 2023
Enhancements¶
Override old default values in
update_defaults
(GH#10159) Gabe JosephAdd a CLI command to
list
andget
a value from dask config (GH#9936) Irina TruongHandle string-based engine argument to
read_json
(GH#9947) Richard (Rick) ZamoraAvoid deprecated
GroupBy.dtypes
(GH#10111) Irina Truong
Bug Fixes¶
Revert
grouper
-related changes (GH#10182) Irina TruongGroupBy.cov
raising for non-numeric grouping column (GH#10171) Patrick HoeflerUpdates for
Index
supportingnumpy
numeric dtypes (GH#10154) Irina TruongPreserve
dtype
for partitioning columns when read withpyarrow
(GH#10115) Patrick HoeflerFix annotations for
to_hdf
(GH#10123) Hendrik MakaitHandle
None
column name when checking if columns are all numeric (GH#10128) Lawrence MitchellFix
valid_divisions
when passed atuple
(GH#10126) Brian PhillipsMaintain annotations in
DataFrame.categorize
(GH#10120) Hendrik MakaitFix handling of missing min/max parquet statistics during filtering (GH#10042) Richard (Rick) Zamora
Deprecations¶
Deprecate
use_nullable_dtypes=
and adddtype_backend=
(GH#10076) Irina TruongDeprecate
convert_dtype
inSeries.apply
(GH#10133) Irina Truong
Documentation¶
Document
Generator
based random number generation (GH#10134) Eray Aslan
Maintenance¶
Update
dataframe.convert_string
todataframe.convert-string
(GH#10191) Irina TruongAdd
python-cityhash
to CI environments (GH#10190) Charles Blackmon-LucaTemporarily pin
scikit-image
to fix Windows CI (GH#10186) Patrick HoeflerHandle pandas deprecation warnings for
to_pydatetime
andapply
(GH#10168) Patrick HoeflerDrop
bokeh<3
restriction (GH#10177) James BourbeauFix failing tests under copy-on-write (GH#10173) Patrick Hoefler
Allow
pyarrow
CI to fail (GH#10176) James BourbeauSwitch to
Generator
for random number generation indask.array
(GH#10003) Eray AslanBump
peter-evans/create-pull-request
from 4 to 5 (GH#10166)Fix flaky
modf
operation intest_arithmetic
(GH#10162) Irina TruongTemporarily remove
xarray
from CI withpandas
2.0 (GH#10153) James BourbeauFix
update_graph
counting logic intest_default_scheduler_on_worker
(GH#10145) James BourbeauFix documentation build with
pandas
2.0 (GH#10138) James BourbeauRemove
dask/gpu
from gpuCI update reviewers (GH#10135) Charles Blackmon-LucaUpdate gpuCI
RAPIDS_VER
to23.06
(GH#10129)Bump
actions/stale
from 6 to 8 (GH#10121)Use declarative
setuptools
(GH#10102) Thomas GraingerRelax
assert_eq
checks onScalar
-like objects (GH#10125) Matthew RocklinUpgrade readthedocs config to ubuntu 22.04 and Python 3.11 (GH#10124) Thomas Grainger
Bump
actions/checkout
from 3.4.0 to 3.5.0 (GH#10122)Fix
test_null_partition_pyarrow
inpyarrow
CI build (GH#10116) Irina TruongDrop distributed pack (GH#9988) Florian Jetter
Make
dask.compatibility
private (GH#10114) Jacob Tomlinson
2023.3.2¶
Released on March 24, 2023
Enhancements¶
Deprecate
observed=False
forgroupby
with categoricals (GH#10095) Irina TruongDeprecate
axis=
for some groupby operations (GH#10094) James BourbeauThe
axis
keyword inDataFrame.rolling/Series.rolling
is deprecated (GH#10110) Irina TruongDataFrame._data
deprecation inpandas
(GH#10081) Irina TruongUse
importlib_metadata
backport to avoid CLIUserWarning
(GH#10070) Thomas GraingerPort option parsing logic from
dask.dataframe.read_parquet
toto_parquet
(GH#9981) Anton Loukianov
Bug Fixes¶
Avoid using
dd.shuffle
in groupby-apply (GH#10043) Richard (Rick) ZamoraEnable null hive partitions with
pyarrow
parquet engine (GH#10007) Richard (Rick) ZamoraSupport unknown shapes in
*_like
functions (GH#10064) Doug Davis
Documentation¶
Add
to_backend
methods to API docs (GH#10093) Lawrence MitchellRemove broken gpuCI link in developer docs (GH#10065) Charles Blackmon-Luca
Maintenance¶
Configure readthedocs sphinx warnings as errors (GH#10104) Thomas Grainger
Un-
xfail
test_division_or_partition
withpyarrow
strings active (GH#10108) Irina TruongUn-
xfail
test_different_columns_are_allowed
withpyarrow
strings active (GH#10109) Irina TruongRestore Entrypoints compatibility (GH#10113) Jacob Tomlinson
Un-
xfail
test_to_dataframe_optimize_graph
withpyarrow
strings active (GH#10087) Irina TruongOnly run
test_development_guidelines_matches_ci
on editable install (GH#10106) Charles Blackmon-LucaUn-
xfail
test_dataframe_cull_key_dependencies_materialized
withpyarrow
strings active (GH#10088) Irina TruongInstall
mimesis
in CI environments (GH#10105) Charles Blackmon-LucaFix for no module named
ipykernel
(GH#10101) Irina TruongFix docs builds by installing
ipykernel
(GH#10103) Thomas GraingerAllow
pyarrow
build to continue on failures (GH#10097) James BourbeauBump
actions/checkout
from 3.3.0 to 3.4.0 (GH#10096)Fix
test_set_index_on_empty
withpyarrow
strings active (GH#10054) Irina TruongUn-
xfail
pyarrow
pickling tests (GH#10082) James BourbeauCI environment file cleanup (GH#10078) James Bourbeau
Un-
xfail
morepyarrow
tests (GH#10066) Irina TruongTemporarily skip
pyarrow_compat
tests with p`andas 2.0 (GH#10063) James BourbeauFix
test_melt
withpyarrow
strings active (GH#10052) Irina TruongFix
test_str_accessor
withpyarrow
strings active (GH#10048) James BourbeauFix
test_better_errors_object_reductions
withpyarrow
strings active (GH#10051) James BourbeauFix
test_loc_with_non_boolean_series
withpyarrow
strings active (GH#10046) James BourbeauFix
test_values
withpyarrow
strings active (GH#10050) James BourbeauTemporarily
xfail
test_upstream_packages_installed
(GH#10047) James Bourbeau
2023.3.1¶
Released on March 10, 2023
Enhancements¶
Support pyarrow strings in
MultiIndex
(GH#10040) Irina TruongImproved support for
pyarrow
strings (GH#10000) Irina TruongFix flaky
RuntimeWarning
during array reductions (GH#10030) James BourbeauExtend
complete
extras (GH#10023) James BourbeauRaise an error with
dataframe.convert-string=True
andpandas<2.0
(GH#10033) Irina TruongRename shuffle/rechunk config option/kwarg to
method
(GH#10013) James BourbeauAdd initial support for converting
pandas
extension dtypes to arrays (GH#10018) James BourbeauRemove
randomgen
support (GH#9987) Eray Aslan
Bug Fixes¶
Skip rechunk when rechunking to the same chunks with unknown sizes (GH#10027) Hendrik Makait
Custom utility to convert parquet filters to
pyarrow
expression (GH#9885) Richard (Rick) ZamoraConsider
numpy
scalars and 0d arrays as scalars when padding (GH#9653) Justus MaginFix parquet overwrite behavior after an adaptive
read_parquet
operation (GH#10002) Richard (Rick) Zamora
Maintenance¶
Remove stale hive-partitioning code from
pyarrow
parquet engine (GH#10039) Richard (Rick) ZamoraIncrease minimum supported
pyarrow
to 7.0 (GH#10024) James BourbeauRevert “Prepare drop packunpack (GH#9994) (GH#10037) Florian Jetter
Have codecov wait for more builds before reporting (GH#10031) James Bourbeau
Prepare drop packunpack (GH#9994) Florian Jetter
Add CI job with
pyarrow
strings turned on (GH#10017) James BourbeauFix
test_groupby_dropna_with_agg
forpandas
2.0 (GH#10001) Irina TruongFix
test_pickle_roundtrip
forpandas
2.0 (GH#10011) James Bourbeau
2023.3.0¶
Released on March 1, 2023
Bug Fixes¶
Bag must not pick p2p as shuffle default (GH#10005) Florian Jetter
Documentation¶
Minor follow-up to P2P by default (GH#10008) James Bourbeau
Maintenance¶
Add minimum version to optional
jinja2
dependency (GH#9999) Charles Blackmon-Luca
2023.2.1¶
Released on February 24, 2023
Note
This release changes the default DataFrame shuffle algorithm to p2p
to improve stability and performance. Learn more here
and please provide any feedback on this discussion.
If you encounter issues with this new algorithm, please see the documentation for more information, and how to switch back to the old mode.
Enhancements¶
Enable P2P shuffling by default (GH#9991) Florian Jetter
P2P rechunking (GH#9939) Hendrik Makait
Efficient dataframe.convert-string support for read_parquet (GH#9979) Irina Truong
Allow p2p shuffle kwarg for DataFrame merges (GH#9900) Florian Jetter
Change
split_row_groups
default to “infer” (GH#9637) Richard (Rick) ZamoraAdd option for converting string data to use
pyarrow
strings (GH#9926) James BourbeauAdd support for multi-column
sort_values
(GH#8263) Charles Blackmon-LucaGenerator
based random-number generation in``dask.array`` (GH#9038) Eray AslanSupport
numeric_only
for simple groupby aggregations forpandas
2.0 compatibility (GH#9889) Irina Truong
Bug Fixes¶
Fix profilers plot not being aligned to context manager enter time (GH#9739) David Hoese
Relax dask.dataframe assert_eq type checks (GH#9989) Matthew Rocklin
Restore
describe
compatibility forpandas
2.0 (GH#9982) James Bourbeau
Documentation¶
Improving deploying Dask docs (GH#9912) Sarah Charlotte Johnson
More docs for
DataFrame.partitions
(GH#9976) Tom AugspurgerUpdate docs with more information on default Delayed scheduler (GH#9903) Guillaume Eynard-Bontemps
Deployment Considerations documentation (GH#9933) Gabe Joseph
Maintenance¶
Temporarily rerun flaky tests (GH#9983) James Bourbeau
Update parsing of FULL_RAPIDS_VER/FULL_UCX_PY_VER (GH#9990) Charles Blackmon-Luca
Increase minimum supported versions to
pandas=1.3
andnumpy=1.21
(GH#9950) James BourbeauFix
std
to work withnumeric_only
forpandas
2.0 (GH#9960) Irina TruongTemporarily
xfail
test_roundtrip_partitioned_pyarrow_dataset
(GH#9977) James BourbeauFix copy on write failure in test_idxmaxmin (GH#9944) Patrick Hoefler
Bump
pre-commit
versions (GH#9955) crusaderkyFix
test_groupby_unaligned_index
forpandas
2.0 (GH#9963) Irina TruongUn-
xfail
test_set_index_overlap_2
forpandas
2.0 (GH#9959) James BourbeauFix
test_merge_by_index_patterns
forpandas
2.0 (GH#9930) Irina TruongBump jacobtomlinson/gha-find-replace from 2 to 3 (GH#9953) James Bourbeau
Fix
test_rolling_agg_aggregate
forpandas
2.0 compatibility (GH#9948) Irina TruongBump
black
to23.1.0
(GH#9956) crusaderkyRun GPU tests on python 3.8 & 3.10 (GH#9940) Charles Blackmon-Luca
Fix
test_to_timestamp
forpandas
2.0 (GH#9932) Irina TruongFix an error with
groupby
value_counts
forpandas
2.0 compatibility (GH#9928) Irina TruongConfig converter: replace all dashes with underscores (GH#9945) Jacob Tomlinson
CI: use nightly wheel to install pyarrow in upstream test build (GH#9873) Joris Van den Bossche
2023.2.0¶
Released on February 10, 2023
Enhancements¶
Update
numeric_only
default inquantile
forpandas
2.0 (GH#9854) Irina TruongMake
repartition
a no-op when divisions match (GH#9924) James BourbeauUpdate
datetime_is_numeric
behavior indescribe
forpandas
2.0 (GH#9868) Irina TruongUpdate
value_counts
to return correct name inpandas
2.0 (GH#9919) Irina TruongSupport new
axis=None
behavior inpandas
2.0 for certain reductions (GH#9867) James BourbeauFilter out all-nan
RuntimeWarning
at the chunk level fornanmin
andnanmax
(GH#9916) Julia SignellFix numeric
meta_nonempty
indexcreation
forpandas
2.0 (GH#9908) James BourbeauFix
DataFrame.info()
tests forpandas
2.0 (GH#9909) James Bourbeau
Bug Fixes¶
Fix
GroupBy.value_counts
handling for multiplegroupby
columns (GH#9905) Charles Blackmon-Luca
Documentation¶
Fix some outdated information/typos in development guide (GH#9893) Patrick Hoefler
Add note about
keep=False
indrop_duplicates
docstring (GH#9887) Jayesh MananiAdd
meta
details to dask Array (GH#9886) Jayesh MananiClarify task stream showing more rows than threads (GH#9906) Gabe Joseph
Maintenance¶
Fix
test_numeric_column_names
forpandas
2.0 (GH#9937) Irina TruongFix
dask/dataframe/tests/test_utils_dataframe.py
tests forpandas
2.0 (GH#9788) James BourbeauReplace
index.is_numeric
withis_any_real_numeric_dtype
forpandas
2.0 compatibility (GH#9918) Irina TruongAvoid
pd.core
import in dask utils (GH#9907) Matthew RoeschkeUse label for
upstream
build on pull requests (GH#9910) James BourbeauBroaden exception catching for
sqlalchemy.exc.RemovedIn20Warning
(GH#9904) James BourbeauTemporarily restrict
sqlalchemy < 2
in CI (GH#9897) James BourbeauUpdate
isort
version to 5.12.0 (GH#9895) Lawrence MitchellRemove unused
skiprows
variable inread_csv
(GH#9892) Patrick Hoefler
2023.1.1¶
Released on January 27, 2023
Enhancements¶
Add
to_backend
method toArray
and_Frame
(GH#9758) Richard (Rick) ZamoraSmall fix for timestamp index divisions in
pandas
2.0 (GH#9872) Irina TruongAdd
numeric_only
toDataFrame.cov
andDataFrame.corr
(GH#9787) James BourbeauFixes related to
group_keys
default change inpandas
2.0 (GH#9855) Irina Truonginfer_datetime_format
compatibility forpandas
2.0 (GH#9783) James Bourbeau
Bug Fixes¶
Fix serialization bug in
BroadcastJoinLayer
(GH#9871) Richard (Rick) ZamoraSatisfy
broadcast
argument inDataFrame.merge
(GH#9852) Richard (Rick) ZamoraFix
pyarrow
parquet columns statistics computation (GH#9772) aywandji
Documentation¶
Fix “duplicate explicit target name” docs warning (GH#9863) Chiara Marmo
Fix code formatting issue in “Defining a new collection backend” docs (GH#9864) Chiara Marmo
Update dashboard documentation for memory plot (GH#9768) Jayesh Manani
Add docs section about
no-worker
tasks (GH#9839) Florian Jetter
Maintenance¶
Additional updates for detecting a
distributed
scheduler (GH#9890) James BourbeauUpdate gpuCI
RAPIDS_VER
to23.04
(GH#9876)Reverse precedence between collection and
distributed
default (GH#9869) Florian JetterUpdate
xarray-contrib/issue-from-pytest-log
to version 1.2.6 (GH#9865) James BourbeauDont require dask config shuffle default (GH#9826) Florian Jetter
Un-
xfail
datetime64
Parquet roundtripping tests for newfastparquet
(GH#9811) James BourbeauAdd option to manually run
upstream
CI build (GH#9853) James BourbeauUse custom timeout in CI builds (GH#9844) James Bourbeau
Remove
kwargs
frommake_blockwise_graph
(GH#9838) Florian JetterIgnore warnings on
persist
call intest_setitem_extended_API_2d_mask
(GH#9843) Charles Blackmon-LucaFix running S3 tests locally (GH#9833) James Bourbeau
2023.1.0¶
Released on January 13, 2023
Enhancements¶
Use
distributed
default clients even if no config is set (GH#9808) Florian JetterImplement
ma.where
andma.nonzero
(GH#9760) Erik HolmgrenUpdate
zarr
store creation functions (GH#9790) Ryan Abernatheyiteritems
compatibility forpandas
2.0 (GH#9785) James BourbeauAccurate
sizeof
forpandas
string[python]
dtype (GH#9781) crusaderkyDeflate
sizeof()
of duplicate references to pandas object types (GH#9776) crusaderkyGroupBy.__getitem__
compatibility forpandas
2.0 (GH#9779) James Bourbeauappend
compatibility forpandas
2.0 (GH#9750) James Bourbeauget_dummies
compatibility forpandas
2.0 (GH#9752) James Bourbeauis_monotonic
compatibility forpandas
2.0 (GH#9751) James Bourbeaunumpy=1.24
compatability (GH#9777) James Bourbeau
Documentation¶
Remove duplicated
encoding
kwarg in docstring forto_json
(GH#9796) Sultan OrazbayevMention
SubprocessCluster
inLocalCluster
documentation (GH#9784) Hendrik MakaitMove Prometheus docs to
dask/distributed
(GH#9761) crusaderky
Maintenance¶
Temporarily ignore
RuntimeWarning
intest_setitem_extended_API_2d_mask
(GH#9828) James BourbeauFix flaky
test_threaded.py::test_interrupt
(GH#9827) Hendrik MakaitUpdate
xarray-contrib/issue-from-pytest-log
inupstream
report (GH#9822) James Bourbeaupip
install dask on gpuCI builds (GH#9816) Charles Blackmon-LucaBump
actions/checkout
from 3.2.0 to 3.3.0 (GH#9815)Resolve
sqlalchemy
import failures inmindeps
testing (GH#9809) Charles Blackmon-LucaIgnore
sqlalchemy.exc.RemovedIn20Warning
(GH#9801) Thomas Graingerxfail
datetime64
Parquet roundtripping tests forpandas
2.0 (GH#9786) James BourbeauReduce size of expected DoK sparse matrix (GH#9775) Elliott Sales de Andrade
Remove executable flag from
dask/dataframe/io/orc/utils.py
(GH#9774) Elliott Sales de Andrade
2022.12.1¶
Released on December 16, 2022
Enhancements¶
Support
dtype_backend="pandas|pyarrow"
configuration (GH#9719) James BourbeauSupport
cupy.ndarray
tocudf.DataFrame
dispatching indask.dataframe
(GH#9579) Richard (Rick) ZamoraMake filesystem-backend configurable in
read_parquet
(GH#9699) Richard (Rick) ZamoraSerialize all
pyarrow
extension arrays efficiently (GH#9740) James Bourbeau
Bug Fixes¶
Fix bug when repartitioning with
tz
-aware datetime index (GH#9741) James BourbeauPartial functions in aggs may have arguments (GH#9724) Irina Truong
Add support for simple operation with
pyarrow
-backed extension dtypes (GH#9717) James BourbeauRename columns correctly in case of
SeriesGroupby
(GH#9716) Lawrence Mitchell
Documentation¶
Update Prometheus docs (GH#9696) Hendrik Makait
Maintenance¶
Add
zarr
to Python 3.11 CI environment (GH#9771) James BourbeauAdd support for Python 3.11 (GH#9708) Thomas Grainger
Bump
actions/checkout
from 3.1.0 to 3.2.0 (GH#9753)Avoid
np.bool8
deprecation warning (GH#9737) James BourbeauMake sure dev packages aren’t overwritten in
upstream
CI build (GH#9731) James BourbeauAvoid adding
data.h5
andmydask.html
files during tests (GH#9726) Thomas Grainger
2022.12.0¶
Released on December 2, 2022
Enhancements¶
Remove statistics-based
set_index
logic fromread_parquet
(GH#9661) Richard (Rick) ZamoraAdd support for
use_nullable_dtypes
todd.read_parquet
(GH#9617) Ian RoseFix
map_overlap
in order to accept pandas arguments (GH#9571) Fabien AulaireFix pandas 1.5+
FutureWarning
in.str.split(..., expand=True)
(GH#9704) Jacob HayesEnable column projection for
groupby
slicing (GH#9667) Richard (Rick) ZamoraImprove error message for failed backend dispatch call (GH#9677) Richard (Rick) Zamora
Bug Fixes¶
Revise meta creation in arrow parquet engine (GH#9672) Richard (Rick) Zamora
Fix
da.fft.fft
for array-like inputs (GH#9688) James BourbeauFix
groupby
-aggregation when grouping on an index by name (GH#9646) Richard (Rick) Zamora
Maintenance¶
Avoid
PytestReturnNotNoneWarning
intest_inheriting_class
(GH#9707) Thomas GraingerFix flaky
test_dataframe_aggregations_multilevel
(GH#9701) Richard (Rick) ZamoraBump
mypy
version (GH#9697) crusaderkyDisable dashboard in
test_map_partitions_df_input
(GH#9687) James BourbeauUse latest
xarray-contrib/issue-from-pytest-log
inupstream
build (GH#9682) James Bourbeauxfail
ttest_1samp
for upstreamscipy
(GH#9670) James BourbeauUpdate gpuCI
RAPIDS_VER
to23.02
(GH#9678)
2022.11.1¶
Released on November 18, 2022
Enhancements¶
Restrict
bokeh=3
support (GH#9673) Gabe JosephUpdates for
fastparquet
evolution (GH#9650) Martin Durant
Maintenance¶
Update
ga-yaml-parser
step in gpuCI updating workflow (GH#9675) Charles Blackmon-LucaRevert
importlib.metadata
workaround (GH#9658) James BourbeauFix
mindeps-distributed
CI build to handlenumpy
/pandas
not being installed (GH#9668) James Bourbeau
2022.11.0¶
Released on November 15, 2022
Enhancements¶
Generalize
from_dict
implementation to allow usage from other backends (GH#9628) GALI PREM SAGAR
Bug Fixes¶
Avoid
pandas
constructors indask.dataframe.core
(GH#9570) Richard (Rick) ZamoraFix
sort_values
withTimestamp
data (GH#9642) James BourbeauGeneralize array checking and remove
pd.Index
call in_get_partitions
(GH#9634) Benjamin ZaitlenFix
read_csv
behavior forheader=0
andnames
(GH#9614) Richard (Rick) Zamora
Documentation¶
Update dashboard docs for queuing (GH#9660) Gabe Joseph
Remove
import dask as d
from docstrings (GH#9644) Matthew RocklinFix link to partitions docs in
read_parquet
docstring (GH#9636) qheuristicsAdd API doc links to
array/bag/dataframe
sections (GH#9630) Matthew Rocklin
Maintenance¶
Use
conda-incubator/setup-miniconda@v2.2.0
(GH#9662) John A KirkhamAllow
bokeh=3
(GH#9659) James BourbeauRun
upstream
build with Python 3.10 (GH#9655) James BourbeauPin
pyyaml
version in mindeps testing (GH#9640) Charles Blackmon-LucaAdd
pre-commit
to catchbreakpoint()
(GH#9638) James BourbeauBump
xarray-contrib/issue-from-pytest-log
from 1.1 to 1.2 (GH#9635)Remove
blosc
references (GH#9625) Naty ClementiUpgrade
mypy
and drop unused comments (GH#9616) Hendrik MakaitHarden
test_repartition_npartitions
(GH#9585) Richard (Rick) Zamora
2022.10.2¶
Released on October 31, 2022
This was a hotfix and has no changes in this repository. The necessary fix was in dask/distributed, but we decided to bump this version number for consistency.
2022.10.1¶
Released on October 28, 2022
Enhancements¶
Add extension dtype support to
set_index
(GH#9566) James BourbeauRedesigning the array HTML repr for clarity (GH#9519) Shingo OKAWA
Documentation¶
Add note about limiting thread oversubscription by default (GH#9592) James Bourbeau
Use
sphinx-click
fordask
CLI (GH#9589) James BourbeauFix Semaphore API docs (GH#9584) James Bourbeau
Render meta description in
map_overlap
docstring (GH#9568) James Bourbeau
Maintenance¶
Require Click 7.0+ in Dask (GH#9595) John A Kirkham
Temporarily restrict
bokeh<3
(GH#9607) James BourbeauResolve
importlib
-related failures inupstream
CI (GH#9604) Charles Blackmon-LucaImprove
upstream
CI report (GH#9603) James BourbeauFix
upstream
CI report (GH#9602) James BourbeauRemove
setuptools
host dep, add CLI entrypoint (GH#9600) Charles Blackmon-LucaMore
Backend
dispatch class type annotations (GH#9573) Ian Rose
2022.10.0¶
Released on October 14, 2022
New Features¶
Backend library dispatching for IO in Dask-Array and Dask-DataFrame (GH#9475) Richard (Rick) Zamora
Add new CLI that is extensible (GH#9283) Doug Davis
Enhancements¶
Fix array copy not being a no-op (GH#9555) David Hoese
Add support for string timedelta in
map_overlap
(GH#9559) Nicolas GrandemangeShuffle-based groupby for single functions (GH#9504) Ian Rose
Make
datetime.datetime
tokenize idempotantly (GH#9532) Martin Durant
Bug Fixes¶
Avoid race condition in lazy dispatch registration (GH#9545) James Bourbeau
Do not allow setitem to
np.nan
forint
dtype (GH#9531) Doug DavisFix project CSV columns when selecting (GH#9534) Martin Durant
Documentation¶
Update Parquet best practice (GH#9537) Matthew Rocklin
Maintenance¶
Restrict
tiledb-py
version to avoid CI failures (GH#9569) James BourbeauBump
actions/github-script
from 3 to 6 (GH#9564)Bump
actions/stale
from 4 to 6 (GH#9551)Bump
peter-evans/create-pull-request
from 3 to 4 (GH#9550)Bump
actions/checkout
from 2 to 3.1.0 (GH#9552)Bump
codecov/codecov-action
from 1 to 3 (GH#9549)Bump
the-coding-turtle/ga-yaml-parser
from 0.1.1 to 0.1.2 (GH#9553)Move dependabot configuration file (GH#9547) James Bourbeau
Add dependabot for GitHub actions (GH#9542) James Bourbeau
Run mypy on Windows and Linux (GH#9530) crusaderky
Update gpuCI
RAPIDS_VER
to22.12
(GH#9524)
2022.9.2¶
Released on September 30, 2022
Enhancements¶
Remove factorization logic from array auto chunking (GH#9507) James Bourbeau
Documentation¶
Add docs on running Dask in a standalone Python script (GH#9513) James Bourbeau
Clarify custom-graph multiprocessing example (GH#9511) nouman
2022.9.1¶
Released on September 16, 2022
New Features¶
Add
DataFrame
andSeries
median
methods (GH#9483) James Bourbeau
Enhancements¶
Filter by list (GH#9419) Greg Hayes
Added
distributed.utils.key_split
functionality todask.utils.key_split
(GH#9464) Luke Conibear
Bug Fixes¶
Fix overlap so that
set_index
doesn’t drop rows (GH#9423) Julia SignellFix assigning pandas
Series
to column whenddf.columns.min()
raises (GH#9485) Erik WelchFix metadata comparison
stack_partitions
(GH#9481) James BourbeauProvide default for
split_out
(GH#9493) Lawrence Mitchell
Deprecations¶
Documentation¶
Fixing
enforce_metadata
documentation, not checking for dtypes (GH#9474) Nicolas GrandemangeFix
it's
–>its
typo (GH#9484) Nat Tabris
Maintenance¶
Workaround for parquet writing failure using some datetime series but not others (GH#9500) Ian Rose
Filter out
numeric_only
warnings frompandas
(GH#9496) James BourbeauAvoid
set_index(..., inplace=True)
where not necessary (GH#9472) James BourbeauAvoid passing groupby key list of length one (GH#9495) James Bourbeau
Update
test_groupby_dropna_cudf
based oncudf
support forgroup_keys
(GH#9482) James BourbeauRemove
dd.from_bcolz
(GH#9479) James BourbeauAdded
flake8-bugbear
topre-commit
hooks (GH#9457) Luke ConibearBind loop variables in function definitions (
B023
) (GH#9461) Luke ConibearAdded assert for comparisons (
B015
) (GH#9459) Luke ConibearSet top-level default shell in CI workflows (GH#9469) James Bourbeau
Removed unused loop control variables (
B007
) (GH#9458) Luke ConibearReplaced
getattr
calls for constant attributes (B009
) (GH#9460) Luke ConibearPin
libprotobuf
to allow nightlypyarrow
in the upstream CI build (GH#9465) Joris Van den BosscheReplaced mutable data structures for default arguments (
B006
) (GH#9462) Luke ConibearChanged
flake8
mirror and updated version (GH#9456) Luke Conibear
2022.9.0¶
Released on September 2, 2022
Enhancements¶
Enable automatic column projection for
groupby
aggregations (GH#9442) Richard (Rick) ZamoraAccept superclasses in NEP-13/17 dispatching (GH#6710) Gabe Joseph
Bug Fixes¶
Rename
by
columns internally for cumulative operations on the sameby
columns (GH#9430) Pavithra EswaramoorthyFix
get_group
with categoricals (GH#9436) Pavithra EswaramoorthyFix caching-related
MaterializedLayer.cull
performance regression (GH#9413) Richard (Rick) Zamora
Documentation¶
Add maintainer documentation page (GH#9309) James Bourbeau
Maintenance¶
Revert skipped fastparquet test (GH#9439) Pavithra Eswaramoorthy
tmpfile
does not end files with period on empty extension (GH#9429) Hendrik MakaitSkip failing fastparquet test with latest release (GH#9432) James Bourbeau
2022.8.1¶
Released on August 19, 2022
New Features¶
Implement
ma.*_like functions
(GH#9378) Ruth Comer
Enhancements¶
Shuffle-based groupby aggregation for high-cardinality groups (GH#9302) Richard (Rick) Zamora
Unpack
namedtuple
(GH#9361) Hendrik Makait
Bug Fixes¶
Fix
SeriesGroupBy
cumulative functions withaxis=1
(GH#9377) Pavithra EswaramoorthyFix
make_meta
while using categorical column with index (GH#9348) Pavithra EswaramoorthyDon’t allow incompatible keywords in
DataFrame.dropna
(GH#9366) Naty ClementiMake
set_index
handle entirely empty dataframes (GH#8896) Julia SignellImprove
dataclass
handling inunpack_collections
(GH#9345) Hendrik MakaitFix bag sampling when there are some smaller partitions (GH#9349) Ian Rose
Add support for empty partitions to
da.min
/da.max
functions (GH#9268) geraninam
Documentation¶
Clarify that
bind()
etc. regenerate the keys (GH#9385) crusaderkyConsolidate dashboard diagnostics documentation (GH#9357) Sarah Charlotte Johnson
Remove outdated
meta
information Pavithra Eswaramoorthy
Maintenance¶
Use
entry_points
utility insizeof
(GH#9390) James BourbeauAdd
entry_points
compatibility utility (GH#9388) Jacob TomlinsonUpload environment file artifact for each CI build (GH#9372) James Bourbeau
Remove
werkzeug
pin in CI (GH#9371) James BourbeauFix type annotations for
dd.from_pandas
anddd.from_delayed
(GH#9362) Jordan Yap
2022.8.0¶
Released on August 5, 2022
Enhancements¶
Ensure
make_meta
doesn’t hold ref to data (GH#9354) Jim Crist-HarifRevise
divisions
logic infrom_pandas
(GH#9221) Richard (Rick) ZamoraWarn if user sets index with existing index (GH#9341) Julia Signell
Add
keepdims
keyword forda.average
(GH#9332) Ruth ComerChange
repr
methods to avoidLayer
materialization (GH#9289) Richard (Rick) Zamora
Bug Fixes¶
Make sure
order
kwarg will not crash theastype
method (GH#9317) Genevieve BuckleyFix bug for
cumsum
on cupy chunked dask arrays (GH#9320) Genevieve BuckleyMatch input and output structure in
_sample_reduce
(GH#9272) Pavithra EswaramoorthyInclude
meta
in array serialization (GH#9240) Frédéric BRIOLFix
Index.memory_usage
(GH#9290) James BourbeauFix division calculation in
dask.dataframe.io.from_dask_array
(GH#9282) Jordan Yap
Documentation¶
Fow to use kwargs with custom task graphs (GH#9322) Genevieve Buckley
Add note to
da.from_array
about how the order is not preserved (GH#9346) Julia SignellAdd I/O info for async functions (GH#9326) Logan Norman
Tidy up docs snippet for futures IO functions (GH#9340) Julia Signell
Use consistent variable names for pandas
df
and Daskddf
indataframe-groupby.rst
(GH#9304) ivojuroroSwitch
js-yaml
foryaml.js
in config converter (GH#9306) Jacob Tomlinson
Maintenance¶
Update
da.linalg.solve
for SciPy 1.9.0 compatibility (GH#9350) Pavithra EswaramoorthyUpdate
test_getitem_avoids_large_chunks_missing
(GH#9347) Pavithra EswaramoorthyFix docs title formatting for “Extend
sizeof
” Doug DavisImport
loop_in_thread
fixture in tests (GH#9337) James BourbeauTemporarily
xfail
test_solve_sym_pos
(GH#9336) Pavithra EswaramoorthyFix small typo in 10 minutes to Dask page (GH#9329) Shaghayegh
Temporarily pin
werkzeug
in CI to avoid test suite hanging (GH#9325) James BourbeauAdd tests for
cupy.angle()
(GH#9312) Peter Andreas EntschevUpdate gpuCI
RAPIDS_VER
to22.10
(GH#9314)Add
pandas[test]
totest
extra (GH#9110) Ben BeasleyAdd
bokeh
andscipy
toupstream
CI build (GH#9265) James Bourbeau
2022.7.1¶
Released on July 22, 2022
Enhancements¶
Return Dask array if all axes are squeezed (GH#9250) Pavithra Eswaramoorthy
Make cycle reported by toposort shorter (GH#9068) Erik Welch
Unknown chunk slicing - raise informative error (GH#9285) Naty Clementi
Bug Fixes¶
Fix bug in
HighLevelGraph.cull
(GH#9267) Richard (Rick) ZamoraSort categories (GH#9264) Pavithra Eswaramoorthy
Use
max
(instead ofsum
) for calculatingwarnsize
(GH#9235) Pavithra EswaramoorthyFix bug when filtering on partitioned column with pyarrow (GH#9252) Richard (Rick) Zamora
Documentation¶
Updated repartition documentation to add note about
partition_size
(GH#9288) Dylan StewartDon’t include docs in
Array
methods, just refer to module docs (GH#9244) Julia SignellRemove outdated reference to scheduler and worker dashboards (GH#9278) Pavithra Eswaramoorthy
Adds an custom aggregate example using numpy methods (GH#9260) geraninam
Maintenance¶
Add type annotations to
dd.from_pandas
anddd.from_delayed
(GH#9237) Michael MiltonUpdate
calculate_divisions
docstring (GH#9275) Tom AugspurgerUpdate
test_plot_multiple
for upcomingbokeh
release (GH#9261) James Bourbeau
2022.7.0¶
Released on July 8, 2022
Enhancements¶
Support
pathlib.PurePath
innormalize_token
(GH#9229) Angus HollandsAdd
AttributeNotImplementedError
for properties so IPython glob search works (GH#9231) Erik Welchmap_overlap
: multiple dataframe handling (GH#9145) Fabien AulaireRead entrypoints in
dask.sizeof
(GH#7688) Angus Hollands
Bug Fixes¶
Fix
TypeError: 'Serialize' object is not subscriptable
when writing parquet dataset withClient(processes=False)
(GH#9015) Lucas Miguel PonceCorrect dtypes when
concat
with an empty dataframe (GH#9193) Pavithra Eswaramoorthy
Documentation¶
Highlight note about persist (GH#9234) Pavithra Eswaramoorthy
Update release-procedure to include more detail and helpful commands (GH#9215) Julia Signell
Better SEO for Futures and Dask vs. Spark pages (GH#9217) Sarah Charlotte Johnson
Maintenance¶
Use
math.prod
instead ofnp.prod
on lists, tuples, and iters (GH#9232) crusaderkyOnly import IPython if type checking (GH#9230) Florian Jetter
Tougher mypy checks (GH#9206) crusaderky
2022.6.1¶
Released on June 24, 2022
Enhancements¶
Create
dask.utils.show_versions
(GH#9144) Sultan OrazbayevBetter error message for unsupported numpy operations on dask.dataframe objects. (GH#9201) Julia Signell
Add
allow_rechunk
kwarg todask.array.overlap
function (GH#7776) Genevieve BuckleyAdd minutes and hours to
dask.utils.format_time
(GH#9116) Matthew RocklinMore retries when writing parquet to remote filesystem (GH#9175) Ian Rose
Bug Fixes¶
Timedelta deterministic hashing (GH#9213) Fabien Aulaire
Enum deterministic hashing (GH#9212) Fabien Aulaire
shuffle_group()
: avoid converting to arrays (GH#9157) Mads R. B. Kristensen
Deprecations¶
Deprecate extra
format_time
utility (GH#9184) James Bourbeau
Documentation¶
Better SEO for 10 Minutes to Dask (GH#9182) Sarah Charlotte Johnson
Better SEO for Delayed and Best Practices (GH#9194) Sarah Charlotte Johnson
Include known inconsistency in DataFrame
str.split
accessor docstring (GH#9177) Richard PelgrimAdd
inconsistencies
keyword toderived_from
(GH#9192) Richard PelgrimAdd missing
append
indelayed
best practices example (GH#9202) BenFix indentation in Best Practices (GH#9196) Sarah Charlotte Johnson
Add link to Genevieve Buckley’s blog on chunk sizes (GH#9199) Pavithra Eswaramoorthy
Update
to_csv
docstring (GH#9094) Sarah Charlotte Johnson
Maintenance¶
Update versioneer: change from using
SafeConfigParser
toConfigParser
(GH#9205) Thomas A CaswellRemove ipython hack in CI(GH#9200) crusaderky
2022.6.0¶
Released on June 10, 2022
Enhancements¶
Add feature to show names of layer dependencies in HLG JupyterLab repr (GH#9081) Angelos Omirolis
Add arrow schema extraction dispatch (GH#9169) GALI PREM SAGAR
Add
sort_results
argument toassert_eq
(GH#9130) Pavithra EswaramoorthyAdd weeks to
parse_timedelta
(GH#9168) Matthew RocklinWarn that cloudpickle is not always deterministic (GH#9148) Pavithra Eswaramoorthy
Switch parquet default engine (GH#9140) Jim Crist-Harif
Use deterministic hashing with
_iLocIndexer
/_LocIndexer
(GH#9108) Fabien AulaireEnfore consistent schema in
to_parquet
pyarrow (GH#9131) Jim Crist-Harif
Bug Fixes¶
Fix
pyarrow.StringArray
pickle (GH#9170) Jim Crist-HarifFix parallel metadata collection in pyarrow engine (GH#9165) Richard (Rick) Zamora
Improve
pyarrow
partitioning logic (GH#9147) James Bourbeaupyarrow
8.0 partitioning fix (GH#9143) James Bourbeau
Documentation¶
Better SEO for Installing Dask and Dask DataFrame Best Practices (GH#9178) Sarah Charlotte Johnson
Update logos page in docs (GH#9167) Sarah Charlotte Johnson
Add example using pandas Series to
map_partition
doctring (GH#9161) Alex-JG3Update docs theme for rebranding (GH#9160) Sarah Charlotte Johnson
Better SEO for docs on Dask DataFrames (GH#9128) Sarah Charlotte Johnson
Maintenance¶
Remove ensure_file from recommended practice for downstream libraries (GH#9171) Matthew Rocklin
Test round-tripping DataFrame parquet I/O including pyspark (GH#9156) Ian Rose
Link best practices to DataFrame-parquet (GH#9150) Tom Augspurger
Fix typo in
map_partitions
func
parameter description (GH#9149) Christopher AkikiUn-
xfail
test_groupby_grouper_dispatch
(GH#9139) GALI PREM SAGARTemporarily import cleanup fixture from distributed (GH#9138) James Bourbeau
Simplify partitioning logic in pyarrow parquet engine (GH#9041) Richard (Rick) Zamora
2022.05.2¶
Released on May 26, 2022
Enhancements¶
Add a dispatch for non-pandas
Grouper
objects and use it inGroupBy
(GH#9074) brandon-b-millerError if
read_parquet
&to_parquet
files intersect (GH#9124) Jim Crist-Harif
Documentation¶
Fix various typos (GH#9126) Ryan Russell
Maintenance¶
Fix flaky
test_filter_nonpartition_columns
(GH#9127) Pavithra EswaramoorthyUpdate gpuCI
RAPIDS_VER
to22.08
(GH#9120)Include
conftest.py`
in sdists (GH#9115) Ben Beasley
2022.05.1¶
Released on May 24, 2022
New Features¶
Add
DataFrame.from_dict
classmethod (GH#9017) Matthew PowersAdd
from_map
function to Dask DataFrame (GH#8911) Richard (Rick) Zamora
Enhancements¶
Improve
to_parquet
error for appended divisions overlap (GH#9102) Jim Crist-HarifEnabled user-defined process-initializer functions (GH#9087) ParticularMiner
Mention
align_dataframes=False
option inmap_partitions
error (GH#9075) Gabe JosephAdd kwarg
enforce_ndim
todask.array.map_blocks()
(GH#8865) ParticularMinerImplement
Series.GroupBy.fillna
/DataFrame.GroupBy.fillna
methods (GH#8869) Pavithra EswaramoorthyAllow
fillna
with Dask DataFrame (GH#8950) Pavithra EswaramoorthyUpdate error message for assignment with 1-d dask array (GH#9036) Pavithra Eswaramoorthy
Collection Protocol (GH#8674) Doug Davis
Patch around
pandas
ArrowStringArray
pickling (GH#9024) Jim Crist-HarifAdd
p2p
shuffle option (GH#8836) Matthew Rocklin
Bug Fixes¶
Fixup column projection with no columns (GH#9106) Jim Crist-Harif
Fix column-projection bug in
from_map
(GH#9078) Richard (Rick) ZamoraPrevent nulls in index for non-numeric dtypes (GH#8963) Jorge López
Fix
is_monotonic
methods for more than 8 partitions (GH#9019) Julia SignellHandle enumerate and generator inputs to
from_map
(GH#9066) Richard (Rick) ZamoraRevert
is_dask_collection
; back to previous implementation (GH#9062) Doug DavisFix
Blockwise.clone
does not handle iterable literal arguments correctly (GH#8979) JSKenyonArray
setitem
hardmask (GH#9027) David HassellFix overlapping divisions error on append (GH#8997) Ian Rose
Deprecations¶
Add pre-deprecation warnings for
read_parquet
kwargschunksize
andaggregate_files
(GH#9052) Richard (Rick) Zamora
Documentation¶
Document
map_partitions
handling ofargs
vskwargs
, usage ofpartition_info
(GH#9084) Charles Blackmon-LucaUpdate custom collection documentation (leverage new collection protocol) (GH#9097) Doug Davis
Better SEO for docs on creating and storing Dask DataFrames (GH#9098) Sarah Charlotte Johnson
Clarify chunking in
imread
docstring (GH#9082) Genevieve BuckleyRearrange docs TOC (GH#9001) Matthew Rocklin
Corrected
map_blocks()
docstring for kwargenforce_ndim
(GH#9071) ParticularMinerUpdate DataFrame SQL docs references to other libraries (GH#9077) Charles Blackmon-Luca
Update page on creating and storing Dask DataFrames (GH#9025) Sarah Charlotte Johnson
Maintenance¶
Include
NUMPY_LICENSE.txt
in license files (GH#9113) Ben BeasleyIncrease retries when installing nightly
pandas
(GH#9103) James BourbeauForce nightly
pyarrow
in the upstream build (GH#9095) Joris Van den BosscheImprove object handling & testing of
ensure_unicode
(GH#9059) John A KirkhamForce nightly
pyarrow
in the upstream build (GH#8993) Joris Van den BosscheAdditional check on
is_dask_collection
(GH#9054) Doug DavisUpdate
ensure_bytes
(GH#9050) John A KirkhamAdd end of file pre-commit hook (GH#9045) James Bourbeau
Add
codespell
pre-commit hook (GH#9040) James BourbeauRemove the HDFS tests (GH#9039) Jim Crist-Harif
Fix flaky
test_reductions_2D
(GH#9037) Jim Crist-HarifPrevent codecov from notifying of failure too soon (GH#9031) Jim Crist-Harif
Only test on Python 3.9 on macos (GH#9029) Jim Crist-Harif
Update
to_timedelta
default unit (GH#9010) Pavithra Eswaramoorthy
2022.05.0¶
Released on May 2, 2022
Highlights¶
This is a bugfix release for this issue.
Documentation¶
Add highlights section to 2022.04.2 release notes (GH#9012) James Bourbeau
2022.04.2¶
Released on April 29, 2022
Highlights¶
This release includes several deprecations/breaking API changes to
dask.dataframe.read_parquet
and dask.dataframe.to_parquet
:
to_parquet
no longer writes_metadata
files by default. If you want to write a_metadata
file, you can pass inwrite_metadata_file=True
.read_parquet
now defaults tosplit_row_groups=False
, which results in one Dask dataframe partition per parquet file when reading in a parquet dataset. If you’re working with large parquet files you may need to setsplit_row_groups=True
to reduce your partition size.read_parquet
no longer calculates divisions by default. If you requireread_parquet
to return dataframes with known divisions, please setcalculate_divisions=True
.read_parquet
has deprecated thegather_statistics
keyword argument. Please use thecalculate_divisions
keyword argument instead.read_parquet
has deprecated therequire_extensions
keyword argument. Please use theparquet_file_extension
keyword argument instead.
New Features¶
Add
removeprefix
andremovesuffix
asStringMethods
(GH#8912) Jorge López
Enhancements¶
Call
fs.invalidate_cache
into_parquet
(GH#8994) Jim Crist-HarifChange
to_parquet
default towrite_metadata_file=None
(GH#8988) Jim Crist-HarifLet arg reductions pass
keepdims
(GH#8926) Julia SignellChange
split_row_groups
default toFalse
inread_parquet
(GH#8981) Richard (Rick) ZamoraImprove
NotImplementedError
message forda.reshape
(GH#8987) Jim Crist-HarifSimplify
to_parquet
compute path (GH#8982) Jim Crist-HarifRaise an error if you try to use
vindex
with a Dask object (GH#8945) Julia SignellAvoid
pre_buffer=True
when a precache method is specified (GH#8957) Richard (Rick) Zamorafrom_dask_array
usesblockwise
instead of merging graphs (GH#8889) Bryan WeberUse
pre_buffer=True
for “pyarrow” Parquet engine (GH#8952) Richard (Rick) Zamora
Bug Fixes¶
Fix
dask-sql
bug caused byblockwise
fusion (GH#8989) Richard (Rick) Zamorato_parquet
errors for non-string column names (GH#8990) Jim Crist-HarifMake sure
da.roll
works even if shape is 0 (GH#8925) Julia SignellFix recursion error issue with
set_index
(GH#8967) Paul HobsonStringify
BlockwiseDepDict
mapping values whenproduces_keys=True
(GH#8972) Richard (Rick) ZamoraUse DataFram`eIOLayer in
DataFrame.from_delayed
(GH#8852) Richard (Rick) ZamoraCheck that values for the
in
predicate inread_parquet
are correct (GH#8846) Bryan WeberFix bug for reduction of zero dimensional arrays (GH#8930) Tom White
Specify
dtype
when deciding division usingnp.linspace
inread_sql_query
(GH#8940) Cheun Hong
Deprecations¶
Deprecate
gather_statistics
fromread_parquet
(GH#8992) Richard (Rick) ZamoraChange
require_extension
to top-levelparquet_file_extension
read_parquet
kwarg (GH#8935) Richard (Rick) Zamora
Documentation¶
Update
write_metadata_file
discussion in documentation (GH#8995) Richard (Rick) ZamoraUpdate
DataFrame.merge
docstring (GH#8966) Pavithra EswaramoorthyAdded description for parameter
align_arrays
inarray.blockwise()
(GH#8977) ParticularMinerecommend not to use
map_block(drop_axis=...)
on chunked axes of an array (GH#8921) ParticularMinerAdd copy button to code snippets in docs (GH#8956) James Bourbeau
Maintenance¶
Add
pytest-timeout
to distributed envs on CI (GH#8986) Julia SignellImprove
read_parquet
docstring formatting (GH#8971) Bryan WeberRemove
pytest.warns(None)
(GH#8924) Pavithra EswaramoorthyDocument Python 3.10 as supported (GH#8976) Eray Aslan
parse_timedelta
option to enforce explicit unit (GH#8969) crusaderkymypy
compatibility (GH#8854) Paul HobsonAdd a docs page for Dask & Parquet (GH#8899) Jim Crist-Harif
Adds configuration to ignore revs in blame (GH#8933) Bryan Weber
2022.04.1¶
Released on April 15, 2022
Enhancements¶
Avoid collecting parquet metadata in pyarrow when
write_metadata_file=False
(GH#8906) Richard (Rick) ZamoraBetter error for failed wildcard path in
dd.read_csv()
(fixes #8878) (GH#8908) Roger FilmyerReturn
da.Array
rather thandd.Series
for non-ufunc elementwise functions ondd.Series
(GH#8558) Julia SignellLet
get_dummies
usemeta
computation inmap_partitions
(GH#8898) Julia SignellMasked scalars input to
da.from_array
(GH#8895) David HassellRaise
ValueError
inmerge_asof
for duplicatekwargs
(GH#8861) Bryan Weber
Bug Fixes¶
Make
is_monotonic
work when some partitions are empty (GH#8897) Julia SignellFix custom getter in
da.from_array
wheninline_array=False
(GH#8903) Ian RoseCorrectly handle dict-specification for rechunk. (GH#8859) Richard
Fix
merge_asof
: drop index column ifleft_on == right_on
(GH#8874) Gil Forsyth
Deprecations¶
Warn users that
engine='auto'
will change in future (GH#8907) Jim Crist-HarifRemove
pyarrow-legacy
engine from parquet API (GH#8835) Richard (Rick) Zamora
Documentation¶
Add note on missing parameter
out
fordask.array.dot
(GH#8913) Francesco AndreuzziUpdate
DataFrame.query
docstring (GH#8890) Pavithra Eswaramoorthy
Maintenance¶
Don’t test
da.prod
on large integer data (GH#8893) Jim Crist-HarifAdd
network
marks to tests that fail without an internet connection (GH#8881) Paul HobsonFix gpuCI GHA version (GH#8891) Charles Blackmon-Luca
xfail
/skip
some flakydistributed
tests (GH#8887) Jim Crist-HarifRemove unused (deprecated) code from
ArrowDatasetEngine
(GH#8885) Richard (Rick) ZamoraAdd mild typing to common utils functions, part 2 (GH#8867) crusaderky
Documentation of Limitation of
sample()
(GH#8858) Nadiem Sissouno
2022.04.0¶
Released on April 1, 2022
Note
This is the first release with support for Python 3.10
New Features¶
Add Python 3.10 support (GH#8566) James Bourbeau
Enhancements¶
Add check on
dtype.itemsize
in order to produce a useful error (GH#8860) Davide GavioAdd mild typing to common utils functions (GH#8848) Matthew Rocklin
Add sanity checks to
divisions
setter
(GH#8806) Jim Crist-HarifUse
Blockwise
andmap_partitions
for more tasks (GH#8831) Bryan Weber
Bug Fixes¶
Fix
dataframe.merge_asof
to preserveright_on
column (GH#8857) Sarah Charlotte JohnsonFix “Buffer dtype mismatch” for pandas >= 1.3 on 32bit (GH#8851) Ben Greiner
Fix slicing fusion by altering
SubgraphCallable
getter
(GH#8827) Ian Rose
Deprecations¶
Remove support for PyPy (GH#8863) James Bourbeau
Drop
setuptools
at runtime (GH#8855) crusaderkyRemove
dataframe.tseries.resample.getnanos
(GH#8834) Sarah Charlotte Johnson
Documentation¶
Organize diagnostic and performance docs (GH#8871) Naty Clementi
Add image to explain
drop_axis
option ofmap_blocks
(GH#8868) ParticularMiner
Maintenance¶
Update gpuCI
RAPIDS_VER
to22.06
(GH#8828)Restore
test_parquet
in http (GH#8850) Bryan WeberSimplify gpuCI updating workflow (GH#8849) Charles Blackmon-Luca
2022.03.0¶
Released on March 18, 2022
New Features¶
Bag: add implementation for reservoir sampling (GH#7636) Daniel Mesejo-León
Add
ma.count
to Dask array (GH#8785) David HassellChange
to_parquet
default tocompression="snappy"
(GH#8814) Jim Crist-HarifAdd
weights
parameter todask.array.reduction
(GH#8805) David HassellAdd
ddf.compute_current_divisions
to get divisions on a sorted index or column (GH#8517) Julia Signell
Enhancements¶
Pass
__name__
and__doc__
through on DelayedLeaf (GH#8820) Leo GaoRaise exception for not implemented merge
how
option (GH#8818) Naty ClementiMove
Bag.map_partitions
toBlockwise
(GH#8646) Richard (Rick) ZamoraImprove error messages for malformed config files (GH#8801) Jim Crist-Harif
Revise column-projection optimization to capture common dask-sql patterns (GH#8692) Richard (Rick) Zamora
Useful error for empty divisions (GH#8789) Pavithra Eswaramoorthy
Scipy 1.8.0 compat: copy private classes into dask/array/stats.py (GH#8694) Julia Signell
Raise warning when using multiple types of schedulers where one is
distributed
(GH#8700) Pedro Silva
Bug Fixes¶
Fix bug in applying != filter in
read_parquet
(GH#8824) Richard (Rick) ZamoraFix
set_index
when directly passed a dask Index (GH#8680) Paul HobsonQuick fix for unbounded memory usage in tensordot (GH#7980) Genevieve Buckley
If hdf file is empty, don’t fail on meta creation (GH#8809) Julia Signell
Update
clone_key("x")
to retain prefix (GH#8792) crusaderkyFix “physical” column bug in pyarrow-based
read_parquet
(GH#8775) Richard (Rick) ZamoraFix
groupby.shift
bug caused by unsorted partitions after shuffle (GH#8782) kori73Fix serialization bug (GH#8786) Richard (Rick) Zamora
Deprecations¶
Bump diagnostics bokeh dependency to 2.4.2 (GH#8791) Charles Blackmon-Luca
Deprecate
bcolz
support (GH#8754) Pavithra EswaramoorthyFinish making
map_overlap
default boundarykwarg
'none'
(GH#8743) Genevieve Buckley
Documentation¶
Custom collection example docs fix (GH#8807) Doug Davis
Add
Series.str
,Series.dt
, andSeries.cat
accessors to docs (GH#8757) Sarah Charlotte JohnsonFix docstring for
ddf.compute_current_divisions
(GH#8793) Julia SignellDashboard docs on /status page (GH#8648) Naty Clementi
Clarify divisions
kwarg
in repartition docstring (GH#8781) Sarah Charlotte JohnsonUpdate Docker images to use ghcr.io (GH#8774) Jacob Tomlinson
Maintenance¶
Reduce gpuci
pytest
parallelism (GH#8826) GALI PREM SAGARabsolufy-imports
- No relative imports - PEP8 (GH#8796) Julia SignellTidy up
assert_eq
calls in array tests (GH#8812) Julia SignellFix
test_describe_empty
to work without global-Werror
(GH#8291) Michał GórnyTemporarily xfail graphviz tests on windows (GH#8794) Jim Crist-Harif
Use
packaging.parse
formd5
compatibility (GH#8763) James BourbeauMake
tokenize
work in a FIPS 140-2 environment (GH#8762) Jim Crist-HarifLabel issues and PRs on open with ‘needs triage’ (GH#8761) Julia Signell
Specify action version and change from
pull_request_target
topull_request
(GH#8767) Julia SignellMake scheduler
kwarg
pass though to sub functions inda.assert_eq
(GH#8755) Julia Signell
2022.02.1¶
Released on February 25, 2022
New Features¶
Add aggregate functions
first
andlast
todask.dataframe.pivot_table
(GH#8649) Knut NordangerAdd
std()
support fordatetime64
dtype
for pandas-like objects (GH#8523) Ben GlossnerAdd materialized task counts to
HighLevelGraph
andLayer
html reprs (GH#8589) kori73
Enhancements¶
Do not allow iterating a
DataFrameGroupBy
(GH#8696) Bryan WeberFix missing newline after
info()
call on emptyDataFrame
(GH#8727) Naty ClementiAdd
groupby.compute
as a not implemented method (GH#8734) DranaxelImprove multi dataframe join performance (GH#8740) Holden Karau
Include
bool
type forIndex
(GH#8732) Naty ClementiAllow
ArrowDatasetEngine
subclass to override pandas->arrow conversion also for partitioned write (GH#8741) Joris Van den BosscheIncrease performance of k-diagonal extraction in
da.diag()
andda.diagonal()
(GH#8689) ParticularMinerChange
linspace
creation to match numpy when num equal to 0 (GH#8676) PeterTokenize
dataclasses
(GH#8557) Gabe JosephUpdate
tokenize
to treatdict
andkwargs
differently (GH#8655) James Bourbeau
Bug Fixes¶
Fix bug in
dask.array.roll()
for roll-shifts that match the size of the input array (GH#8723) ParticularMinerFix for
normalize_function
dataclass
methods (GH#8527) Sarah Charlotte JohnsonFix rechunking with zero-size-chunks (GH#8703) ParticularMiner
Move creation of
sqlalchemy
connection for picklability (GH#8745) Julia Signell
Deprecations¶
Drop Python 3.7 (GH#8572) James Bourbeau
Deprecate
iteritems
(GH#8660) James BourbeauDeprecate
dataframe.tseries.resample.getnanos
(GH#8752) Sarah Charlotte JohnsonAdd deprecation warning for pyarrow-legacy engine (GH#8758) Richard (Rick) Zamora
Documentation¶
Update link typos in changelog (GH#8717) James Bourbeau
Update Docker example to use current best practices (GH#8731) Jacob Tomlinson
Update docs to include
distributed.Client.preload
(GH#8679) Bryan WeberDocument monthly social meeting (GH#8595) Thomas Grainger
Add docs for Gen2 access with RBAC/ACL i.e. security principal (GH#8748) Martin Thøgersen
Use Dask configuration extension from
dask-sphinx-theme
(GH#8751) Benjamin Zaitlen
Maintenance¶
Unpin
coverage
in CI (GH#8690) James BourbeauAdd manual trigger for running test suite (GH#8716) James Bourbeau
Xfail
scheduler_HLG_unpack_import
; flaky test (GH#8724) Mike McCartyTemporarily remove
scipy
upstream CI build (GH#8725) James BourbeauBump pre-release version to be greater than stable releases (GH#8728) Charles Blackmon-Luca
Move custom sort function logic to internal
sort_values
(GH#8571) Charles Blackmon-LucaPin
cloudpickle
andscipy
in docs requirements (GH#8737) Julia SignellMake the labeler not delete labels, and look for the docs at the right spot (GH#8746) Julia Signell
Fix docs build warnings (GH#8432) Kristopher Overholt
Update test status badge (GH#8747) James Bourbeau
Fix parquet
test_pandas_timestamp_overflow_pyarrow
test (GH#8733) Joris Van den BosscheOnly run PR builds on changes to relevant files (GH#8756) Charles Blackmon-Luca
2022.02.0¶
Released on February 11, 2022
Note
This is the last release with support for Python 3.7
New Features¶
Add
region
toto_zarr
when using existing array (GH#8590) Chris RoatAdd
engine_kwargs
support todask.dataframe.to_sql
(GH#8609) Amir KadivarAdd
include_path_column
arg toread_json
(GH#8603) Bryan Weber
Enhancements¶
Add scheduler option to
assert_eq
utilities (GH#8610) Xinrong MengFix eye inconsistency with NumPy for
dtype=None
(GH#8685) Tom WhiteFix concatenate inconsistency with NumPy for
axis=None
(GH#8686) Tom WhiteType annotations, part 1 (GH#8295) crusaderky
Really allow any iterable to be passed as a
meta
(GH#8629) Julia SignellUse
map_partitions
(Blockwise) into_parquet
(GH#8487) Richard (Rick) Zamora
Bug Fixes¶
Result of reducing an array should not depend on its chunk-structure (GH#8637) ParticularMiner
Pass place-holder metadata to
map_partitions
in ACA code path (GH#8643) Richard (Rick) Zamora
Deprecations¶
Deprecate
is_monotonic
(GH#8653) James BourbeauRemove some deprecations (GH#8605) James Bourbeau
Documentation¶
Add Domino Data Lab to Hosted / managed Dask clusters (GH#8675) Ray Bell
Fix inter-linking and remove deprecated function (GH#8715) Julia Signell
Fix imbalanced backticks. (GH#8693) Matthias Bussonnier
Add documentation for high level graph visualization (GH#8483) Genevieve Buckley
Update documentation of
ProgressBar
out
parameter (GH#8604) Pedro SilvaImprove documentation of
dask.config.set
(GH#8705) crusaderkyRevert mention to
mypy
among type checkers (GH#8699) crusaderky
Maintenance¶
Update warning handling in
get_dummies
tests (GH#8651) James BourbeauAdd a github changelog template (GH#8714) Julia Signell
Update year in LICENSE.txt (GH#8665) David Hoese
Update
pre-commit
version (GH#8691) James BourbeauInclude
scipy
in upstream CI build (GH#8681) James BourbeauTemporarily pin
scipy < 1.8.0
in CI (GH#8683) James BourbeauPin
scipy
to less than 1.8.0 in GPU CI (GH#8698) Julia SignellAvoid
pytest.warns(None)
intest_multi.py
(GH#8678) James BourbeauUpdate GHA concurrent job cancellation (GH#8652) James Bourbeau
Make
test__get_paths
robust tosite.PREFIXES
being set (GH#8644) James BourbeauBump gpuCI PYTHON_VER to 3.9 (GH#8642) Charles Blackmon-Luca
2022.01.1¶
Released on January 28, 2022
New Features¶
Add
dask.dataframe.series.view()
(GH#8533) Pavithra Eswaramoorthy
Enhancements¶
Update
tz
forfastparquet
+pandas
1.4.0 (GH#8626) Martin DurantCleaning up misc tests for
pandas
compat (GH#8623) Julia SignellPandas compat: Filter sparse warnings (GH#8621) Julia Signell
Fail if
meta
is not apandas
object (GH#8563) Julia SignellUse
fsspec.parquet
module for better remote-storageread_parquet
performance (GH#8339) Richard (Rick) ZamoraMove DataFrame ACA aggregations to HLG (GH#8468) Richard (Rick) Zamora
Add optional information about originating function call in
DataFrameIOLayer
(GH#8453) Richard (Rick) ZamoraRefactor config default search path retrieval (GH#8573) James Bourbeau
Add
optimize_graph
flag toBag.to_dataframe
function (GH#8486) Maxim LippeveldMake sure that delayed output operations still return lists of paths (GH#8498) Julia Signell
Pandas compat: Fix
to_frame
name
to not passNone
(GH#8554) Julia SignellPandas compat: Fix
axis=None
warning (GH#8555) Julia SignellExpand Dask YAML config search directories (GH#8531) abergou
Bug Fixes¶
Fix
groupby.cumsum
with series grouped by index (GH#8588) Julia SignellFix
derived_from
forpandas
methods (GH#8612) Thomas J. FanEnforce boolean
ascending
forsort_values
(GH#8440) Charles Blackmon-LucaFix parsing of
__setitem__
indices (GH#8601) David HassellAvoid divide by zero in slicing (GH#8597) Doug Davis
Deprecations¶
Downgrade
meta
error in (GH#8563) to warning (GH#8628) Julia SignellPandas compat: Deprecate
append
whenpandas >= 1.4.0
(GH#8617) Julia Signell
Documentation¶
Replace outdated
columns
argument withmeta
in DataFrame constructor (GH#8614) kori73Refactor deploying docs (GH#8602) Jacob Tomlinson
Maintenance¶
Pin
coverage
in CI (GH#8631) James BourbeauMove
cached_cumsum
imports to be fromdask.utils
(GH#8606) James BourbeauUpdate gpuCI
RAPIDS_VER
to22.04
(GH#8600)Update cocstring for
from_delayed
function (GH#8576) Kirito1397Handle
plot_width
/plot_height
deprecations (GH#8544) Bryan Van de VenRemove unnecessary
pyyaml
importorskip
(GH#8562) James BourbeauSpecify scheduler in DataFrame
assert_eq
(GH#8559) Gabe Joseph
2022.01.0¶
Released on January 14, 2022
New Features¶
Add
DataFrame.nunique
(GH#8479) Sarah Charlotte JohnsonAdd
da.ndim
to matchnp.ndim
(GH#8502) Julia Signell
Enhancements¶
Only show
percentile
interpolation=
keyword warning if NumPy version >= 1.22 (GH#8564) Julia SignellRaise
PerformanceWarning
whenlimit
and"array.slicing.split-large-chunks"
areNone
(GH#8511) Julia SignellDefine
normalize_seq
function at import time (GH#8521) IllviljanEnsure that divisions are alway tuples (GH#8393) Charles Blackmon-Luca
Allow a callable scheduler for
bag.groupby
(GH#8492) Julia SignellMake byte blocks more even in
read_bytes
(GH#8459) Martin DurantImproved the efficiency of
matmul()
by completely removing concatenation (GH#8423) ParticularMinerLimit max chunk size when reshaping dask arrays (GH#8124) Genevieve Buckley
Changes for fastparquet superthrift (GH#8470) Martin Durant
Bug Fixes¶
Fix boolean indices in array assignment (GH#8538) David Hassell
Detect default
dtype
on array-likes (GH#8501) aeisenbarthFix
optimize_blockwise
bug for duplicate dependency names (GH#8542) Richard (Rick) ZamoraUpdate warnings for
DataFrame.GroupBy.apply
and transform (GH#8507) Sarah Charlotte JohnsonTrack HLG layer name in
Delayed
(GH#8452) Gabe JosephFix single item
nanmin
andnanmax
reductions (GH#8484) Julia SignellMake
read_csv
withcomment
kwarg
work even if there is a comment in the header (GH#8433) Julia Signell
Deprecations¶
Replace
interpolation
withmethod
andmethod
withinternal_method
(GH#8525) Julia SignellRemove daily stock demo utility (GH#8477) James Bourbeau
Documentation¶
Add a join example in docs that be run with copy/paste (GH#8520) kori73
Fix changelog section hyperlinks (GH#8534) Aneesh Nema
Hyphenate “single-machine scheduler” for consistency (GH#8519) Deepyaman Datta
Normalize whitespace in doctests in
slicing.py
(GH#8512) Maren WestermannBest practices storage line typo (GH#8529) Michael Delgado
Update figures (GH#8401) Sarah Charlotte Johnson
Remove
pyarrow
-only reference fromsplit_row_groups
inread_parquet
docstring (GH#8490) Naty Clementi
Maintenance¶
Remove obsolete
LocalFileSystem
tests that fail forfsspec>=2022.1.0
(GH#8565) Richard (Rick) ZamoraTweak: “RuntimeWarning: invalid value encountered in reciprocal” (GH#8561) crusaderky
Fix
skipna=None
forDataFrame.sem
(GH#8556) Julia SignellFix
PANDAS_GT_140
(GH#8552) Julia SignellCollections with HLG must always implement
__dask_layers__
(GH#8548) crusaderkyWork around race condition in
import llvmlite
(GH#8550) crusaderkySet a minimum version for
pyyaml
(GH#8545) Gaurav SheniAdding
nodefaults
to environments to fixtiledb
+ mac issue (GH#8505) Julia SignellSet ceiling for
setuptools
(GH#8509) Julia SignellAdd workflow / recipe to generate Dask nightlies (GH#8469) Charles Blackmon-Luca
Bump gpuCI
CUDA_VER
to 11.5 (GH#8489) Charles Blackmon-Luca
2021.12.0¶
Released on December 10, 2021
New Features¶
Add
Series
andIndex
is_monotonic*
methods (GH#8304) Daniel Mesejo-León
Enhancements¶
Blockwise
map_partitions
withpartition_info
(GH#8310) Gabe JosephBetter error message for length of array with unknown chunk sizes (GH#8436) Doug Davis
Use
by
instead ofindex
internally on the Groupby class (GH#8441) Julia SignellAllow custom sort functions for
sort_values
(GH#8345) Charles Blackmon-LucaAdd warning to
read_parquet
when statistics and partitions are misaligned (GH#8416) Richard (Rick) ZamoraMake visualize more consistent with compute (GH#8328) JSKenyon
Bug Fixes¶
Fix
map_blocks
not using own arguments inname
generation (GH#8462) David HoeseFix for index error with reading empty parquet file (GH#8410) Sarah Charlotte Johnson
Fix nullable-dtype error when writing partitioned parquet data (GH#8400) Richard (Rick) Zamora
Fix CSV header bug (GH#8413) Richard (Rick) Zamora
Fix empty chunk causes exception in
nanmin
/nanmax
(GH#8375) Boaz Mohar
Deprecations¶
Deprecate
token
keyword argument tomap_blocks
(GH#8464) James BourbeauDeprecation warning for default value of boundary kwarg in
map_overlap
(GH#8397) Genevieve Buckley
Documentation¶
Clarify
block_info
documentation (GH#8425) Genevieve BuckleyOutput from alt text sprint (GH#8456) Sarah Charlotte Johnson
Update talks and presentations (GH#8370) Naty Clementi
Update Anaconda link in “Paid support” section of docs (GH#8427) Martin Durant
Fixed broken
dask-gateway
link inecosystem.rst
(GH#8424) ofirrFix CuPy doctest error (GH#8412) Genevieve Buckley
Maintenance¶
Bump Bokeh min version to 2.1.1 (GH#8431) Bryan Van de Ven
Fix following
fsspec=2021.11.1
release (GH#8428) Martin DurantAdd
dask/ml.py
to pytest exclude list (GH#8414) Genevieve BuckleyUpdate gpuCI
RAPIDS_VER
to22.02
(GH#8394)Unpin
graphviz
and improve package management in environment-3.7 (GH#8411) Julia Signell
2021.11.2¶
Released on November 19, 2021
Only run gpuCI bump script daily (GH#8404) Charles Blackmon-Luca
Actually ignore index when asked in
assert_eq
(GH#8396) Gabe JosephEnsure single-partition join
divisions
istuple
(GH#8389) Charles Blackmon-LucaTry to make divisions behavior clearer (GH#8379) Julia Signell
Fix typo in
set_index
partition_size
parameter description (GH#8384) FredericOdermattUse
blockwise
insingle_partition_join
(GH#8341) Gabe JosephUse more explicit keyword arguments (GH#8354) Boaz Mohar
Fix
.loc
of DataFrame with nullable booleandtype
(GH#8368) Marco RossiParameterize shuffle implementation in tests (GH#8250) Ian Rose
Remove some doc build warnings (GH#8369) Boaz Mohar
Include properties in array API docs (GH#8356) Julia Signell
Fix Zarr for upstream (GH#8367) Julia Signell
Pin
graphviz
to avoid issue with windows and Python 3.7 (GH#8365) Julia SignellImport
graphviz.Diagraph
from top of module, not fromdot
(GH#8363) Julia Signell
2021.11.1¶
Released on November 8, 2021
Patch release to update distributed
dependency to version 2021.11.1
.
2021.11.0¶
Released on November 5, 2021
Fx
required_extension
behavior inread_parquet
(GH#8351) Richard (Rick) ZamoraAdd
align_dataframes
tomap_partitions
to broadcast a dataframe passed as an arg (GH#6628) Julia SignellBetter handling for arrays/series of keys in
dask.dataframe.loc
(GH#8254) Julia SignellAdd
name_function
option toto_parquet
(GH#7682) Matthew PowersGet rid of
environment-latest.yml
and update to Python 3.9 (GH#8275) Julia SignellRequire newer
s3fs
in CI (GH#8336) James BourbeauGroupby Rolling (GH#8176) Julia Signell
Add more ordering diagnostics to
dask.visualize
(GH#7992) Erik WelchUse
HighLevelGraph
optimizations fordelayed
(GH#8316) Ian Rosedemo_tuples
produces malformedHighLevelGraph
(GH#8325) crusaderkyDask calendar should show events in local time (GH#8312) Genevieve Buckley
Fix flaky
test_interrupt
(GH#8314) crusaderkyDeprecate
AxisError
(GH#8305) crusaderkyFix name of cuDF in extension documentation. (GH#8311) Vyas Ramasubramani
Add single eq operator (=) to parquet filters (GH#8300) Ayush Dattagupta
Improve support for Spark output in
read_parquet
(GH#8274) Richard (Rick) ZamoraAdd
dask.ml
module (GH#6384) Matthew RocklinCI fixups (GH#8298) James Bourbeau
Make slice errors match NumPy (GH#8248) Julia Signell
Fix API docs misrendering with new sphinx theme (GH#8296) Julia Signell
Replace
block
property withblockview
for array-like operations on blocks (GH#8242) Davis BennettDeprecate
file_path
and make it possible to save from within a notebook (GH#8283) Julia Signell
2021.10.0¶
Released on October 22, 2021
da.store
to create well-formedHighLevelGraph
(GH#8261) crusaderkyCI: force nightly
pyarrow
in the upstream build (GH#8281) Joris Van den BosscheRemove
chest
(GH#8279) James BourbeauSkip doctests if optional dependencies are not installed (GH#8258) Genevieve Buckley
Update
tmpdir
andtmpfile
context manager docstrings (GH#8270) Daniel Mesejo-LeónUnregister callbacks in doctests (GH#8276) James Bourbeau
Stale label GitHub action (GH#8244) Genevieve Buckley
Client-shutdown method appears twice (GH#8273) German Shiklov
Add pre-commit to test requirements (GH#8257) Genevieve Buckley
Refactor
read_metadata
infastparquet
engine (GH#8092) Richard (Rick) ZamoraSupport
Path
objects infrom_zarr
(GH#8266) Samuel GaistMake nested redirects work (GH#8272) Julia Signell
Set
memory_usage
toTrue
ifverbose
isTrue
in info (GH#8222) Kinshuk DuaRemove individual API doc pages from sphinx toctree (GH#8238) James Bourbeau
Ignore whitespace in gufunc
signature
(GH#8267) James BourbeauAdd workflow to update gpuCI (GH#8215) Charles Blackmon-Luca
DataFrame.head
shouldn’t warn when there’s one partition (GH#8091) Pankaj PatilIgnore arrow doctests if
pyarrow
not installed (GH#8256) Genevieve BuckleyFix
debugging.html
redirect (GH#8251) James BourbeauFix null sorting for single partition dataframes (GH#8225) Charles Blackmon-Luca
Fix
setup.html
redirect (GH#8249) Florian JetterRun
pyupgrade
in CI (GH#8246) crusaderkyFix label typo in upstream CI build (GH#8237) James Bourbeau
Add support for “dependent” columns in DataFrame.assign (GH#8086) Suriya Senthilkumar
add NumPy array of Dask keys to
Array
(GH#7922) Davis BennettRemove unnecessary
dask.multiprocessing
import in docs (GH#8240) Ray BellAdjust retrieving
_max_workers
fromExecutor
(GH#8228) John A KirkhamUpdate function signatures in
delayed
best practices docs (GH#8231) Vũ Trung ĐứcDocs reoganization (GH#7984) Julia Signell
Fix
df.quantile
on all missing data (GH#8129) Julia SignellAdd
tokenize.ensure-deterministic
config option (GH#7413) Hristo GeorgievUse
inclusive
rather thanclosed
withpandas>=1.4.0
andpd.date_range
(GH#8213) Julia SignellAdd
dask-gateway
, Coiled, and Saturn-Cloud to list of Dask setup tools (GH#7814) Kristopher OverholtEnsure existing futures get passed as deps when serializing
HighLevelGraph
layers (GH#8199) Jim Crist-HarifMake sure that the divisions of the single partition merge is left (GH#8162) Julia Signell
Refactor
read_metadata
inpyarrow
parquet engines (GH#8072) Richard (Rick) ZamoraSupport negative
drop_axis
inmap_blocks
andmap_overlap
(GH#8192) Gregory R. LeeFix upstream tests (GH#8205) Julia Signell
Add support for scalar item assignment by Series (GH#8195) Charles Blackmon-Luca
Add some basic examples to doc strings on
dask.bag
all
,any
,count
methods (GH#7630) Nathan DanielsenDon’t have upstream report depend on commit message (GH#8202) James Bourbeau
Ensure upstream CI cron job runs (GH#8200) James Bourbeau
Use
pytest.param
to properly label param-specific GPU tests (GH#8197) Charles Blackmon-LucaAdd
test_set_index
to tests ran on gpuCI (GH#8198) Charles Blackmon-LucaSuppress
tmpfile
OSError (GH#8191) James BourbeauUse
s.isna
instead ofpd.isna(s)
inset_partitions_pre
(fix cudf CI) (GH#8193) Charles Blackmon-LucaOpen an issue for
test-upstream
failures (GH#8067) Wallace ReisFix
to_parquet
bug in call topyarrow.parquet.read_metadata
(GH#8186) Richard (Rick) ZamoraAdd handling for null values in
sort_values
(GH#8167) Charles Blackmon-LucaBump
RAPIDS_VER
for gpuCI (GH#8184) Charles Blackmon-LucaDispatch walks MRO for lazily registered handlers (GH#8185) Jim Crist-Harif
Preserve
HighLevelGraphs
inDataFrame.from_delayed
(GH#8174) Gabe JosephDeprecate
inplace
argument for Dask series renaming (GH#8136) Marcel CoetzeeFix rolling for compatibility with
pandas > 1.3.0
(GH#8150) Julia SignellRaise error when
setitem
on unknown chunks (GH#8166) Julia SignellInclude divisions when doing
Index.to_series
(GH#8165) Julia Signell
2021.09.1¶
Released on September 21, 2021
Fix
groupby
for future pandas (GH#8151) Julia SignellRemove warning filters in tests that are no longer needed (GH#8155) Julia Signell
Add link to diagnostic visualize function in local diagnostic docs (GH#8157) David Hoese
Add
datetime_is_numeric
todataframe.describe
(GH#7719) Julia SignellRemove references to
pd.Int64Index
in anticipation of deprecation (GH#8144) Julia SignellUse
loc
if needed for series__get_item__
(GH#7953) Julia SignellSpecifically ignore warnings on mean for empty slices (GH#8125) Julia Signell
Skip
groupby
nunique
test for pandas >= 1.3.3 (GH#8142) Julia SignellImplement
ascending
arg forsort_values
(GH#8130) Charles Blackmon-LucaReplace
operator.getitem
(GH#8015) Naty ClementiDeprecate
zero_broadcast_dimensions
andhomogeneous_deepmap
(GH#8134) SnkSynthesisAllow
scheduler
to be anExecutor
(GH#8112) John A KirkhamHandle
asarray
/asanyarray
cases wherelike
is adask.Array
(GH#8128) Peter Andreas EntschevFix
index_col
duplication ifindex_col
is typestr
(GH#7661) McToelAdd
dtype
andorder
toasarray
andasanyarray
definitions (GH#8106) Julia SignellDeprecate
dask.dataframe.Series.__contains__
(GH#7914) Julia SignellFix edge case with
like
-arrays in_wrapped_qr
(GH#8122) Peter Andreas EntschevDeprecate
boundary_slice
kwarg:kind
for pandas compat (GH#8037) Julia Signell
2021.09.0¶
Released on September 3, 2021
Fewer open files (GH#7303) Julia Signell
Add
FileNotFound
to expected http errors (GH#8109) Martin DurantAdd
DataFrame.sort_values
to API docs (GH#8107) Benjamin ZaitlenChange to
dask.order
: be more eager at times (GH#7929) Erik WelchAdd pytest color to CI (GH#8090) James Bourbeau
FIX:
make_people
works withprocesses
scheduler (GH#8103) DahnAdds
deep
param to Dataframe copy method and restrict it toFalse
(GH#8068) João Paulo LacerdaFix typo in configuration docs (GH#8104) Robert Hales
Update formatting in
DataFrame.query
docstring (GH#8100) James BourbeauUn-xfail
sparse
tests for 0.13.0 release (GH#8102) James BourbeauAdd axes property to DataFrame and Series (GH#8069) Jordan Jensen
Add CuPy support in
da.unique
(values only) (GH#8021) Peter Andreas EntschevUnit tests for
sparse.zeros_like
(xfailed) (GH#8093) crusaderkyAdd explicit
like
kwarg support to array creation functions (GH#8054) Peter Andreas EntschevSeparate Array and DataFrame mindeps builds (GH#8079) James Bourbeau
Fork out
percentile_dispatch
todask.array
(GH#8083) GALI PREM SAGAREnsure
filepath
exists into_parquet
(GH#8057) James BourbeauUpdate scheduler plugin usage in
test_scheduler_highlevel_graph_unpack_import
(GH#8080) James BourbeauAdd
DataFrame.shuffle
to API docs (GH#8076) Martin FleischmannOrder requirements alphabetically (GH#8073) John A Kirkham
2021.08.1¶
Released on August 20, 2021
Add
ignore_metadata_file
option toread_parquet
(pyarrow-dataset
andfastparquet
support only) (GH#8034) Richard (Rick) ZamoraAdd reference to
pytest-xdist
in dev docs (GH#8066) Julia SignellInclude
tz
in meta fromto_datetime
(GH#8000) Julia SignellCI Infra Docs (GH#7985) Benjamin Zaitlen
Include invalid DataFrame key in
assert_eq
check (GH#8061) James BourbeauUse
__class__
when creating DataFrames (GH#8053) Mads R. B. KristensenUse development version of
distributed
in gpuCI build (GH#7976) James BourbeauIgnore whitespace when gufunc
signature
(GH#8049) James BourbeauMove pandas import and percentile dispatch refactor (GH#8055) GALI PREM SAGAR
Add colors to represent high level layer types (GH#7974) Freyam Mehta
Upstream instance fix (GH#8060) Jacob Tomlinson
Add
dask.widgets
and migrate HTML reprs tojinja2
(GH#8019) Jacob TomlinsonRemove
wrap_func_like_safe
, not required with NumPy >= 1.17 (GH#8052) Peter Andreas EntschevFix threaded scheduler memory backpressure regression (GH#8040) David Hoese
Add percentile dispatch (GH#8029) GALI PREM SAGAR
Use a publicly documented attribute
obj
ingroupby
rather than private_selected_obj
(GH#8038) GALI PREM SAGARUse
dict
to store data for {nan,}arg{min,max} in certain cases (GH#8014) Peter Andreas EntschevFix
blocksize
description formatting inread_pandas
(GH#8047) Louis MaddoxFix “point” -> “pointers” typo in docs (GH#8043) David Chudzicki
2021.08.0¶
Released on August 13, 2021
Fix
to_orc
delayed compute behavior (GH#8035) Richard (Rick) ZamoraDon’t convert to low-level task graph in
compute_as_if_collection
(GH#7969) James BourbeauFix multifile read for hdf (GH#8033) Julia Signell
Resolve warning in
distributed
tests (GH#8025) James BourbeauUpdate
to_orc
collection name (GH#8024) James BourbeauRaise
NotImplementedError
for non-indexable arg passed toto_datetime
(GH#7989) Doug DavisEnsure we error on warnings from
distributed
(GH#8002) James BourbeauAdded
dict
format into_bag
accessories of DataFrame (GH#7932) gurunathAdd tooltips to graphviz high-level graphs (GH#7973) Freyam Mehta
Close 2021 User Survey (GH#8007) Julia Signell
Reorganize CuPy tests into multiple files (GH#8013) Peter Andreas Entschev
Refactor and Expand Dask-Dataframe ORC API (GH#7756) Richard (Rick) Zamora
Don’t enforce columns if
enforce=False
(GH#7916) Julia SignellFix
map_overlap
trimming behavior whendrop_axis
is notNone
(GH#7894) Gregory R. LeeMark gpuCI CuPy test as flaky (GH#7994) Peter Andreas Entschev
Avoid using
Delayed
into_csv
andto_parquet
(GH#7968) Matthew RocklinUse
pytest.warns
instead of raises for checking parquet engine deprecation (GH#7993) Joris Van den BosscheBump
RAPIDS_VER
in gpuCI to 21.10 (GH#7991) Charles Blackmon-LucaAdd back
pyarrow-legacy
test coverage forpyarrow>=5
(GH#7988) Richard (Rick) ZamoraAllow
pyarrow>=5
into_parquet
andread_parquet
(GH#7967) Richard (Rick) ZamoraSkip CuPy tests requiring NEP-35 when NumPy < 1.20 is available (GH#7982) Peter Andreas Entschev
Add
tail
andhead
toSeriesGroupby
(GH#7935) Daniel Mesejo-LeónUpdate Zoom link for monthly meeting (GH#7979) James Bourbeau
Add gpuCI build script (GH#7966) Charles Blackmon-Luca
Deprecate
daily_stock
utility (GH#7949) James BourbeauAdd
distributed.nanny
to configuration reference docs (GH#7955) James BourbeauRequire NumPy 1.18+ & Pandas 1.0+ (GH#7939) John A Kirkham
2021.07.2¶
Released on July 30, 2021
Note
This is the last release with support for NumPy 1.17 and pandas 0.25. Beginning with the next release, NumPy 1.18 and pandas 1.0 will be the minimum supported versions.
Add
dask.array
SVG to the HTML Repr (GH#7886) Freyam MehtaAvoid use of
Delayed
into_parquet
(GH#7958) Matthew RocklinTemporarily pin
pyarrow<5
in CI (GH#7960) James BourbeauAdd deprecation warning for top-level
ucx
andrmm
config values (GH#7956) James BourbeauRemove skips from doctests (4 of 6) (GH#7865) Zhengnan Zhao
Remove skips from doctests (5 of 6) (GH#7864) Zhengnan Zhao
Adds missing prepend/append functionality to
da.diff
(GH#7946) Peter Andreas EntschevChange graphviz font family to sans (GH#7931) Freyam Mehta
Fix read-csv name - when path is different, use different name for task (GH#7942) Julia Signell
Update configuration reference for
ucx
andrmm
changes (GH#7943) James BourbeauAdd meta support to
__setitem__
(GH#7940) Peter Andreas EntschevNEP-35 support for
slice_with_int_dask_array
(GH#7927) Peter Andreas EntschevUnpin fastparquet in CI (GH#7928) James Bourbeau
Remove skips from doctests (3 of 6) (GH#7872) Zhengnan Zhao
2021.07.1¶
Released on July 23, 2021
Make array
assert_eq
check dtype (GH#7903) Julia SignellRemove skips from doctests (6 of 6) (GH#7863) Zhengnan Zhao
Remove experimental feature warning from actors docs (GH#7925) Matthew Rocklin
Remove skips from doctests (2 of 6) (GH#7873) Zhengnan Zhao
Separate out Array and Bag API (GH#7917) Julia Signell
Implement lazy
Array.__iter__
(GH#7905) Julia SignellClean up places where we inadvertently iterate over arrays (GH#7913) Julia Signell
Add
numeric_only
kwarg to DataFrame reductions (GH#7831) Julia SignellAdd pytest marker for GPU tests (GH#7876) Charles Blackmon-Luca
Add support for
histogram2d
indask.array
(GH#7827) Doug DavisRemove skips from doctests (1 of 6) (GH#7874) Zhengnan Zhao
Add node size scaling to the Graphviz output for the high level graphs (GH#7869) Freyam Mehta
Update old Bokeh links (GH#7915) Bryan Van de Ven
Temporarily pin
fastparquet
in CI (GH#7907) James BourbeauAdd
dask.array
import to progress bar docs (GH#7910) Fabian GebhartUse separate files for each DataFrame API function and method (GH#7890) Julia Signell
Fix
pyarrow-dataset
ordering bug (GH#7902) Richard (Rick) ZamoraGeneralize unique aggregate (GH#7892) GALI PREM SAGAR
Raise
NotImplementedError
when usingpd.Grouper
(GH#7857) Ruben van de GeerAdd
aggregate_files
argument to enable multi-file partitions inread_parquet
(GH#7557) Richard (Rick) ZamoraUn-
xfail
test_daily_stock
(GH#7895) James BourbeauUpdate access configuration docs (GH#7837) Naty Clementi
Use packaging for version comparisons (GH#7820) Elliott Sales de Andrade
Handle infinite loops in
merge_asof
(GH#7842) gerrymanoim
2021.07.0¶
Released on July 9, 2021
Include
fastparquet
in upstream CI build (GH#7884) James BourbeauBlockwise: handle non-string constant dependencies (GH#7849) Mads R. B. Kristensen
fastparquet
now supports new time types, including ns precision (GH#7880) Martin DurantAvoid
ParquetDataset
API when appending inArrowDatasetEngine
(GH#7544) Richard (Rick) ZamoraAdd retry logic to
test_shuffle_priority
(GH#7879) Richard (Rick) ZamoraUse strict channel priority in CI (GH#7878) James Bourbeau
Support nested
dask.distributed
imports (GH#7866) Matthew RocklinShould check module name only, not the entire directory filepath (GH#7856) Genevieve Buckley
Updates due to https://github.com/dask/fastparquet/pull/623 (GH#7875) Martin Durant
da.eye
fix forchunks=-1
(GH#7854) Naty ClementiTemporarily xfail
test_daily_stock
(GH#7858) James BourbeauSet priority annotations in
SimpleShuffleLayer
(GH#7846) Richard (Rick) ZamoraBlockwise: stringify constant key inputs (GH#7838) Mads R. B. Kristensen
Allow mixing dask and numpy arrays in
@guvectorize
(GH#6863) Julia SignellDon’t sample dict result of a shuffle group when calculating its size (GH#7834) Florian Jetter
Fix scipy tests (GH#7841) Julia Signell
Deterministically tokenize
datetime.date
(GH#7836) James BourbeauAdd
sample_rows
toread_csv
-like (GH#7825) Martin DurantFix typo in
config.deserialize
docstring (GH#7830) Geoffrey LentnerRemove warning filter in
test_dataframe_picklable
(GH#7822) James BourbeauImprovements to
histogramdd
(for handling inputs that are sequences-of-arrays). (GH#7634) Doug DavisMake
PY_VERSION
private (GH#7824) James Bourbeau
2021.06.2¶
Released on June 22, 2021
layers.py
compareparts_out
withset(self.parts_out)
(GH#7787) Genevieve BuckleyMake
check_meta
understand pandas dtypes better (GH#7813) Julia SignellRemove “Educational Resources” doc page (GH#7818) James Bourbeau
2021.06.1¶
Released on June 18, 2021
Replace funding page with ‘Supported By’ section on dask.org (GH#7817) James Bourbeau
Add initial deprecation utilities (GH#7810) James Bourbeau
Enforce dtype conservation in ufuncs that explicitly use
dtype=
(GH#7808) Doug DavisAdd Coiled to list of paid support organizations (GH#7811) Kristopher Overholt
Small tweaks to the HTML repr for
Layer
&HighLevelGraph
(GH#7812) Genevieve BuckleyAdd dark mode support to HLG HTML repr (GH#7809) Jacob Tomlinson
Remove compatibility entries for old distributed (GH#7801) Elliott Sales de Andrade
Implementation of HTML repr for
HighLevelGraph
layers (GH#7763) Genevieve BuckleyUpdate default
blockwise
token to avoid DataFrame column name clash (GH#6546) James BourbeauUse dispatch
concat
formerge_asof
(GH#7806) Julia SignellFix upstream freq tests (GH#7795) Julia Signell
Use more context managers from the standard library (GH#7796) James Bourbeau
Simplify skips in parquet tests (GH#7802) Elliott Sales de Andrade
Remove check for outdated bokeh (GH#7804) Elliott Sales de Andrade
More test coverage uploads (GH#7799) James Bourbeau
Remove
ImportError
catching fromdask/__init__.py
(GH#7797) James BourbeauAllow
DataFrame.join()
to take a list of DataFrames to merge with (GH#7578) Krishan BhasinFix maximum recursion depth exception in
dask.array.linspace
(GH#7667) Daniel Mesejo-LeónFix docs links (GH#7794) Julia Signell
Initial
da.select()
implementation and test (GH#7760) Gabriel MirettiLayers must implement
get_output_keys
method (GH#7790) Genevieve BuckleyDon’t include or expect
freq
in divisions (GH#7785) Julia SignellA
HighLevelGraph
abstract layer formap_overlap
(GH#7595) Genevieve BuckleyAlways include kwarg name in
drop
(GH#7784) Julia SignellOnly rechunk for median if needed (GH#7782) Julia Signell
Add
add_(prefix|suffix)
to DataFrame and Series (GH#7745) tsugaMove
read_hdf
toBlockwise
(GH#7625) Richard (Rick) ZamoraMake
Layer.get_output_keys
officially an abstract method (GH#7775) Genevieve BuckleyNon-dask-arrays and broadcasting in
ravel_multi_index
(GH#7594) Gabe JosephFix for paths ending with “/” in parquet overwrite (GH#7773) Martin Durant
Fixing calling
.visualize()
withfilename=None
(GH#7740) Freyam MehtaGenerate unique names for
SubgraphCallable
(GH#7637) Bruce MerryPin
fsspec
to2021.5.0
in CI (GH#7771) James BourbeauEvaluate graph lazily if meta is provided in
from_delayed
(GH#7769) Florian JetterAdd
meta
support forDatetimeTZDtype
(GH#7627) gerrymanoimAdd dispatch label to automatic PR labeler (GH#7701) James Bourbeau
Fix HDFS tests (GH#7752) Julia Signell
2021.06.0¶
Released on June 4, 2021
Remove abstract tokens from graph keys in
rewrite_blockwise
(GH#7721) Richard (Rick) ZamoraEnsure correct column order in csv
project_columns
(GH#7761) Richard (Rick) ZamoraRenamed inner loop variables to avoid duplication (GH#7741) Boaz Mohar
Do not return delayed object from
to_zarr
(GH#7738) Chris RoatArray: correct number of outputs in
apply_gufunc
(GH#7669) Gabe JosephRewrite
da.fromfunction
withda.blockwise
(GH#7704) John A KirkhamRename
make_meta_util
tomake_meta
(GH#7743) GALI PREM SAGARRepartition before shuffle if the requested partitions are less than input partitions (GH#7715) Vibhu Jawa
Blockwise: handle constant key inputs (GH#7734) Mads R. B. Kristensen
Added raise to
apply_gufunc
(GH#7744) Boaz MoharShow failing tests summary in CI (GH#7735) Genevieve Buckley
sizeof
sets in Python 3.9 (GH#7739) Mads R. B. KristensenWarn if using pandas datetimelike string in
dataframe.__getitem__
(GH#7749) Julia SignellHighlight the
client.dashboard_link
(GH#7747) Genevieve BuckleyEasier link for subscribing to the Google calendar (GH#7733) Genevieve Buckley
Automatically show graph visualization in Jupyter notebooks (GH#7716) Genevieve Buckley
Add
autofunction
forunify_chunks
in API docs (GH#7730) James Bourbeau
2021.05.1¶
Released on May 28, 2021
Pandas compatibility (GH#7712) Julia Signell
Fix
optimize_dataframe_getitem
bug (GH#7698) Richard (Rick) ZamoraUpdate
make_meta
import in docs (GH#7713) Benjamin ZaitlenFix format string in error message (GH#7706) Jiaming Yuan
Fix
read_sql_table
returning wrong result for single column loads (GH#7572) c-thielAdd slack join link in
support.rst
(GH#7679) Naty ClementiRemove unused alphabet variable (GH#7700) James Bourbeau
Fix meta creation incase of
object
(GH#7586) GALI PREM SAGARAdd dispatch for
union_categoricals
(GH#7699) GALI PREM SAGARConsolidate array
Dispatch
objects (GH#7505) James BourbeauMove DataFrame
dispatch.registers
to their own file (GH#7503) Julia SignellFix delayed with
dataclasses
whereinit=False
(GH#7656) Julia SignellAllow a column to be named
divisions
(GH#7605) Julia SignellStack nd array with unknown chunks (GH#7562) Chris Roat
Promote the 2021 Dask User Survey (GH#7694) Genevieve Buckley
Fix typo in
DataFrame.set_index()
(GH#7691) James LambCleanup array API reference links (GH#7684) David Hoese
Accept
axis
tuple forflip
to be consistent with NumPy (GH#7675) Andrew ChampionBump
pre-commit
hook versions (GH#7676) James BourbeauCleanup
to_zarr
docstring (GH#7683) David HoeseFix the docstring of
read_orc
(GH#7678) Justus MaginDoc
ipyparallel
&mpi4py
concurrent.futures
(GH#7665) John A KirkhamUpdate tests to support CuPy 9 (GH#7671) Peter Andreas Entschev
Fix some
HighLevelGraph
documentation inaccuracies (GH#7662) Mads R. B. KristensenFix spelling in Series
getitem
error message (GH#7659) Maisie Marshall
2021.05.0¶
Released on May 14, 2021
Remove deprecated
kind
kwarg to comply with pandas 1.3.0 (GH#7653) Julia SignellFix bug in DataFrame column projection (GH#7645) Richard (Rick) Zamora
Merge global annotations when packing (GH#7565) Mads R. B. Kristensen
Avoid
inplace=
in pandasset_categories
(GH#7633) James BourbeauChange the active-fusion default to
False
for Dask-Dataframe (GH#7620) Richard (Rick) ZamoraArray: remove extraneous code from
RandomState
(GH#7487) Gabe JosephImplement
str.concat
whenothers=None
(GH#7623) Daniel Mesejo-LeónFix
dask.dataframe
in sandboxed environments (GH#7601) Noah D. BrenowitzSupport for
cupyx.scipy.linalg
(GH#7563) Benjamin ZaitlenMove
timeseries
and daily-stock toBlockwise
(GH#7615) Richard (Rick) ZamoraFix bugs in broadcast join (GH#7617) Richard (Rick) Zamora
Use
Blockwise
for DataFrame IO (parquet, csv, and orc) (GH#7415) Richard (Rick) ZamoraAdding chunk & type information to Dask
HighLevelGraph
s (GH#7309) Genevieve BuckleyRemove skip on test freq (GH#7608) Julia Signell
Remove
ignore_abc_warning
(GH#7606) Julia SignellHarden DataFrame merge between column-selection and index (GH#7575) Richard (Rick) Zamora
Get rid of
ignore_abc
decorator (GH#7604) Julia SignellRemove kwarg validation for bokeh (GH#7597) Julia Signell
Add
loky
example (GH#7590) Naty ClementiDelayed:
nout
when arguments become tasks (GH#7593) Gabe JosephUpdate distributed version in mindep CI build (GH#7602) James Bourbeau
Support all or no overlap between partition columns and real columns (GH#7541) Richard (Rick) Zamora
2021.04.1¶
Released on April 23, 2021
Handle
Blockwise
HLG pack/unpack forconcatenate=True
(GH#7455) Richard (Rick) Zamoramap_partitions
: use tokenized info as name of theSubgraphCallable
(GH#7524) Mads R. B. KristensenUsing
tmp_path
andtmpdir
to avoid temporary files and directories hanging in the repo (GH#7592) Naty ClementiContributing to docs (development guide) (GH#7591) Naty Clementi
Add more packages to Python 3.9 CI build (GH#7588) James Bourbeau
Array: Fix NEP-18 dispatching in finalize (GH#7508) Gabe Joseph
Misc fixes for
numpydoc
(GH#7569) Matthias BussonnierAvoid pandas
level=
keyword deprecation (GH#7577) James BourbeauMap e.g.
.repartition(freq="M")
to.repartition(freq="MS")
(GH#7504) Ruben van de GeerRemove hash seeding in parallel CI runs (GH#7128) Elliott Sales de Andrade
Simplify transpose axes cleanup (GH#7561) Julia Signell
Make
ValueError in len(index_names) > 1
explicit it’s usingfastparquet
(GH#7556) Ray BellFix
dict
-column appending forpyarrow
parquet engines (GH#7527) Richard (Rick) ZamoraAdd a documentation auto label (GH#7560) Doug Davis
Add
dask.delayed.Delayed
to docs so it can be referenced by other sphinx docs (GH#7559) Doug DavisFix upstream
idxmaxmin
for unevensplit_every
(GH#7538) Julia SignellMake
normalize_token
for pandasSeries
/DataFrame
future proof (no direct block access) (GH#7318) Joris Van den BosscheRedesigned
__setitem__
implementation (GH#7393) David Hassellhistogram
,histogramdd
improvements (docs; return consistencies) (GH#7520) Doug DavisForce nightly
pyarrow
in the upstream build (GH#7530) Joris Van den BosscheFix Configuration Reference (GH#7533) Benjamin Zaitlen
Use
.to_parquet
ondask.dataframe
in doc string (GH#7528) Ray BellAvoid double
msgpack
serialization of HLGs (GH#7525) Mads R. B. KristensenEncourage usage of
yaml.safe_load()
in configuration doc (GH#7529) Hristo GeorgievFix
reshape
bug. Add relevant test. Fixes #7171. (GH#7523) JSKenyonSupport
custom_metadata=
argument into_parquet
(GH#7359) Richard (Rick) ZamoraClean some documentation warnings (GH#7518) Daniel Mesejo-León
Getting rid of more docs warnings (GH#7426) Julia Signell
Added
product
(alias ofprod
) (GH#7517) Freyam MehtaFix upstream
__array_ufunc__
tests (GH#7494) Julia SignellEscape from
map_overlap
tomap_blocks
if depth is zero (GH#7481) Genevieve BuckleyAdd
check_type
to arrayassert_eq
(GH#7491) Julia Signell
2021.04.0¶
Released on April 2, 2021
Adding support for multidimensional histograms with
dask.array.histogramdd
(GH#7387) Doug DavisUpdate docs on number of threads and workers in default
LocalCluster
(GH#7497) cameron16Add labels automatically when certain files are touched in a PR (GH#7506) Julia Signell
Extract
ignore_order
fromkwargs
(GH#7500) GALI PREM SAGAROnly provide installation instructions when distributed is missing (GH#7498) Matthew Rocklin
Start adding
isort
(GH#7370) Julia SignellAdd
ignore_order
parameter indd.concat
(GH#7473) Daniel Mesejo-LeónUse powers-of-two when displaying RAM (GH#7484) crusaderky
Added License Classifier (GH#7485) Tom Augspurger
Replace conda with mamba (GH#7227) crusaderky
Fix typo in array docs (GH#7478) James Lamb
Use
concurrent.futures
in local scheduler (GH#6322) John A Kirkham
2021.03.1¶
Released on March 26, 2021
Add a dispatch for
is_categorical_dtype
to handle non-pandas objects (GH#7469) brandon-b-millerUse
multiprocessing.Pool
intest_read_text
(GH#7472) John A KirkhamAdd missing
meta
kwarg to gufunc class (GH#7423) Peter Andreas EntschevExample for memory-mapped Dask array (GH#7380) Dieter Weber
Fix NumPy upstream failures
xfail
pandas and fastparquet failures (GH#7441) Julia SignellFix bug in repartition with freq (GH#7357) Ruben van de Geer
Fix
__array_function__
dispatching fortril
/triu
(GH#7457) Peter Andreas EntschevUse
concurrent.futures.Executors
in a few tests (GH#7429) John A KirkhamRequire NumPy >=1.16 (GH#7383) crusaderky
Minor
sort_values
housekeeping (GH#7462) Ryan WilliamsEnsure natural sort order in parquet part paths (GH#7249) Ryan Williams
Remove global env mutation upon running
test_config.py
(GH#7464) Hristo GeorgievUpdate NumPy intersphinx URL (GH#7460) Gabe Joseph
Add
rot90
(GH#7440) Trevor ManzUpdate docs for required package for endpoint (GH#7454) Nick Vazquez
Master -> main in
slice_array
docstring (GH#7453) Gabe JosephExpand
dask.utils.is_arraylike
docstring (GH#7445) Doug DavisSimplify
BlockwiseIODeps
importing (GH#7420) Richard (Rick) ZamoraUpdate layer annotation packing method (GH#7430) James Bourbeau
Drop duplicate test in
test_describe_empty
(GH#7431) John A KirkhamAdded df
kurtosis
-method and testing (GH#7273) Jan BorchmannAvoid quadratic-time performance for HLG culling (GH#7403) Bruce Merry
Temporarily skip problematic
sparse
test (GH#7421) James BourbeauUpdate some CI workflow names (GH#7422) James Bourbeau
Fix HDFS test (GH#7418) Julia Signell
Make changelog subtitles match the hierarchy (GH#7419) Julia Signell
Add support for normalize in
value_counts
(GH#7342) Julia SignellAvoid unnecessary imports for HLG Layer unpacking and materialization (GH#7381) Richard (Rick) Zamora
Bincount fix slicing (GH#7391) Genevieve Buckley
Add
sliding_window_view
(GH#7234) Deepak CherianFix typo in
docs/source/develop.rst
(GH#7414) Hristo GeorgievSwitch documentation builds for PRs to readthedocs (GH#7397) James Bourbeau
Adds
sort_values
to dask.DataFrame (GH#7286) gerrymanoimPin
sqlalchemy<1.4.0
in CI (GH#7405) James BourbeauComment fixes (GH#7215) Ryan Williams
Dead code removal / fixes (GH#7388) Ryan Williams
Use single thread for
pa.Table.from_pandas
calls (GH#7347) Richard (Rick) ZamoraReplace
'container'
with'image'
(GH#7389) James LambPass delimiter to
fsspec
inbag.read_text
(GH#7349) Martin DurantEmbed literals in
SubgraphCallable
when packingBlockwise
(GH#7353) Mads R. B. KristensenUpdate
test_hdf.py
to not reuse file handlers (GH#7044) rs9w33Require additional dependencies: cloudpickle, partd, fsspec, toolz (GH#7345) Julia Signell
Prepare
Blockwise
+ IO infrastructure (GH#7281) Richard (Rick) ZamoraRemove duplicated imports from
test_slicing.py
(GH#7365) Hristo GeorgievAdd test deps for pip development (GH#7360) Julia Signell
Support int slicing for non-NumPy arrays (GH#7364) Peter Andreas Entschev
Automatically cancel previous CI builds (GH#7348) James Bourbeau
dask.array.asarray
should handle case wherexarray
class is in top-level namespace (GH#7335) Tom WhiteHighLevelGraph
length without materializing layers (GH#7274) Gabe JosephDrop support for Python 3.6 (GH#7006) James Bourbeau
Fix fsspec usage in
create_metadata_file
(GH#7295) Richard (Rick) ZamoraChange default branch from master to main (GH#7198) Julia Signell
Add Xarray to CI software environment (GH#7338) James Bourbeau
Update repartition argument name in error text (GH#7336) Eoin Shanaghy
Run upstream tests based on commit message (GH#7329) James Bourbeau
Use
pytest.register_assert_rewrite
on util modules (GH#7278) Bruce MerryAdd example on using specific chunk sizes in
from_array()
(GH#7330) James LambMove NumPy skip into test (GH#7247) Julia Signell
2021.03.0¶
Released on March 5, 2021
Note
This is the first release with support for Python 3.9 and the last release with support for Python 3.6
Bump minimum version of
distributed
(GH#7328) James BourbeauFix
percentiles_summary
withdask_cudf
(GH#7325) Peter Andreas EntschevTemporarily revert recent
Array.__setitem__
updates (GH#7326) James BourbeauBlockwise.clone
(GH#7312) crusaderkyNEP-35 duck array update (GH#7321) James Bourbeau
Don’t allow setting
.name
for array (GH#7222) Julia SignellUse nearest interpolation for creating percentiles of integer input (GH#7305) Kyle Barron
Test
exp
with CuPy arrays (GH#7322) John A KirkhamCheck that computed chunks have right size and dtype (GH#7277) Bruce Merry
pytest.mark.flaky
(GH#7319) crusaderkyContributing docs: add note to pull the latest git tags before pip installing Dask (GH#7308) Genevieve Buckley
Support for Python 3.9 (GH#7289) crusaderky
Add broadcast-based merge implementation (GH#7143) Richard (Rick) Zamora
Add
split_every
tograph_manipulation
(GH#7282) crusaderkyTypo in optimize docs (GH#7306) Julius Busecke
dask.graph_manipulation
support forxarray.Dataset
(GH#7276) crusaderkyAdd plot width and height support for Bokeh 2.3.0 (GH#7297) James Bourbeau
Add NumPy functions
tri
,triu_indices
,triu_indices_from
,tril_indices
,tril_indices_from
(GH#6997) IllviljanRemove “cleanup” task in DataFrame on-disk shuffle (GH#7260) Sinclair Target
Use development version of
distributed
in CI (GH#7279) James BourbeauMoving high level graph pack/unpack Dask (GH#7179) Mads R. B. Kristensen
Improve performance of
merge_percentiles
(GH#7172) Ashwin SrinathExample for working with categoricals and parquet (GH#7085) McToel
Adds tree reduction to
bincount
(GH#7183) Thomas J. FanImprove documentation of
name
infrom_array
(GH#7264) Bruce MerryFix
cumsum
for empty partitions (GH#7230) Julia SignellAdd
map_blocks
example to dask array creation docs (GH#7221) Julia SignellFix performance issue in
dask.graph_manipulation.wait_on()
(GH#7258) crusaderkyReplace coveralls with codecov.io (GH#7246) crusaderky
Pin to a particular
black
rev in pre-commit (GH#7256) Julia SignellMinor typo in documentation:
array-chunks.rst
(GH#7254) Magnus NordFix bugs in
Blockwise
andShuffleLayer
(GH#7213) Richard (Rick) ZamoraFix parquet filtering bug for
"pyarrow-dataset"
with pyarrow-3.0.0 (GH#7200) Richard (Rick) Zamoragraph_manipulation
without NumPy (GH#7243) crusaderkySupport for NEP-35 (GH#6738) Peter Andreas Entschev
Avoid running unit tests during doctest CI build (GH#7240) James Bourbeau
Run doctests on CI (GH#7238) Julia Signell
Cleanup code quality on set arithmetics (GH#7196) crusaderky
Add
dask.array.delete
(GH#7125) Julia SignellUnpin graphviz now that new conda-forge recipe is built (GH#7235) Julia Signell
Don’t use NumPy 1.20 from conda-forge on Mac (GH#7211) crusaderky
map_overlap
: Don’t rechunk axes without overlap (GH#7233) Deepak CherianPin graphviz to avoid issue with latest conda-forge build (GH#7232) Julia Signell
Use
html_css_files
in docs for custom CSS (GH#7220) James BourbeauGraph manipulation:
clone
,bind
,checkpoint
,wait_on
(GH#7109) crusaderkyFix handling of filter expressions in parquet
pyarrow-dataset
engine (GH#7186) Joris Van den BosscheExtend
__setitem__
to more closely match numpy (GH#7033) David HassellClean up Python 2 syntax (GH#7195) crusaderky
Fix regression in
Delayed._length
(GH#7194) crusaderky__dask_layers__()
tests and tweaks (GH#7177) crusaderkyProperly convert
HighLevelGraph
in multiprocessing scheduler (GH#7191) Jim Crist-HarifDon’t fail fast in CI (GH#7188) James Bourbeau
2021.02.0¶
Released on February 5, 2021
Add
percentile
support for NEP-35 (GH#7162) Peter Andreas EntschevAdded support for
Float64
in column assignment (GH#7173) Nils BraunCoarsen rechunking error (GH#7127) Davis Bennett
Fix upstream CI tests (GH#6896) Julia Signell
Revise
HighLevelGraph
Mapping API (GH#7160) crusaderkyUpdate low-level graph spec to use any hashable for keys (GH#7163) James Bourbeau
Generically rebuild a collection with different keys (GH#7142) crusaderky
Allow
dask.array.ravel
to acceptarray_like
argument (GH#7138) D-StacksFixes link in array design doc (GH#7152) Thomas J. Fan
Fix example of using
blockwise
for an outer product (GH#7119) Bruce MerryDeprecate
HighlevelGraph.dicts
in favor of.layers
(GH#7145) Amit KumarAlign
FastParquetEngine
with pyarrow engines (GH#7091) Richard (Rick) ZamoraSimplify contents of parts list in
read_parquet
(GH#7066) Richard (Rick) Zamoracheck_meta(
): use__class__
when checking DataFrame types (GH#7099) Mads R. B. KristensenFix parquet
getitem
optimization (GH#7106) Richard (Rick) ZamoraAdd cytoolz back to CI environment (GH#7103) James Bourbeau
2021.01.1¶
Released on January 22, 2021
Partially fix
cumprod
(GH#7089) Julia SignellTest pandas 1.1.x / 1.2.0 releases and pandas nightly (GH#6996) Joris Van den Bossche
Use assign to avoid
SettingWithCopyWarning
(GH#7092) Julia Signell'mode'
argument passed tobokeh.output_file()
(GH#7034) (GH#7075) patquemSkip empty partitions when doing
groupby.value_counts
(GH#7073) Julia SignellAdd error messages to
assert_eq()
(GH#7083) James Lamb
2021.01.0¶
Released on January 15, 2021
map_partitions
with review comments (GH#6776) Kumar Bharath PrabhuMake sure that
population
is a real list (GH#7027) Julia SignellPropagate
storage_options
inread_csv
(GH#7074) Richard (Rick) ZamoraRemove all
BlockwiseIO
code (GH#7067) Richard (Rick) ZamoraFix CI (GH#7069) James Bourbeau
Add option to control rechunking in
reshape
(GH#6753) Tom AugspurgerFix
linalg.lstsq
for complex inputs (GH#7056) Johnnie GrayAdd
compression='infer'
default toread_csv
(GH#6960) Richard (Rick) ZamoraRevert parameter changes in
svd_compressed
#7003 (GH#7004) Eric CzechSkip failing s3 test (GH#7064) Martin Durant
Revert
BlockwiseIO
(GH#7048) Richard (Rick) ZamoraAdd some cross-references to
DataFrame.to_bag()
andSeries.to_bag()
(GH#7049) Rob MaloufRewrite
matmul
asblockwise
without contraction/concatenate (GH#7000) Rafal WojdylaUse
functools.cached_property
inda.shape
(GH#7023) IllviljanUse meta value in series
non_empty
(GH#6976) Julia SignellRevert “Temporarly pin sphinx version to 3.3.1 (GH#7002)” (GH#7014) Rafal Wojdyla
Revert
python-graphviz
pinning (GH#7037) Julia SignellAccidentally committed print statement (GH#7038) Julia Signell
Pass
dropna
andobserved
inagg
(GH#6992) Julia SignellAdd index to
meta
after.str.split
with expand (GH#7026) Ruben van de GeerCI: test pyarrow 2.0 and nightly (GH#7030) Joris Van den Bossche
Temporarily pin
python-graphviz
in CI (GH#7031) James BourbeauUnderline section in
numpydoc
(GH#7013) Matthias BussonnierKeep normal optimizations when adding custom optimizations (GH#7016) Matthew Rocklin
Temporarily pin sphinx version to 3.3.1 (GH#7002) Rafal Wojdyla
DOC: Misc formatting (GH#6998) Matthias Bussonnier
Add
inline_array
option tofrom_array
(GH#6773) Tom AugspurgerRevert “Initial pass at blockwise array creation routines (GH#6931)” (:pr:`6995) James Bourbeau
Set
npartitions
inset_index
(GH#6978) Julia SignellUpstream
config
serialization and inheritance (GH#6987) Jacob TomlinsonBump the minimum time in
test_minimum_time
(GH#6988) Martin DurantFix pandas
dtype
inference forread_parquet
(GH#6985) Richard (Rick) ZamoraAvoid data loss in
set_index
withsorted=True
(GH#6980) Richard (Rick) ZamoraBugfix in
read_parquet
for handling un-named indices withindex=False
(GH#6969) Richard (Rick) ZamoraUse
__class__
when comparing meta data (GH#6981) Mads R. B. KristensenComparing string versions won’t always work (GH#6979) Rafal Wojdyla
Initial pass at blockwise array creation routines (GH#6931) Ian Rose
Simplify
has_parallel_type()
(GH#6927) Mads R. B. KristensenHandle annotation unpacking in
BlockwiseIO
(GH#6934) Simon PerkinsAvoid deprecated
yield_fixture
intest_sql.py
(GH#6968) Richard (Rick) ZamoraRemove bad graph logic in
BlockwiseIO
(GH#6933) Richard (Rick) ZamoraGet config item if variable is
None
(GH#6862) Jacob TomlinsonUpdate
from_pandas
docstring (GH#6957) Richard (Rick) ZamoraPrevent
fuse_roots
from clobbering annotations (GH#6955) Simon Perkins
2020.12.0¶
Released on December 10, 2020
Highlights¶
Switched to CalVer for versioning scheme.
Introduced new APIs for
HighLevelGraph
to enable sending high-level representations of task graphs to the distributed scheduler.Introduced new
HighLevelGraph
layer objects includingBasicLayer
,Blockwise
,BlockwiseIO
,ShuffleLayer
, and more.Added support for applying custom
Layer
-level annotations likepriority
,retries
, etc. with thedask.annotations
context manager.Updated minimum supported version of pandas to 0.25.0 and NumPy to 1.15.1.
Support for the
pyarrow.dataset
API toread_parquet
.Several fixes to Dask Array’s SVD.
All changes¶
Make
observed
kwarg optional (GH#6952) Julia SignellMin supported pandas 0.25.0 numpy 1.15.1 (GH#6895) Julia Signell
Make order of categoricals unambiguous (GH#6949) Julia Signell
Improve “pyarrow-dataset” statistics performance for
read_parquet
(GH#6918) Richard (Rick) ZamoraAdd
observed
keyword togroupby
(GH#6854) Julia SignellMake sure
include_path_column
works when there are multiple partitions per file (GH#6911) Julia SignellFix:
array.overlap
andarray.map_overlap
block sizes are incorrect when depth is an unsigned bit type (GH#6909) GFleishmanReturn a
Bag
fromsample
(GH#6941) Shang WangEnable parquet metadata collection in parallel (GH#6921) Richard (Rick) Zamora
Avoid using
_file
inprogressbar
if it isNone
(GH#6938) Mark HarfoucheAdd Zarr to upstream CI build (GH#6932) James Bourbeau
Introduce
BlockwiseIO
layer (GH#6878) Richard (Rick) ZamoraTransmit
Layer
Annotations to Scheduler (GH#6889) Simon PerkinsUpdate opportunistic caching page to remove experimental warning (GH#6926) Timost
Allow
pyarrow >2.0.0
(GH#6772) Richard (Rick) ZamoraSupport
pyarrow.dataset
API forread_parquet
(GH#6534) Richard (Rick) ZamoraAdd more informative error message to
da.coarsen
when coarsening factors do not divide shape (GH#6908) Davis BennettOnly run the cron CI on
dask/dask
not forks (GH#6905) Jacob TomlinsonAdd
annotations
toShuffleLayers
(GH#6913) Matthew RocklinTemporarily xfail
test_from_s3
(GH#6915) James BourbeauAdded dataframe
skew
method (GH#6881) Jan BorchmannFix
dtype
in arraymeta
(GH#6893) Julia SignellMissing
name
arg inhelm install ...
(GH#6903) Ruben van de GeerFix: exception when reading an item with filters (GH#6901) Martin Durant
Add support for
cupyx
sparse todask.array.dot
(GH#6846) Akira NarusePin array mindeps up a bit to get the tests to pass [test-mindeps] (GH#6894) Julia Signell
Update/remove pandas and numpy in mindeps (GH#6888) Julia Signell
Fix
ArrowEngine
bug in use ofclear_known_categories
(GH#6887) Richard (Rick) ZamoraFix documentation about task scheduler (GH#6879) Zhengnan Zhao
Add human relative time formatting utility (GH#6883) Jacob Tomlinson
Possible fix for 6864
set_index
issue (GH#6866) Richard (Rick) ZamoraBasicLayer
: remove dependency arguments (GH#6859) Mads R. B. KristensenSerialization of
Blockwise
(GH#6848) Mads R. B. KristensenAddress
columns=[]
bug (GH#6871) Richard (Rick) ZamoraAvoid duplicate parquet schema communication (GH#6841) Richard (Rick) Zamora
Add
create_metadata_file
utility for existing parquet datasets (GH#6851) Richard (Rick) ZamoraImprove ordering for workloads with a common terminus (GH#6779) Tom Augspurger
Stringify utilities (GH#6852) Mads R. B. Kristensen
Add keyword
overwrite=True
toto_parquet
to remove dangling files when overwriting a pyarrowDataset
. (GH#6825) Greg HayesRemoved
map_tasks()
andmap_basic_layers()
(GH#6853) Mads R. B. KristensenIntroduce QR iteration to
svd_compressed
(GH#6813) RogerMoens__dask_distributed_pack__()
now takes aclient
argument (GH#6850) Mads R. B. KristensenUse
map_partitions
instead ofdelayed
inset_index
(GH#6837) Mads R. B. KristensenAdd doc hit for
as_completed().update(futures)
(GH#6817) manuelsBump GHA
setup-miniconda
version (GH#6847) Jacob TomlinsonRemove nans when setting sorted index (GH#6829) Rockwell Weiner
Fix transpose of u in SVD (GH#6799) RogerMoens
Migrate to GitHub Actions (GH#6794) Jacob Tomlinson
Fix sphinx
currentmodule
usage (GH#6839) James BourbeauFix minimum dependencies CI builds (GH#6838) James Bourbeau
Avoid graph materialization during
Blockwise
culling (GH#6815) Richard (Rick) ZamoraFixed typo (GH#6834) Devanshu Desai
Use
HighLevelGraph.merge
incollections_to_dsk
(GH#6836) Mads R. B. KristensenRespect
dtype
in svdcompression_matrix
#2849 (GH#6802) RogerMoensAdd blocksize to task name (GH#6818) Julia Signell
Check for all-NaN partitions (GH#6821) Rockwell Weiner
Change “institutional” SQL doc section to point to main SQL doc (GH#6823) Martin Durant
Fix:
DataFrame.join
doesn’t accept Series as other (GH#6809) David KatzRemove
to_delayed
operations fromto_parquet
(GH#6801) Richard (Rick) ZamoraLayer annotation docstrings improvements (GH#6806) Simon Perkins
Avro reader (GH#6780) Martin Durant
Rechunk array if smallest chunk size is smaller than depth (GH#6708) Julia Signell
Add Layer Annotations (GH#6767) Simon Perkins
Add optional IO-subgraph to
Blockwise
Layers (GH#6715) Richard (Rick) ZamoraAdd high level graph pack/unpack for distributed (GH#6786) Mads R. B. Kristensen
Add missing methods of the Dataframe API (GH#6789) Stephannie Jimenez Gacha
Add doc on managing environments (GH#6778) Martin Durant
HLG:
get_all_external_keys()
(GH#6774) Mads R. B. KristensenAvoid rechunking in reshape with
chunksize=1
(GH#6748) Tom AugspurgerTry to make categoricals work on join (GH#6205) Julia Signell
Fix some minor typos and trailing whitespaces in
array-slice.rst
(GH#6771) Magnus NordBugfix for parquet metadata writes of empty dataframe partitions (pyarrow) (GH#6741) Callum Noble
Document
meta
kwarg inmap_blocks
andmap_overlap
. (GH#6763) Peter Andreas EntschevBegin experimenting with parallel prefix scan for
cumsum
andcumprod
(GH#6675) Erik WelchClarify differences in boolean indexing between dask and numpy arrays (GH#6764) Illviljan
Efficient serialization of shuffle layers (GH#6760) James Bourbeau
Config array optimize to skip fusion and return a HLG (GH#6751) Mads R. B. Kristensen
Temporarily use
pyarrow<2
in CI (GH#6759) James BourbeauFix meta for
min
/max
reductions (GH#6736) Peter Andreas EntschevAdd 2D possibility to
da.linalg.lstsq
- mirroring numpy (GH#6749) Pascal BourgaultCI: Fixed bug causing flaky test failure in pivot (GH#6752) Tom Augspurger
Serialization of layers (GH#6693) Mads R. B. Kristensen
Removed Mutable Default Argument (GH#6747) Mads R. B. Kristensen
Adjust parquet
ArrowEngine
to allow more easy subclass for writing (GH#6505) Joris Van den BosscheAdd
ShuffleStage
HLG Layer (GH#6650) Richard (Rick) ZamoraHandle literal in
meta_from_array
(GH#6731) Peter Andreas EntschevDo balanced rechunking even if chunks are the same (GH#6735) Chris Roat
Fix docstring
DataFrame.set_index
(GH#6739) Gil ForsythEnsure
HighLevelGraph
layers always containLayer
instances (GH#6716) James BourbeauMap on
HighLevelGraph
Layers (GH#6689) Mads R. B. KristensenUpdate overlap
*_like
function calls and CuPy tests (GH#6728) Peter Andreas EntschevFixes for
svd
with__array_function__
(GH#6727) Peter Andreas EntschevAdded doctest extension for documentation (GH#6397) Jim Circadian
Minor fix to #5628 using @pentschev’s suggestion (GH#6724) John A Kirkham
Change type of Dask array when meta type changes (GH#5628) Matthew Rocklin
HLG:
get_dependencies()
of single keys (GH#6699) Mads R. B. KristensenRevert “Revert “Use HighLevelGraph layers everywhere in collections (GH#6510)” (GH#6697)” (GH#6707) Tom Augspurger
Allow
*_like
array creation functions to respect input array type (GH#6680) Genevieve BuckleyUpdate
dask-sphinx-theme
version (GH#6700) Gil Forsyth
2.30.0 / 2020-10-06¶
Array¶
Allow
rechunk
to evenly split into N chunks (GH#6420) Scott Sievert
2.29.0 / 2020-10-02¶
Array¶
_repr_html_
: color sides darker instead of drawing all the lines (GH#6683) Julia SignellRemoves warning from
nanstd
andnanvar
(GH#6667) Thomas J. FanGet shape of output from original array -
map_overlap
(GH#6682) Julia SignellReplace
np.searchsorted
withbisect
in indexing (GH#6669) Joachim B Haga
Bag¶
Make sure subprocesses have a consistent hash for bag
groupby
(GH#6660) Itamar Turner-Trauring
Core¶
Revert “Use
HighLevelGraph
layers everywhere in collections (GH#6510)” (GH#6697) Tom AugspurgerUse
pandas.testing
(GH#6687) John A KirkhamImprove 128-bit floating-point skip in tests (GH#6676) Elliott Sales de Andrade
DataFrame¶
Allow setting dataframe items using a bool dataframe (GH#6608) Julia Signell
2.28.0 / 2020-09-25¶
Array¶
Partially reverted changes to
Array
indexing that produces large changes. This restores the behavior from Dask 2.25.0 and earlier, with a warning when large chunks are produced. A configuration option is provided to avoid creating the large chunks, see Efficiency. (GH#6665) Tom AugspurgerAdd
meta
toto_dask_array
(GH#6651) Kyle NicholsonFix GH#6631 and GH#6611 (GH#6632) Rafal Wojdyla
Infer object in array reductions (GH#6629) Daniel Saxton
Adding
v_based
flag forsvd_flip
(GH#6658) Eric CzechFix flakey array
mean
(GH#6656) Sam Grayson
Core¶
Removed
dsk
equality check fromSubgraphCallable.__eq__
(GH#6666) Mads R. B. KristensenUse
HighLevelGraph
layers everywhere in collections (GH#6510) Mads R. B. KristensenAdds hash dunder method to
SubgraphCallable
for caching purposes (GH#6424) Andrew FultonStop writing commented out config files by default (GH#6647) Matthew Rocklin
DataFrame¶
Add support for collect list aggregation via
agg
API (GH#6655) Madhur TandonSlightly better error message (GH#6657) Julia Signell
2.27.0 / 2020-09-18¶
Array¶
Preserve
dtype
insvd
(GH#6643) Eric Czech
Core¶
store()
: create a single HLG layer (GH#6601) Mads R. B. KristensenAdd pre-commit CI build (GH#6645) James Bourbeau
Update
.pre-commit-config
to latest black. (GH#6641) Julia SignellUpdate super usage to remove Python 2 compatibility (GH#6630) Poruri Sai Rahul
Remove u string prefixes (GH#6633) Poruri Sai Rahul
DataFrame¶
Improve error message for
to_sql
(GH#6638) Julia SignellUse empty list as categories (GH#6626) Julia Signell
Documentation¶
Add
autofunction
to array api docs for more ufuncs (GH#6644) James BourbeauAdd a number of missing ufuncs to
dask.array
docs (GH#6642) Ralf GommersAdd
HelmCluster
docs (GH#6290) Jacob Tomlinson
2.26.0 / 2020-09-11¶
Array¶
Backend-aware dtype inference for single-chunk svd (GH#6623) Eric Czech
Make
array.reduction
docstring match for dtype (GH#6624) Martin DurantSet lower bound on compression level for
svd_compressed
using rows and cols (GH#6622) Eric CzechImprove SVD consistency and small array handling (GH#6616) Eric Czech
Add
svd_flip
#6599 (GH#6613) Eric CzechHandle sequences containing dask Arrays (GH#6595) Gabe Joseph
Avoid large chunks from
getitem
with lists (GH#6514) Tom AugspurgerEagerly slice numpy arrays in
from_array
(GH#6605) Deepak CherianRestore ability to pickle dask arrays (GH#6594) Noah D. Brenowitz
Add SVD support for short-and-fat arrays (GH#6591) Eric Czech
Add simple chunk type registry and defer as appropriate to upcast types (GH#6393) Jon Thielen
Align coarsen chunks by default (GH#6580) Deepak Cherian
Fixup reshape on unknown dimensions and other testing fixes (GH#6578) Ryan Williams
Core¶
Add validation and fixes for
HighLevelGraph
dependencies (GH#6588) Mads R. B. KristensenFix linting issue (GH#6598) Tom Augspurger
Skip
bokeh
version 2.0.0 (GH#6572) John A Kirkham
DataFrame¶
Added bytes/row calculation when using meta (GH#6585) McToel
Handle
min_count
inSeries.sum
/prod
(GH#6618) Daniel SaxtonAlways compute 0 and 1 quantiles during quantile calculations (GH#6564) Erik Welch
Fix wrong path when reading empty csv file (GH#6573) Abdulelah Bin Mahfoodh
Documentation¶
Doc: Troubleshooting dashboard 404 (GH#6215) Kilian Lieret
Fixup
extraConfig
example (GH#6625) Tom AugspurgerUpdate supported Python versions (GH#6609) Julia Signell
Document dask/daskhub helm chart (GH#6560) Tom Augspurger
2.25.0 / 2020-08-28¶
Core¶
Compare key hashes in
subs()
(GH#6559) Mads R. B. KristensenRerun with latest
black
release (GH#6568) James BourbeauLicense update (GH#6554) Tom Augspurger
Documentation¶
Remove version from documentation page names (GH#6558) James Bourbeau
Update
kubernetes-helm.rst
(GH#6523) David SheldonStop 2020 survey (GH#6547) Tom Augspurger
2.24.0 / 2020-08-22¶
Array¶
Fix setting random seed in tests. (GH#6518) Elliott Sales de Andrade
Support meta in apply gufunc (GH#6521) joshreback
Replace cupy.sparse with cupyx.scipy.sparse (GH#6530) John A Kirkham
Dataframe¶
Bump up tolerance for rolling tests (GH#6502) Julia Signell
Implement DatFrame.__len__ (GH#6515) Tom Augspurger
Infer arrow schema in to_parquet (for ArrowEngine`) (GH#6490) Richard (Rick) Zamora
Fix parquet test when no pyarrow (GH#6524) Martin Durant
Remove problematic
filter
arguments in ArrowEngine (GH#6527) Richard (Rick) ZamoraAvoid schema validation by default in ArrowEngine (GH#6536) Richard (Rick) Zamora
Core¶
Use unpack_collections in make_blockwise_graph (GH#6517) Thomas J. Fan
Move key_split() from optimization.py to utils.py (GH#6529) Mads R. B. Kristensen
Make tests run on moto server (GH#6528) Martin Durant
2.23.0 / 2020-08-14¶
Array¶
Reduce
np.zeros
,ones
, andfull
array size with broadcasting (GH#6491) Matthias BussonnierAdd missing
meta=
fortrim
inmap_overlap
(GH#6494) Peter Andreas Entschev
Bag¶
Bag repartition partition size (GH#6371) joshreback
Core¶
Scalar.__dask_layers__()
to returnself._name
instead ofself.key
(GH#6507) Mads R. B. KristensenUpdate dependencies correctly in
fuse_root
optimization (GH#6508) Mads R. B. Kristensen
DataFrame¶
Adds
items
to dataframe (GH#6503) Thomas J. FanInclude compression in
write_table
call (GH#6499) Julia SignellFixed warning in
nonempty_series
(GH#6485) Tom AugspurgerIntelligently determine partitions based on type of first arg (GH#6479) Matthew Rocklin
Fix pyarrow
mkdirs
(GH#6475) Julia SignellFix duplicate parquet output in
to_parquet
(GH#6451) michaelnarodovitch
Documentation¶
Fix documentation
da.histogram
(GH#6439) Roberto PanaiFixed a few typos in the SQL docs (GH#6489) Mike McCarty
Docs for SQLing (GH#6453) Martin Durant
2.22.0 / 2020-07-31¶
Array¶
Compatibility for NumPy dtype deprecation (GH#6430) Tom Augspurger
Core¶
Implement
sizeof
for somebytes
-like objects (GH#6457) John A KirkhamHTTP error for new
fsspec
(GH#6446) Martin DurantWhen
RecursionError
is raised, return uuid fromtokenize
function (GH#6437) Julia SignellInstall deps of upstream-dev packages (GH#6431) Tom Augspurger
Use updated link in
setup.cfg
(GH#6426) Zhengnan Zhao
DataFrame¶
Add single quotes around column names if strings (GH#6471) Gil Forsyth
Refactor
ArrowEngine
for betterread_parquet
performance (GH#6346) Richard (Rick) ZamoraAdd
tolist
dispatch (GH#6444) GALI PREM SAGARCompatibility with pandas 1.1.0rc0 (GH#6429) Tom Augspurger
Multi value pivot table (GH#6428) joshreback
Duplicate argument definitions in
to_csv
docstring (GH#6411) Jun Han (Johnson) Ooi
Documentation¶
Add utility to docs to convert YAML config to env vars and back (GH#6472) Jacob Tomlinson
Fix parameter server rendering (GH#6466) Scott Sievert
Fixes broken links (GH#6403) Jim Circadian
Complete parameter server implementation in docs (GH#6449) Scott Sievert
Fix typo (GH#6436) Jack Xiaosong Xu
2.21.0 / 2020-07-17¶
Array¶
Correct error message in
array.routines.gradient()
(GH#6417) johnomotaniFix blockwise concatenate for array with some
dimension=1
(GH#6342) Matthias Bussonnier
Bag¶
Fix
bag.take
example (GH#6418) Roberto Panai
Core¶
Groups values in optimization pass should only be graph and keys – not an optimization + keys (GH#6409) Benjamin Zaitlen
Call custom optimizations once, with
kwargs
provided (GH#6382) Clark ZinzowInclude
pickle5
for testing on Python 3.7 (GH#6379) John A Kirkham
DataFrame¶
Correct typo in error message (GH#6422) Tom McTiernan
Use
pytest.warns
to check forUserWarning
(GH#6378) Richard (Rick) ZamoraParse
bytes_per_chunk keyword
from string (GH#6370) Matthew Rocklin
Documentation¶
Numpydoc formatting (GH#6421) Matthias Bussonnier
Unpin
numpydoc
following 1.1 release (GH#6407) Gil ForsythNumpydoc formatting (GH#6402) Matthias Bussonnier
Add instructions for using conda when installing code for development (GH#6399) Ray Bell
Update
visualize
docstrings (GH#6383) Zhengnan Zhao
2.20.0 / 2020-07-02¶
Array¶
Register
sizeof
for numpy zero-strided arrays (GH#6343) Matthias BussonnierUse
concatenate_lookup
inconcatenate
(GH#6339) John A KirkhamFix rechunking of arrays with some zero-length dimensions (GH#6335) Matthias Bussonnier
DataFrame¶
Dispatch
iloc`
calls togetitem
(GH#6355) Gil ForsythHandle unnamed pandas
RangeIndex
in fastparquet engine (GH#6350) Richard (Rick) ZamoraPreserve index when writing partitioned parquet datasets with pyarrow (GH#6282) Richard (Rick) Zamora
Use
ignore_index
for pandas’group_split_dispatch
(GH#6251) Richard (Rick) Zamora
2.19.0 / 2020-06-19¶
Array¶
Cast chunk sizes to python int
dtype
(GH#6326) Gil ForsythAdd
shape=None
to*_like()
array creation functions (GH#6064) Anderson Banihirwe
Core¶
Update expected error msg for protocol difference in fsspec (GH#6331) Gil Forsyth
Fix for floats < 1 in
parse_bytes
(GH#6311) Gil ForsythFix exception causes all over the codebase (GH#6308) Ram Rachum
Fix duplicated tests (GH#6303) James Lamb
Remove unused testing function (GH#6304) James Lamb
DataFrame¶
Add high-level CSV Subgraph (GH#6262) Gil Forsyth
Fix
ValueError
when merging an index-only 1-partition dataframe (GH#6309) Krishan BhasinMake
index.map
clear divisions. (GH#6285) Julia Signell
Documentation¶
Add link to 2020 survey (GH#6328) Tom Augspurger
Update
bag.rst
(GH#6317) Ben Shaver
2.18.1 / 2020-06-09¶
Array¶
Don’t try to set name on
full
(GH#6299) Julia SignellHistogram: support lazy values for range/bins (another way) (GH#6252) Gabe Joseph
Core¶
Fix exception causes in
utils.py
(GH#6302) Ram RachumImprove performance of
HighLevelGraph
construction (GH#6293) Julia Signell
Documentation¶
Now readthedocs builds unrelased features’ docstrings (GH#6295) Antonio Ercole De Luca
Add
asyncssh
intersphinx mappings (GH#6298) Jacob Tomlinson
2.18.0 / 2020-06-05¶
Array¶
Cast slicing index to dask array if same shape as original (GH#6273) Julia Signell
Fix
stack
error message (GH#6268) Stephanie Gottfull
&full_like
: error on non-scalarfill_value
(GH#6129) HuiteSupport for multiple arrays in
map_overlap
(GH#6165) Eric CzechPad resample divisions so that edges are counted (GH#6255) Julia Signell
Bag¶
Random sampling of k elements from a dask bag #4799 (GH#6239) Antonio Ercole De Luca
DataFrame¶
Add
dropna
,sort
, andascending
tosort_values
(GH#5880) Julia SignellGeneralize
from_dask_array
(GH#6263) GALI PREM SAGARAdd derived docstring for
SeriesGroupby.nunique
(GH#6284) Julia SignellRemove
NotImplementedError
in resample with rule (GH#6274) Abdulelah Bin MahfoodhAdd
dd.to_sql
(GH#6038) Ryan Williams
2.17.2 / 2020-05-28¶
Core¶
Re-add the
complete
extra (GH#6257) Jim Crist-Harif
DataFrame¶
Raise error if
resample
isn’t going to give right answer (GH#6244) Julia Signell
2.17.1 / 2020-05-28¶
Array¶
Empty array rechunk (GH#6233) Andrew Fulton
Core¶
Make
pyyaml
required (GH#6250) Jim Crist-HarifFix install commands from
ImportError
(GH#6238) Gaurav SheniRemove issue template (GH#6249) Jacob Tomlinson
DataFrame¶
Pass
ignore_index
todd_shuffle
fromDataFrame.shuffle
(GH#6247) Richard (Rick) ZamoraCope with missing HDF keys (GH#6204) Martin Durant
Generalize
describe
&quantile
apis (GH#5137) GALI PREM SAGAR
2.17.0 / 2020-05-26¶
Array¶
Bag¶
Random Choice on Bags (GH#6208) Antonio Ercole De Luca
Core¶
Raise warning
delayed.visualise()
(GH#6216) Amol UmbarkarEnsure other pickle arguments work (GH#6229) John A Kirkham
Overhaul
fuse()
config (GH#6198) crusaderkyUpdate
dask.order.order
to consider “next” nodes using both FIFO and LIFO (GH#5872) Erik Welch
DataFrame¶
Use 0 as
fill_value
for more agg methods (GH#6245) Julia SignellGeneralize
rearrange_by_column_tasks
and addDataFrame.shuffle
(GH#6066) Richard (Rick) ZamoraXfail
test_rolling_numba_engine
for newer numba and older pandas (GH#6236) James BourbeauGeneralize
fix_overlap
(GH#6240) GALI PREM SAGARAvoid shuffle when setting a presorted index with overlapping divisions (GH#6226) Krishan Bhasin
Adjust the Parquet engine classes to allow more easily subclassing (GH#6211) Marius van Niekerk
Fix
dd.merge_asof
withleft_on='col'
&right_index=True
(GH#6192) noreentryMove
AUTO_BLOCKSIZE
out ofread_csv
signature (GH#6214) Jim Crist-Harif.loc
indexing with callable (GH#6185) Endre Mark BorzaAvoid apply in
_compute_sum_of_squares
for groupby std agg (GH#6186) Richard (Rick) ZamoraMinor correction to
test_parquet
(GH#6190) Brian LarsenAdhering to the passed pat for delimeter join and fix error message (GH#6194) GALI PREM SAGAR
Skip
test_to_parquet_with_get
if no parquet libs available (GH#6188) Scott Sanderson
Documentation¶
Added documentation for
distributed.Event
class (GH#6231) Nils Braun
2.16.0 / 2020-05-08¶
Array¶
Fix array general-reduction name (GH#6176) Nick Evans
Replace
dim
withshape
inunravel_index
(GH#6155) Julia SignellMoment: handle all elements being masked (GH#5339) Gabe Joseph
Core¶
Remove Redundant string concatenations in dask code-base (GH#6137) GALI PREM SAGAR
Upstream compat (GH#6159) Tom Augspurger
Ensure
sizeof
of dict and sequences returns an integer (GH#6179) James BourbeauEstimate python collection sizes with random sampling (GH#6154) Florian Jetter
Update test upstream (GH#6146) Tom Augspurger
Skip test for mindeps build (GH#6144) Tom Augspurger
Switch default multiprocessing context to “spawn” (GH#4003) Itamar Turner-Trauring
Update manifest to include dask-schema (GH#6140) Benjamin Zaitlen
DataFrame¶
Harden inconsistent-schema handling in pyarrow-based
read_parquet
(GH#6160) Richard (Rick) ZamoraAdd compute
kwargs
to methods that write data to disk (GH#6056) Krishan BhasinFix issue where
unique
returns an index like result from backends (GH#6153) GALI PREM SAGARFix internal error in
map_partitions
with collections (GH#6103) Tom Augspurger
Documentation¶
Add phase of computation to index TOC (GH#6157) Benjamin Zaitlen
Remove unused imports in scheduling script (GH#6138) James Lamb
Fix indent (GH#6147) Martin Durant
Add Tom’s log config example (GH#6143) Martin Durant
2.15.0 / 2020-04-24¶
Array¶
Update
dask.array.from_array
to warn when passed a Dask collection (GH#6122) James BourbeauUn-numpy like behaviour in
dask.array.pad
(GH#6042) Mark BoerAdd support for
repeats=0
inda.repeat
(GH#6080) James Bourbeau
Core¶
Fix yaml layout for schema (GH#6132) Benjamin Zaitlen
Configuration Reference (GH#6069) Benjamin Zaitlen
Add configuration option to turn off task fusion (GH#6087) Matthew Rocklin
Skip pyarrow on windows (GH#6094) Tom Augspurger
Set limit to maximum length of fused key (GH#6057) Lucas Rademaker
Add test against #6062 (GH#6072) Martin Durant
Bump checkout action to v2 (GH#6065) James Bourbeau
DataFrame¶
Generalize categorical calls to support cudf
Categorical
(GH#6113) GALI PREM SAGARAvoid reading
_metadata
on every worker (GH#6017) Richard (Rick) ZamoraUse
group_split_dispatch
andignore_index
inapply_concat_apply
(GH#6119) Richard (Rick) ZamoraHandle new (dtype) pandas metadata with pyarrow (GH#6090) Richard (Rick) Zamora
Skip
test_partition_on_cats_pyarrow
if pyarrow is not installed (GH#6112) James BourbeauUpdate DataFrame len to handle columns with the same name (GH#6111) James Bourbeau
ArrowEngine
bug fixes and test coverage (GH#6047) Richard (Rick) ZamoraAdded mode (GH#5958) Adam Lewis
Documentation¶
Extend preload documentation (GH#6077) Matthew Rocklin
Fixed small typo in DataFrame
map_partitions()
docstring (GH#6115) Eugene HuangFix typo: “double” should be times, not plus (GH#6091) David Chudzicki
Fix first line of
array.random.*
docs (GH#6063) Martin DurantAdd section about
Semaphore
in distributed (GH#6053) Florian Jetter
2.14.0 / 2020-04-03¶
Array¶
Added
np.iscomplexobj
implementation (GH#6045) Tom Augspurger
Core¶
Update
test_rearrange_disk_cleanup_with_exception
to pass without cloudpickle installed (GH#6052) James BourbeauFixed flaky
test-rearrange
(GH#5977) Tom Augspurger
DataFrame¶
Use
_meta_nonempty
for dtype casting instack_partitions
(GH#6061) mlondschienFix bugs in
_metadata
creation and filtering in parquetArrowEngine
(GH#6023) Richard (Rick) Zamora
Documentation¶
DOC: Add name caveats (GH#6040) Tom Augspurger
2.13.0 / 2020-03-25¶
Array¶
Support
dtype
and other keyword arguments inda.random
(GH#6030) Matthew RocklinRegister support for
cupy
sparsehstack
/vstack
(GH#5735) Corey J. NoletForce
self.name
tostr
indask.array
(GH#6002) Chuanzhu Xu
Bag¶
Set
rename_fused_keys
toNone
by default inbag.optimize
(GH#6000) Lucas Rademaker
Core¶
Copy dict in
to_graphviz
to prevent overwriting (GH#5996) JulianWgsStricter pandas
xfail
(GH#6024) Tom AugspurgerFix CI failures (GH#6013) James Bourbeau
Update
toolz
to 0.8.2 and usetlz
(GH#5997) Ryan GroutMove Windows CI builds to GitHub Actions (GH#5862) James Bourbeau
DataFrame¶
Improve path-related exceptions in
read_hdf
(GH#6032) psimajFix
dtype
handling indd.concat
(GH#6006) mlondschienHandle cudf’s leftsemi and leftanti joins (GH#6025) Richard J Zamora
Remove unused
npartitions
variable indd.from_pandas
(GH#6019) Daniel Saxton
Documentation¶
Fix indentation in scheduler-overview docs (GH#6022) Matthew Rocklin
Update task graphs in optimize docs (GH#5928) Julia Signell
Optionally get rid of intermediary boxes in visualize, and add more labels (GH#5976) Julia Signell
2.12.0 / 2020-03-06¶
Array¶
Improve reuse of temporaries with numpy (GH#5933) Bruce Merry
Make
map_blocks
withblock_info
produce aBlockwise
(GH#5896) Bruce MerryOptimize
make_blockwise_graph
(GH#5940) Bruce MerryFix axes ordering in
da.tensordot
(GH#5975) Gil ForsythAdds empty mode to
array.pad
(GH#5931) Thomas J. Fan
Core¶
Remove
toolz.memoize
dependency indask.utils
(GH#5978) Ryan GroutClose pool leaking subprocess (GH#5979) Tom Augspurger
Pin
numpydoc
to0.8.0
(fix double autoescape) (GH#5961) Gil ForsythRegister deterministic tokenization for
range
objects (GH#5947) James BourbeauUnpin
msgpack
in CI (GH#5930) JAmes BourbeauEnsure dot results are placed in unique files. (GH#5937) Elliott Sales de Andrade
Add remaining optional dependencies to Travis 3.8 CI build environment (GH#5920) James Bourbeau
DataFrame¶
Skip parquet
getitem
optimization for some keys (GH#5917) Tom AugspurgerAdd
ignore_index
argument torearrange_by_column
code path (GH#5973) Richard J ZamoraAdd DataFrame and Series
memory_usage_per_partition
methods (GH#5971) James Bourbeauxfail
test_describe when using Pandas 0.24.2 (GH#5948) James BourbeauImplement
dask.dataframe.to_numeric
(GH#5929) Julia SignellAdd new error message content when columns are in a different order (GH#5927) Julia Signell
Use shallow copy for assign operations when possible (GH#5740) Richard J Zamora
Documentation¶
Changed above to below in
dask.array.triu
docs (GH#5984) Henrik AnderssonArray slicing: fix typo in
slice_with_int_dask_array
error message (GH#5981) Gabe JosephGrammar and formatting updates to docstrings (GH#5963) James Lamb
Update title of DataFrame extension docs (GH#5954) James Bourbeau
Fixed typos in documentation (GH#5962) James Lamb
Add original class or module as a
kwarg
on_bind_*
methods (GH#5946) Julia SignellUpdate optimization doc for python 3 (GH#5926) Julia Signell
2.11.0 / 2020-02-19¶
Array¶
Cache result of
Array.shape
(GH#5916) Bruce MerryImprove accuracy of
estimate_graph_size
forrechunk
(GH#5907) Bruce MerrySkip rechunk steps that do not alter chunking (GH#5909) Bruce Merry
Support
dtype
and otherkwargs
incoarsen
(GH#5903) Matthew RocklinPush chunk override from
map_blocks
into blockwise (GH#5895) Bruce MerryAvoid using
rewrite_blockwise
for a singleton (GH#5890) Bruce MerryOptimize
slices_from_chunks
(GH#5891) Bruce MerryAvoid unnecessary
__getitem__
inblock()
when chunks have correct dimensionality (GH#5884) Thomas Robitaille
Bag¶
Add
include_path
option fordask.bag.read_text
(GH#5836) Yifan GuFixes
ValueError
in delayed execution of bagged NumPy array (GH#5828) Surya Avala
Core¶
CI: Pin
msgpack
(GH#5923) Tom AugspurgerRename
test_inner
totest_outer
(GH#5922) Shiva Raisinghaniquote
should quote dicts too (GH#5905) Bruce MerryRegister a normalizer for literal (GH#5898) Bruce Merry
Improve layer name synthesis for non-HLGs (GH#5888) Bruce Merry
Replace flake8 pre-commit-hook with upstream (GH#5892) Julia Signell
Call pip as a module to avoid warnings (GH#5861) Cyril Shcherbin
Close
ThreadPool
at exit (GH#5852) Tom AugspurgerRemove
dask.dataframe
import in tokenization code (GH#5855) James Bourbeau
DataFrame¶
Require
pandas>=0.23
(GH#5883) Tom AugspurgerRemove lambda from dataframe aggregation (GH#5901) Matthew Rocklin
Fix exception chaining in
dataframe/__init__.py
(GH#5882) Ram RachumAdd support for reductions on empty dataframes (GH#5804) Shiva Raisinghani
Expose
sort=
argument for groupby (GH#5801) Richard J ZamoraUse parquet read speed-ups from
fastparquet.api.paths_to_cats
. (GH#5821) Igor Gotlibovych
Documentation¶
Deprecate
doc_wraps
(GH#5912) Tom AugspurgerUpdate array internal design docs for HighLevelGraph era (GH#5889) Bruce Merry
Move over dashboard connection docs (GH#5877) Matthew Rocklin
Move prometheus docs from distributed.dask.org (GH#5876) Matthew Rocklin
Removing duplicated DO block at the end (GH#5878) K.-Michael Aye
map_blocks
see also (GH#5874) Tom AugspurgerMore derived from (GH#5871) Julia Signell
Fix typo (GH#5866) Yetunde Dada
Fix typo in
cloud.rst
(GH#5860) Andrew ThomasAdd note pointing to code of conduct and diversity statement (GH#5844) Matthew Rocklin
2.10.1 / 2020-01-30¶
Fix Pandas 1.0 version comparison (GH#5851) Tom Augspurger
Fix typo in distributed diagnostics documentation (GH#5841) Gerrit Holl
2.10.0 / 2020-01-28¶
Support for pandas 1.0’s new
BooleanDtype
andStringDtype
(GH#5815) Tom AugspurgerCompatibility with pandas 1.0’s API breaking changes and deprecations (GH#5792) Tom Augspurger
Fixed non-deterministic tokenization of some extension-array backed pandas objects (GH#5813) Tom Augspurger
Fixed handling of dataclass class objects in collections (GH#5812) Matteo De Wint
Fixed resampling with tz-aware dates when one of the endpoints fell in a non-existent time (GH#5807) dfonnegra
Delay initial Zarr dataset creation until the computation occurs (GH#5797) Chris Roat
Use parquet dataset statistics in more cases with the
pyarrow
engine (GH#5799) Richard J ZamoraFixed exception in
groupby.std()
when some of the keys were large integers (GH#5737) H. Thomson Comer
2.9.2 / 2020-01-16¶
Array¶
Unify chunks in
broadcast_arrays
(GH#5765) Matthew Rocklin
Core¶
xfail
CSV encoding tests (GH#5791) Tom AugspurgerUpdate order to handle empty dask graph (GH#5789) James Bourbeau
Redo
dask.order.order
(GH#5646) Erik Welch
DataFrame¶
Add transparent compression for on-disk shuffle with
partd
(GH#5786) Christian WespFix
repr
for empty dataframes (GH#5781) Shiva RaisinghaniPandas 1.0.0RC0 compat (GH#5784) Tom Augspurger
Remove buggy assertions (GH#5783) Tom Augspurger
Pandas 1.0 compat (GH#5782) Tom Augspurger
Fix bug in pyarrow-based
read_parquet
on partitioned datasets (GH#5777) Richard J ZamoraCompat for pandas 1.0 (GH#5779) Tom Augspurger
Fix groupby/mean error with with categorical index (GH#5776) Richard J Zamora
Support empty partitions when performing cumulative aggregation (GH#5730) Matthew Rocklin
set_index
accepts single-item unnested list (GH#5760) Wes RoachFixed partitioning in set index for ordered
Categorical
(GH#5715) Tom Augspurger
Documentation¶
Note additional use case for
normalize_token.register
(GH#5766) Thomas A CaswellSmall typos (GH#5771) Maarten Breddels
Fix typo in Task Expectations docs (GH#5767) James Bourbeau
Add docs section on task expectations to graph page (GH#5764) Devin Petersohn
2.9.1 / 2019-12-27¶
Array¶
Support Array.view with dtype=None (GH#5736) Anderson Banihirwe
Add dask.array.nanmedian (GH#5684) Deepak Cherian
Core¶
xfail test_temporary_directory on Python 3.8 (GH#5734) James Bourbeau
Add support for Python 3.8 (GH#5603) James Bourbeau
Use id to dedupe constants in rewrite_blockwise (GH#5696) Jim Crist
DataFrame¶
Raise error when converting a dask dataframe scalar to a boolean (GH#5743) James Bourbeau
Ensure dataframe groupby-variance is greater than zero (GH#5728) Matthew Rocklin
Fix DataFrame.__iter__ (GH#5719) Tom Augspurger
Support Parquet filters in disjunctive normal form, like PyArrow (GH#5656) Matteo De Wint
Auto-detect categorical columns in ArrowEngine-based read_parquet (GH#5690) Richard J Zamora
Skip parquet getitem optimization tests if no engine found (GH#5697) James Bourbeau
Fix independent optimization of parquet-getitem (GH#5613) Tom Augspurger
Documentation¶
Link to examples.dask.org in several places (GH#5733) Tom Augspurger
Add missing ” in performance report example (GH#5724) James Bourbeau
Resolve several documentation build warnings (GH#5685) James Bourbeau
add info on performance_report (GH#5713) Benjamin Zaitlen
Add more docs disclaimers (GH#5710) Julia Signell
Update numpydoc dependency (GH#5694) James Bourbeau
2.9.0 / 2019-12-06¶
Array¶
Fix
da.std
to work with NumPy arrays (GH#5681) James Bourbeau
Core¶
Register
sizeof
functions for Numba and RMM (GH#5668) John A KirkhamUpdate meeting time (GH#5682) Tom Augspurger
DataFrame¶
Modify
dd.DataFrame.drop
to use shallow copy (GH#5675) Richard J ZamoraFix bug in
_get_md_row_groups
(GH#5673) Richard J ZamoraClose sqlalchemy engine after querying DB (GH#5629) Krishan Bhasin
Allow
dd.map_partitions
to not enforce meta (GH#5660) Matthew RocklinGeneralize
concat_unindexed_dataframes
to support cudf-backend (GH#5659) Richard J ZamoraAdd dataframe resample methods (GH#5636) Benjamin Zaitlen
Compute length of dataframe as length of first column (GH#5635) Matthew Rocklin
Documentation¶
Doc fixup (GH#5665) James Bourbeau
Update doc build instructions (GH#5640) James Bourbeau
Add documentation build (GH#5617) James Bourbeau
2.8.1 / 2019-11-22¶
Array¶
Use auto rechunking in
da.rechunk
if no value given (GH#5605) Matthew Rocklin
Core¶
Add simple action to activate GH actions (GH#5619) James Bourbeau
DataFrame¶
Fix “file_path_0” bug in
aggregate_row_groups
(GH#5627) Richard J ZamoraAdd
chunksize
argument toread_parquet
(GH#5607) Richard J ZamoraChange
test_repartition_npartitions
to support arch64 architecture (GH#5620) ossdev07Categories lost after groupby + agg (GH#5423) Oliver Hofkens
Fixed relative path issue with parquet metadata file (GH#5608) Nuno Gomes Silva
Enable gpu-backed covariance/correlation in dataframes (GH#5597) Richard J Zamora
Documentation¶
Fix institutional faq and unknown doc warnings (GH#5616) James Bourbeau
Add doc for some utils (GH#5609) Tom Augspurger
Removes
html_extra_path
(GH#5614) James BourbeauFixed See Also referencence (GH#5612) Tom Augspurger
2.8.0 / 2019-11-14¶
Array¶
Implement complete dask.array.tile function (GH#5574) Bouwe Andela
Add median along an axis with automatic rechunking (GH#5575) Matthew Rocklin
Allow da.asarray to chunk inputs (GH#5586) Matthew Rocklin
Bag¶
Use key_split in Bag name (GH#5571) Matthew Rocklin
Core¶
Switch Doctests to Py3.7 (GH#5573) Ryan Nazareth
Relax get_colors test to adapt to new Bokeh release (GH#5576) Matthew Rocklin
Add dask.blockwise.fuse_roots optimization (GH#5451) Matthew Rocklin
Add sizeof implementation for small dicts (GH#5578) Matthew Rocklin
Update fsspec, gcsfs, s3fs (GH#5588) Tom Augspurger
DataFrame¶
Add dropna argument to groupby (GH#5579) Richard J Zamora
Revert “Remove import of dask_cudf, which is now a part of cudf (GH#5568)” (GH#5590) Matthew Rocklin
Documentation¶
Add best practice for dask.compute function (GH#5583) Matthew Rocklin
Create FUNDING.yml (GH#5587) Gina Helfrich
Add screencast for coordination primitives (GH#5593) Matthew Rocklin
Move funding to .github repo (GH#5589) Tom Augspurger
Update calendar link (GH#5569) Tom Augspurger
2.7.0 / 2019-11-08¶
This release drops support for Python 3.5
Array¶
Update da.array to always return a dask array (GH#5510) James Bourbeau
Skip transpose on trivial inputs (GH#5523) Ryan Abernathey
Avoid NumPy scalar string representation in tokenize (GH#5527) James Bourbeau
Remove unnecessary tiledb shape constraint (GH#5545) Norman Barker
Removes bytes from sparse array HTML repr (GH#5556) James Bourbeau
Core¶
Drop Python 3.5 (GH#5528) James Bourbeau
Update the use of fixtures in distributed tests (GH#5497) Matthew Rocklin
Changed deprecated bokeh-port to dashboard-address (GH#5507) darindf
Avoid updating with identical dicts in ensure_dict (GH#5501) James Bourbeau
Test Upstream (GH#5516) Tom Augspurger
Accelerate reverse_dict (GH#5479) Ryan Grout
Update test_imports.sh (GH#5534) James Bourbeau
Support cgroups limits on cpu count in multiprocess and threaded schedulers (GH#5499) Albert DeFusco
Update minimum pyarrow version on CI (GH#5562) James Bourbeau
Make cloudpickle optional (GH#5511) crusaderky
DataFrame¶
Add an example of index_col usage (GH#3072) Bruno Bonfils
Explicitly use iloc for row indexing (GH#5500) Krishan Bhasin
Accept dask arrays on columns assignemnt (GH#5224) Henrique Ribeiro-
Implement unique and value_counts for SeriesGroupBy (GH#5358) Scott Sievert
Add sizeof definition for pyarrow tables and columns (GH#5522) Richard J Zamora
Enable row-group task partitioning in pyarrow-based read_parquet (GH#5508) Richard J Zamora
Removes npartitions=’auto’ from dd.merge docstring (GH#5531) James Bourbeau
Apply enforce error message shows non-overlapping columns. (GH#5530) Tom Augspurger
Optimize meta_nonempty for repetitive dtypes (GH#5553) Petio Petrov
Remove import of dask_cudf, which is now a part of cudf (GH#5568) Mads R. B. Kristensen
Documentation¶
Make capitalization more consistent in FAQ docs (GH#5512) Matthew Rocklin
Add CONTRIBUTING.md (GH#5513) Jacob Tomlinson
Document optional dependencies (GH#5456) Prithvi MK
Update helm chart docs to reflect new chart repo (GH#5539) Jacob Tomlinson
Add Resampler to API docs (GH#5551) James Bourbeau
Add adaptive deployments screencast [skip ci] (GH#5566) Matthew Rocklin
2.6.0 / 2019-10-15¶
Core¶
Call
ensure_dict
on graphs before enteringtoolz.merge
(GH#5486) Matthew RocklinConsolidating hash dispatch functions (GH#5476) Richard J Zamora
DataFrame¶
Support Python 3.5 in Parquet code (GH#5491) Benjamin Zaitlen
Avoid identity check in
warn_dtype_mismatch
(GH#5489) Tom AugspurgerEnable unused groupby tests (GH#3480) Jörg Dietrich
Remove old parquet and bcolz dataframe optimizations (GH#5484) Matthew Rocklin
Add getitem optimization for
read_parquet
(GH#5453) Tom AugspurgerUse
_constructor_sliced
method to determine Series type (GH#5480) Richard J ZamoraFix map(series) for unsorted base series index (GH#5459) Justin Waugh
Fix
KeyError
with Groupby label (GH#5467) Ryan Nazareth
Documentation¶
Use Zoom meeting instead of appear.in (GH#5494) Matthew Rocklin
Update SSH docs to include
SSHCluster
(GH#5482) Matthew RocklinUpdate “Why Dask?” page (GH#5473) Matthew Rocklin
2.5.2 / 2019-10-04¶
Array¶
Correct chunk size logic for asymmetric overlaps (GH#5449) Ben Jeffery
Make da.unify_chunks public API (GH#5443) Matthew Rocklin
DataFrame¶
Fix dask.dataframe.fillna handling of Scalar object (GH#5463) Zhenqing Li
Documentation¶
Remove boxes in Spark comparison page (GH#5445) Matthew Rocklin
Update cloud documentation (GH#5444) Matthew Rocklin
2.5.0 / 2019-09-27¶
Core¶
Add sentinel no_default to get_dependencies task (GH#5420) James Bourbeau
Update fsspec version (GH#5415) Matthew Rocklin
DataFrame¶
Add option to not check meta in dd.from_delayed (GH#5436) Christopher J. Wright
Fix test_timeseries_nulls_in_schema failures with pyarrow master (GH#5421) Richard J Zamora
Reduce read_metadata output size in pyarrow/parquet (GH#5391) Richard J Zamora
Test numeric edge case for repartition with npartitions. (GH#5433) amerkel2
Unxfail pandas-datareader test (GH#5430) Tom Augspurger
Add DataFrame.pop implementation (GH#5422) Matthew Rocklin
Enable merge/set_index for cudf-based dataframes with cupy
values
(GH#5322) Richard J Zamoradrop_duplicates support for positional subset parameter (GH#5410) Wes Roach
Documentation¶
Add screencasts to array, bag, dataframe, delayed, futures and setup (GH#5429) (GH#5424) Matthew Rocklin
Fix delimeter parsing documentation (GH#5428) Mahmut Bulut
Update overview image (GH#5404) James Bourbeau
2.4.0 / 2019-09-13¶
Array¶
Adds explicit
h5py.File
mode (GH#5390) James BourbeauProvides method to compute unknown array chunks sizes (GH#5312) Scott Sievert
Ignore runtime warning in Array
compute_meta
(GH#5356) estebanagAdd
_meta
toArray.__dask_postpersist__
(GH#5353) Benoit BovyFixup
da.asarray
andda.asanyarray
for datetime64 dtype and xarray objects (GH#5334) Stephan HoyerAdd shape implementation (GH#5293) Tom Augspurger
Add chunktype to array text repr (GH#5289) James Bourbeau
Array.random.choice: handle array-like non-arrays (GH#5283) Gabe Joseph
Core¶
Fix
funcname
when vectorized func has no__name__
(GH#5399) James BourbeauTruncate
funcname
to avoid long key names (GH#5383) Matthew RocklinAdd support for
numpy.vectorize
infuncname
(GH#5396) James BourbeauFixed HDFS upstream test (GH#5395) Tom Augspurger
Support numbers and None in
parse_bytes
/timedelta
(GH#5384) Matthew RocklinFix tokenizing of subindexes on memmapped numpy arrays (GH#5351) Henry Pinkard
Upstream fixups (GH#5300) Tom Augspurger
DataFrame¶
Allow pandas to cast type of statistics (GH#5402) Richard J Zamora
Preserve index dtype after applying
dd.pivot_table
(GH#5385) therhaagImplement explode for Series and DataFrame (GH#5381) Arpit Solanki
set_index
on categorical fails with less categories than partitions (GH#5354) Oliver HofkensSupport output to a single CSV file (GH#5304) Hongjiu Zhang
Add
groupby().transform()
(GH#5327) Oliver HofkensAdding filter kwarg to pyarrow dataset call (GH#5348) Richard J Zamora
Implement and check compression defaults for parquet (GH#5335) Sarah Bird
Pass sqlalchemy params to delayed objects (GH#5332) Arpit Solanki
Fixing schema handling in arrow-parquet (GH#5307) Richard J Zamora
Add support for DF and Series
groupby().idxmin/max()
(GH#5273) Oliver HofkensAdd correlation calculation and add test (GH#5296) Benjamin Zaitlen
Documentation¶
Minor edits to Array chunk documentation (GH#5372) Scott Sievert
Add methods to API docs (GH#5387) Tom Augspurger
Add namespacing to configuration example (GH#5374) Matthew Rocklin
Add get_task_stream and profile to the diagnostics page (GH#5375) Matthew Rocklin
Add best practice to load data with Dask (GH#5369) Matthew Rocklin
Add threads and processes note to the best practices (GH#5340) Matthew Rocklin
Update cuDF links (GH#5328) James Bourbeau
Fixed small typo with parentheses placement (GH#5311) Eugene Huang
Update link in reshape docstring (GH#5297) James Bourbeau
2.3.0 / 2019-08-16¶
Array¶
Raise exception when
from_array
is given a dask array (GH#5280) David HoeseAvoid adjusting gufunc’s meta dtype twice (GH#5274) Peter Andreas Entschev
Add
meta=
keyword to map_blocks and add test with sparse (GH#5269) Matthew RocklinAdd rollaxis and moveaxis (GH#4822) Tobias de Jong
Always increment old chunk index (GH#5256) James Bourbeau
Shuffle dask array (GH#3901) Tom Augspurger
Fix ordering when indexing a dask array with a bool dask array (GH#5151) James Bourbeau
Bag¶
Add workaround for memory leaks in bag generators (GH#5208) Marco Neumann
Core¶
Set strict xfail option (GH#5220) James Bourbeau
test-upstream (GH#5267) Tom Augspurger
Fixed HDFS CI failure (GH#5234) Tom Augspurger
Ensure parquet tests are skipped if fastparquet and pyarrow not installed (GH#5217) James Bourbeau
Add fsspec to readthedocs (GH#5207) Matthew Rocklin
Bump NumPy and Pandas to 1.17 and 0.25 in CI test (GH#5179) John A Kirkham
DataFrame¶
Fix
DataFrame.query
docstring (incorrect numexpr API) (GH#5271) Doug DavisParquet metadata-handling improvements (GH#5218) Richard J Zamora
Improve messaging around sorted parquet columns for index (GH#5265) Martin Durant
Add
rearrange_by_divisions
andset_index
support for cudf (GH#5205) Richard J ZamoraFix
groupby.std()
with integer colum names (GH#5096) Nicolas HugGeneralize
hash_pandas_object
to work for non-pandas backends (GH#5184) GALI PREM SAGARAdd rolling cov (GH#5154) Ivars Geidans
Add columns argument in drop function (GH#5223) Henrique Ribeiro
Documentation¶
Update institutional FAQ doc (GH#5277) Matthew Rocklin
Add draft of institutional FAQ (GH#5214) Matthew Rocklin
Make boxes for dask-spark page (GH#5249) Martin Durant
Add motivation for shuffle docs (GH#5213) Matthew Rocklin
Fix links and API entries for best-practices (GH#5246) Martin Durant
Remove “bytes” (internal data ingestion) doc page (GH#5242) Martin Durant
Redirect from our local distributed page to distributed.dask.org (GH#5248) Matthew Rocklin
Cleanup API page (GH#5247) Matthew Rocklin
Remove excess endlines from install docs (GH#5243) Matthew Rocklin
Remove item list in phases of computation doc (GH#5245) Martin Durant
Remove custom graphs from the TOC sidebar (GH#5241) Matthew Rocklin
Remove experimental status of custom collections (GH#5236) James Bourbeau
Adds table of contents to Why Dask? (GH#5244) James Bourbeau
Moves bag overview to top-level bag page (GH#5240) James Bourbeau
Remove use-cases in favor of stories.dask.org (GH#5238) Matthew Rocklin
Removes redundant TOC information in index.rst (GH#5235) James Bourbeau
Elevate dashboard in distributed diagnostics documentation (GH#5239) Martin Durant
Updates “add” layer in HLG docs example (GH#5237) James Bourbeau
Update GUFunc documentation (GH#5232) Matthew Rocklin
2.2.0 / 2019-08-01¶
Array¶
Use da.from_array(…, asarray=False) if input follows NEP-18 (GH#5074) Matthew Rocklin
Add missing attributes to from_array documentation (GH#5108) Peter Andreas Entschev
Fix meta computation for some reduction functions (GH#5035) Peter Andreas Entschev
Raise informative error in to_zarr if unknown chunks (GH#5148) James Bourbeau
Remove invalid pad tests (GH#5122) Tom Augspurger
Ignore NumPy warnings in compute_meta (GH#5103) Peter Andreas Entschev
Fix kurtosis calc for single dimension input array (GH#5177) @andrethrill
Support Numpy 1.17 in tests (GH#5192) Matthew Rocklin
Bag¶
Supply pool to bag test to resolve intermittent failure (GH#5172) Tom Augspurger
Core¶
Base dask on fsspec (GH#5064) (GH#5121) Martin Durant
Various upstream compatibility fixes (GH#5056) Tom Augspurger
Make distributed tests optional again. (GH#5128) Elliott Sales de Andrade
Fix HDFS in dask (GH#5130) Martin Durant
Ignore some more invalid value warnings. (GH#5140) Elliott Sales de Andrade
DataFrame¶
Fix pd.MultiIndex size estimate (GH#5066) Brett Naul
Generalizing has_known_categories (GH#5090) GALI PREM SAGAR
Refactor Parquet engine (GH#4995) Richard J Zamora
Add divide method to series and dataframe (GH#5094) msbrown47
fix flaky partd test (GH#5111) Tom Augspurger
Adjust is_dataframe_like to adjust for value_counts change (GH#5143) Tom Augspurger
Generalize rolling windows to support non-Pandas dataframes (GH#5149) Nick Becker
Avoid unnecessary aggregation in pivot_table (GH#5173) Daniel Saxton
Add column names to apply_and_enforce error message (GH#5180) Matthew Rocklin
Add schema keyword argument to to_parquet (GH#5150) Sarah Bird
Allow fastparquet to handle gather_statistics=False for file lists (GH#5157) Richard J Zamora
Documentation¶
Adds NumFOCUS badge to the README (GH#5086) James Bourbeau
Document DataFrame.set_index computataion behavior Natalya Rapstine
Use pip install . instead of calling setup.py (GH#5139) Matthias Bussonier
Close user survey (GH#5147) Tom Augspurger
Fix Google Calendar meeting link (GH#5155) Loïc Estève
Add docker image customization example (GH#5171) James Bourbeau
Update remote-data-services after fsspec (GH#5170) Martin Durant
Fix typo in spark.rst (GH#5164) Xavier Holt
Update setup/python docs for async/await API (GH#5163) Matthew Rocklin
Update Local Storage HPC documentation (GH#5165) Matthew Rocklin