Changelog¶
2021.03.0¶
Released on March 5, 2021
Note
This is the first release with support for Python 3.9 and the last release with support for Python 3.6
- Bump minimum version of
distributed
(GH#7328) James Bourbeau - Fix
percentiles_summary
withdask_cudf
(GH#7325) Peter Andreas Entschev - Temporarily revert recent
Array.__setitem__
updates (GH#7326) James Bourbeau Blockwise.clone
(GH#7312) Guido Imperiale- NEP-35 duck array update (GH#7321) James Bourbeau
- Don’t allow setting
.name
for array (GH#7222) Julia Signell - Use nearest interpolation for creating percentiles of integer input (GH#7305) Kyle Barron
- Test
exp
with CuPy arrays (GH#7322) John A Kirkham - Check that computed chunks have right size and dtype (GH#7277) Bruce Merry
pytest.mark.flaky
(GH#7319) Guido Imperiale- Contributing docs: add note to pull the latest git tags before pip installing Dask (GH#7308) Genevieve Buckley
- Support for Python 3.9 (GH#7289) Guido Imperiale
- Add broadcast-based merge implementation (GH#7143) Richard (Rick) Zamora
- Add
split_every
tograph_manipulation
(GH#7282) Guido Imperiale - Typo in optimize docs (GH#7306) Julius Busecke
dask.graph_manipulation
support forxarray.Dataset
(GH#7276) Guido Imperiale- Add plot width and height support for Bokeh 2.3.0 (GH#7297) James Bourbeau
- Add NumPy functions
tri
,triu_indices
,triu_indices_from
,tril_indices
,tril_indices_from
(GH#6997) Illviljan - Remove “cleanup” task in DataFrame on-disk shuffle (GH#7260) Sinclair Target
- Use development version of
distributed
in CI (GH#7279) James Bourbeau - Moving high level graph pack/unpack Dask (GH#7179) Mads R. B. Kristensen
- Improve performance of
merge_percentiles
(GH#7172) Ashwin Srinath - DOC: add
dask-sql
andfugue
(GH#7129) Ray Bell - Example for working with categoricals and parquet (GH#7085) McToel
- Adds tree reduction to
bincount
(GH#7183) Thomas J. Fan - Improve documentation of
name
infrom_array
(GH#7264) Bruce Merry - Fix
cumsum
for empty partitions (GH#7230) Julia Signell - Add
map_blocks
example to dask array creation docs (GH#7221) Julia Signell - Fix performance issue in
dask.graph_manipulation.wait_on()
(GH#7258) Guido Imperiale - Replace coveralls with codecov.io (GH#7246) Guido Imperiale
- Pin to a particular
black
rev in pre-commit (GH#7256) Julia Signell - Minor typo in documentation:
array-chunks.rst
(GH#7254) Magnus Nord - Fix bugs in
Blockwise
andShuffleLayer
(GH#7213) Richard (Rick) Zamora - Fix parquet filtering bug for
"pyarrow-dataset"
with pyarrow-3.0.0 (GH#7200) Richard (Rick) Zamora graph_manipulation
without NumPy (GH#7243) Guido Imperiale- Support for NEP-35 (GH#6738) Peter Andreas Entschev
- Avoid running unit tests during doctest CI build (GH#7240) James Bourbeau
- Run doctests on CI (GH#7238) Julia Signell
- Cleanup code quality on set arithmetics (GH#7196) Guido Imperiale
- Add
dask.array.delete
(GH#7125) Julia Signell - Unpin graphviz now that new conda-forge recipe is built (GH#7235) Julia Signell
- Don’t use NumPy 1.20 from conda-forge on Mac (GH#7211) Guido Imperiale
map_overlap
: Don’t rechunk axes without overlap (GH#7233) Deepak Cherian- Pin graphviz to avoid issue with latest conda-forge build (GH#7232) Julia Signell
- Use
html_css_files
in docs for custom CSS (GH#7220) James Bourbeau - Graph manipulation:
clone
,bind
,checkpoint
,wait_on
(GH#7109) Guido Imperiale - Fix handling of filter expressions in parquet
pyarrow-dataset
engine (GH#7186) Joris Van den Bossche - Extend
__setitem__
to more closely match numpy (GH#7033) David Hassell - Clean up Python 2 syntax (GH#7195) Guido Imperiale
- Fix regression in
Delayed._length
(GH#7194) Guido Imperiale __dask_layers__()
tests and tweaks (GH#7177) Guido Imperiale- Properly convert
HighLevelGraph
in multiprocessing scheduler (GH#7191) Jim Crist-Harif - Don’t fail fast in CI (GH#7188) James Bourbeau
2021.02.0¶
Released on February 5, 2021
- Add
percentile
support for NEP-35 (GH#7162) Peter Andreas Entschev - Added support for
Float64
in column assignment (GH#7173) Nils Braun - Coarsen rechunking error (GH#7127) Davis Bennett
- Fix upstream CI tests (GH#6896) Julia Signell
- Revise
HighLevelGraph
Mapping API (GH#7160) Guido Imperiale - Update low-level graph spec to use any hashable for keys (GH#7163) James Bourbeau
- Generically rebuild a collection with different keys (GH#7142) Guido Imperiale
- Make easier to link issues in PRs (GH#7130) Ray Bell
- Add
dask.array.append
(GH#7146) D-Stacks - Allow
dask.array.ravel
to acceptarray_like
argument (GH#7138) D-Stacks - Fixes link in array design doc (GH#7152) Thomas J. Fan
- Fix example of using
blockwise
for an outer product (GH#7119) Bruce Merry - Deprecate
HighlevelGraph.dicts
in favor of.layers
(GH#7145) Amit Kumar - Align
FastParquetEngine
with pyarrow engines (GH#7091) Richard (Rick) Zamora - Merge annotations (GH#7102) Ian Rose
- Simplify contents of parts list in
read_parquet
(GH#7066) Richard (Rick) Zamora check_meta(
): use__class__
when checking DataFrame types (GH#7099) Mads R. B. Kristensen- Cache several properties (GH#7104) Illviljan
- Fix parquet
getitem
optimization (GH#7106) Richard (Rick) Zamora - Add cytoolz back to CI environment (GH#7103) James Bourbeau
2021.01.1¶
Released on January 22, 2021
- Partially fix
cumprod
(GH#7089) Julia Signell - Test pandas 1.1.x / 1.2.0 releases and pandas nightly (GH#6996) Joris Van den Bossche
- Use assign to avoid
SettingWithCopyWarning
(GH#7092) Julia Signell 'mode'
argument passed tobokeh.output_file()
(GH#7034) (GH#7075) patquem- Skip empty partitions when doing
groupby.value_counts
(GH#7073) Julia Signell - Add error messages to
assert_eq()
(GH#7083) James Lamb - Make cached properties read-only (GH#7077) Illviljan
2021.01.0¶
Released on January 15, 2021
map_partitions
with review comments (GH#6776) Kumar Bharath Prabhu- Make sure that
population
is a real list (GH#7027) Julia Signell - Propagate
storage_options
inread_csv
(GH#7074) Richard (Rick) Zamora - Remove all
BlockwiseIO
code (GH#7067) Richard (Rick) Zamora - Fix CI (GH#7069) James Bourbeau
- Add option to control rechunking in
reshape
(GH#6753) Tom Augspurger - Fix
linalg.lstsq
for complex inputs (GH#7056) Johnnie Gray - Add
compression='infer'
default toread_csv
(GH#6960) Richard (Rick) Zamora - Revert parameter changes in
svd_compressed
#7003 (GH#7004) Eric Czech - Skip failing s3 test (GH#7064) Martin Durant
- Revert
BlockwiseIO
(GH#7048) Richard (Rick) Zamora - Add some cross-references to
DataFrame.to_bag()
andSeries.to_bag()
(GH#7049) Rob Malouf - Rewrite
matmul
asblockwise
without contraction/concatenate (GH#7000) Rafal Wojdyla - Use
functools.cached_property
inda.shape
(GH#7023) Illviljan - Use meta value in series
non_empty
(GH#6976) Julia Signell - Revert “Temporarly pin sphinx version to 3.3.1 (GH#7002)” (GH#7014) Rafal Wojdyla
- Revert
python-graphviz
pinning (GH#7037) Julia Signell - Accidentally committed print statement (GH#7038) Julia Signell
- Pass
dropna
andobserved
inagg
(GH#6992) Julia Signell - Add index to
meta
after.str.split
with expand (GH#7026) Ruben van de Geer - CI: test pyarrow 2.0 and nightly (GH#7030) Joris Van den Bossche
- Temporarily pin
python-graphviz
in CI (GH#7031) James Bourbeau - Underline section in
numpydoc
(GH#7013) Matthias Bussonnier - Keep normal optimizations when adding custom optimizations (GH#7016) Matthew Rocklin
- Temporarily pin sphinx version to 3.3.1 (GH#7002) Rafal Wojdyla
- DOC: Misc formatting (GH#6998) Matthias Bussonnier
- Add
inline_array
option tofrom_array
(GH#6773) Tom Augspurger - Revert “Initial pass at blockwise array creation routines (GH#6931)” (:pr:`6995) James Bourbeau
- Set
npartitions
inset_index
(GH#6978) Julia Signell - Upstream
config
serialization and inheritance (GH#6987) Jacob Tomlinson - Bump the minimum time in
test_minimum_time
(GH#6988) Martin Durant - Fix pandas
dtype
inference forread_parquet
(GH#6985) Richard (Rick) Zamora - Avoid data loss in
set_index
withsorted=True
(GH#6980) Richard (Rick) Zamora - Bugfix in
read_parquet
for handling un-named indices withindex=False
(GH#6969) Richard (Rick) Zamora - Use
__class__
when comparing meta data (GH#6981) Mads R. B. Kristensen - Comparing string versions won’t always work (GH#6979) Rafal Wojdyla
- Fix GH#6925 (GH#6982) sdementen
- Initial pass at blockwise array creation routines (GH#6931) Ian Rose
- Simplify
has_parallel_type()
(GH#6927) Mads R. B. Kristensen - Handle annotation unpacking in
BlockwiseIO
(GH#6934) Simon Perkins - Avoid deprecated
yield_fixture
intest_sql.py
(GH#6968) Richard (Rick) Zamora - Remove bad graph logic in
BlockwiseIO
(GH#6933) Richard (Rick) Zamora - Get config item if variable is
None
(GH#6862) Jacob Tomlinson - Update
from_pandas
docstring (GH#6957) Richard (Rick) Zamora - Prevent
fuse_roots
from clobbering annotations (GH#6955) Simon Perkins
2020.12.0¶
Released on December 10, 2020
Highlights¶
- Switched to CalVer for versioning scheme.
- Introduced new APIs for
HighLevelGraph
to enable sending high-level representations of task graphs to the distributed scheduler. - Introduced new
HighLevelGraph
layer objects includingBasicLayer
,Blockwise
,BlockwiseIO
,ShuffleLayer
, and more. - Added support for applying custom
Layer
-level annotations likepriority
,retries
, etc. with thedask.annotations
context manager. - Updated minimum supported version of pandas to 0.25.0 and NumPy to 1.15.1.
- Support for the
pyarrow.dataset
API toread_parquet
. - Several fixes to Dask Array’s SVD.
All changes¶
- Make
observed
kwarg optional (GH#6952) Julia Signell - Min supported pandas 0.25.0 numpy 1.15.1 (GH#6895) Julia Signell
- Make order of categoricals unambiguous (GH#6949) Julia Signell
- Improve “pyarrow-dataset” statistics performance for
read_parquet
(GH#6918) Richard (Rick) Zamora - Add
observed
keyword togroupby
(GH#6854) Julia Signell - Make sure
include_path_column
works when there are multiple partitions per file (GH#6911) Julia Signell - Fix:
array.overlap
andarray.map_overlap
block sizes are incorrect when depth is an unsigned bit type (GH#6909) GFleishman - Fix syntax error in HLG docs example (GH#6946) Mark
- Return a
Bag
fromsample
(GH#6941) Shang Wang - Add
ravel_multi_index
(GH#6939) Illviljan - Enable parquet metadata collection in parallel (GH#6921) Richard (Rick) Zamora
- Avoid using
_file
inprogressbar
if it isNone
(GH#6938) Mark Harfouche - Add Zarr to upstream CI build (GH#6932) James Bourbeau
- Introduce
BlockwiseIO
layer (GH#6878) Richard (Rick) Zamora - Transmit
Layer
Annotations to Scheduler (GH#6889) Simon Perkins - Update opportunistic caching page to remove experimental warning (GH#6926) Timost
- Allow
pyarrow >2.0.0
(GH#6772) Richard (Rick) Zamora - Support
pyarrow.dataset
API forread_parquet
(GH#6534) Richard (Rick) Zamora - Add more informative error message to
da.coarsen
when coarsening factors do not divide shape (GH#6908) Davis Bennett - Only run the cron CI on
dask/dask
not forks (GH#6905) Jacob Tomlinson - Add
annotations
toShuffleLayers
(GH#6913) Matthew Rocklin - Temporarily xfail
test_from_s3
(GH#6915) James Bourbeau - Added dataframe
skew
method (GH#6881) Jan Borchmann - Fix
dtype
in arraymeta
(GH#6893) Julia Signell - Missing
name
arg inhelm install ...
(GH#6903) Ruben van de Geer - Fix: exception when reading an item with filters (GH#6901) Martin Durant
- Add support for
cupyx
sparse todask.array.dot
(GH#6846) Akira Naruse - Pin array mindeps up a bit to get the tests to pass [test-mindeps] (GH#6894) Julia Signell
- Update/remove pandas and numpy in mindeps (GH#6888) Julia Signell
- Fix
ArrowEngine
bug in use ofclear_known_categories
(GH#6887) Richard (Rick) Zamora - Fix documentation about task scheduler (GH#6879) Zhengnan Zhao
- Add human relative time formatting utility (GH#6883) Jacob Tomlinson
- Possible fix for 6864
set_index
issue (GH#6866) Richard (Rick) Zamora BasicLayer
: remove dependency arguments (GH#6859) Mads R. B. Kristensen- Serialization of
Blockwise
(GH#6848) Mads R. B. Kristensen - Address
columns=[]
bug (GH#6871) Richard (Rick) Zamora - Avoid duplicate parquet schema communication (GH#6841) Richard (Rick) Zamora
- Add
create_metadata_file
utility for existing parquet datasets (GH#6851) Richard (Rick) Zamora - Improve ordering for workloads with a common terminus (GH#6779) Tom Augspurger
- Stringify utilities (GH#6852) Mads R. B. Kristensen
- Add keyword
overwrite=True
toto_parquet
to remove dangling files when overwriting a pyarrowDataset
. (GH#6825) Greg Hayes - Removed
map_tasks()
andmap_basic_layers()
(GH#6853) Mads R. B. Kristensen - Introduce QR iteration to
svd_compressed
(GH#6813) RogerMoens __dask_distributed_pack__()
now takes aclient
argument (GH#6850) Mads R. B. Kristensen- Use
map_partitions
instead ofdelayed
inset_index
(GH#6837) Mads R. B. Kristensen - Add doc hit for
as_completed().update(futures)
(GH#6817) manuels - Bump GHA
setup-miniconda
version (GH#6847) Jacob Tomlinson - Remove nans when setting sorted index (GH#6829) Rockwell Weiner
- Fix transpose of u in SVD (GH#6799) RogerMoens
- Migrate to GitHub Actions (GH#6794) Jacob Tomlinson
- Fix sphinx
currentmodule
usage (GH#6839) James Bourbeau - Fix minimum dependencies CI builds (GH#6838) James Bourbeau
- Avoid graph materialization during
Blockwise
culling (GH#6815) Richard (Rick) Zamora - Fixed typo (GH#6834) Devanshu Desai
- Use
HighLevelGraph.merge
incollections_to_dsk
(GH#6836) Mads R. B. Kristensen - Respect
dtype
in svdcompression_matrix
#2849 (GH#6802) RogerMoens - Add blocksize to task name (GH#6818) Julia Signell
- Check for all-NaN partitions (GH#6821) Rockwell Weiner
- Change “institutional” SQL doc section to point to main SQL doc (GH#6823) Martin Durant
- Fix:
DataFrame.join
doesn’t accept Series as other (GH#6809) David Katz - Remove
to_delayed
operations fromto_parquet
(GH#6801) Richard (Rick) Zamora - Layer annotation docstrings improvements (GH#6806) Simon Perkins
- Avro reader (GH#6780) Martin Durant
- Rechunk array if smallest chunk size is smaller than depth (GH#6708) Julia Signell
- Add Layer Annotations (GH#6767) Simon Perkins
- Add “view code” links to documentation (GH#6793) manuels
- Add optional IO-subgraph to
Blockwise
Layers (GH#6715) Richard (Rick) Zamora - Add high level graph pack/unpack for distributed (GH#6786) Mads R. B. Kristensen
- Add missing methods of the Dataframe API (GH#6789) Stephannie Jimenez Gacha
- Add doc on managing environments (GH#6778) Martin Durant
- HLG:
get_all_external_keys()
(GH#6774) Mads R. B. Kristensen - Avoid rechunking in reshape with
chunksize=1
(GH#6748) Tom Augspurger - Try to make categoricals work on join (GH#6205) Julia Signell
- Fix some minor typos and trailing whitespaces in
array-slice.rst
(GH#6771) Magnus Nord - Bugfix for parquet metadata writes of empty dataframe partitions (pyarrow) (GH#6741) Callum Noble
- Document
meta
kwarg inmap_blocks
andmap_overlap
. (GH#6763) Peter Andreas Entschev - Begin experimenting with parallel prefix scan for
cumsum
andcumprod
(GH#6675) Erik Welch - Clarify differences in boolean indexing between dask and numpy arrays (GH#6764) Illviljan
- Efficient serialization of shuffle layers (GH#6760) James Bourbeau
- Config array optimize to skip fusion and return a HLG (GH#6751) Mads R. B. Kristensen
- Temporarily use
pyarrow<2
in CI (GH#6759) James Bourbeau - Fix meta for
min
/max
reductions (GH#6736) Peter Andreas Entschev - Add 2D possibility to
da.linalg.lstsq
- mirroring numpy (GH#6749) Pascal Bourgault - CI: Fixed bug causing flaky test failure in pivot (GH#6752) Tom Augspurger
- Serialization of layers (GH#6693) Mads R. B. Kristensen
- Add
attrs
property to Series/Dataframe (GH#6742) Illviljan - Removed Mutable Default Argument (GH#6747) Mads R. B. Kristensen
- Adjust parquet
ArrowEngine
to allow more easy subclass for writing (GH#6505) Joris Van den Bossche - Add
ShuffleStage
HLG Layer (GH#6650) Richard (Rick) Zamora - Handle literal in
meta_from_array
(GH#6731) Peter Andreas Entschev - Do balanced rechunking even if chunks are the same (GH#6735) Chris Roat
- Fix docstring
DataFrame.set_index
(GH#6739) Gil Forsyth - Ensure
HighLevelGraph
layers always containLayer
instances (GH#6716) James Bourbeau - Map on
HighLevelGraph
Layers (GH#6689) Mads R. B. Kristensen - Update overlap
*_like
function calls and CuPy tests (GH#6728) Peter Andreas Entschev - Fixes for
svd
with__array_function__
(GH#6727) Peter Andreas Entschev - Added doctest extension for documentation (GH#6397) Jim Circadian
- Minor fix to #5628 using @pentschev’s suggestion (GH#6724) John A Kirkham
- Change type of Dask array when meta type changes (GH#5628) Matthew Rocklin
- Add
az
(GH#6719) Ray Bell - HLG:
get_dependencies()
of single keys (GH#6699) Mads R. B. Kristensen - Revert “Revert “Use HighLevelGraph layers everywhere in collections (GH#6510)” (GH#6697)” (GH#6707) Tom Augspurger
- Allow
*_like
array creation functions to respect input array type (GH#6680) Genevieve Buckley - Update
dask-sphinx-theme
version (GH#6700) Gil Forsyth
2.30.0 / 2020-10-06¶
- Allow
rechunk
to evenly split into N chunks (GH#6420) Scott Sievert
2.29.0 / 2020-10-02¶
_repr_html_
: color sides darker instead of drawing all the lines (GH#6683) Julia Signell- Removes warning from
nanstd
andnanvar
(GH#6667) Thomas J. Fan - Get shape of output from original array -
map_overlap
(GH#6682) Julia Signell - Replace
np.searchsorted
withbisect
in indexing (GH#6669) Joachim B Haga
- Make sure subprocesses have a consistent hash for bag
groupby
(GH#6660) Itamar Turner-Trauring
- Revert “Use
HighLevelGraph
layers everywhere in collections (GH#6510)” (GH#6697) Tom Augspurger - Use
pandas.testing
(GH#6687) John A Kirkham - Improve 128-bit floating-point skip in tests (GH#6676) Elliott Sales de Andrade
- Allow setting dataframe items using a bool dataframe (GH#6608) Julia Signell
2.28.0 / 2020-09-25¶
- Partially reverted changes to
Array
indexing that produces large changes. This restores the behavior from Dask 2.25.0 and earlier, with a warning when large chunks are produced. A configuration option is provided to avoid creating the large chunks, see Efficiency. (GH#6665) Tom Augspurger - Add
meta
toto_dask_array
(GH#6651) Kyle Nicholson - Fix GH#6631 and GH#6611 (GH#6632) Rafal Wojdyla
- Infer object in array reductions (GH#6629) Daniel Saxton
- Adding
v_based
flag forsvd_flip
(GH#6658) Eric Czech - Fix flakey array
mean
(GH#6656) Sam Grayson
- Removed
dsk
equality check fromSubgraphCallable.__eq__
(GH#6666) Mads R. B. Kristensen - Use
HighLevelGraph
layers everywhere in collections (GH#6510) Mads R. B. Kristensen - Adds hash dunder method to
SubgraphCallable
for caching purposes (GH#6424) Andrew Fulton - Stop writing commented out config files by default (GH#6647) Matthew Rocklin
- Add support for collect list aggregation via
agg
API (GH#6655) Madhur Tandon - Slightly better error message (GH#6657) Julia Signell
2.27.0 / 2020-09-18¶
- Preserve
dtype
insvd
(GH#6643) Eric Czech
store()
: create a single HLG layer (GH#6601) Mads R. B. Kristensen- Add pre-commit CI build (GH#6645) James Bourbeau
- Update
.pre-commit-config
to latest black. (GH#6641) Julia Signell - Update super usage to remove Python 2 compatibility (GH#6630) Poruri Sai Rahul
- Remove u string prefixes (GH#6633) Poruri Sai Rahul
- Improve error message for
to_sql
(GH#6638) Julia Signell - Use empty list as categories (GH#6626) Julia Signell
- Add
autofunction
to array api docs for more ufuncs (GH#6644) James Bourbeau - Add a number of missing ufuncs to
dask.array
docs (GH#6642) Ralf Gommers - Add
HelmCluster
docs (GH#6290) Jacob Tomlinson
2.26.0 / 2020-09-11¶
- Backend-aware dtype inference for single-chunk svd (GH#6623) Eric Czech
- Make
array.reduction
docstring match for dtype (GH#6624) Martin Durant - Set lower bound on compression level for
svd_compressed
using rows and cols (GH#6622) Eric Czech - Improve SVD consistency and small array handling (GH#6616) Eric Czech
- Add
svd_flip
#6599 (GH#6613) Eric Czech - Handle sequences containing dask Arrays (GH#6595) Gabe Joseph
- Avoid large chunks from
getitem
with lists (GH#6514) Tom Augspurger - Eagerly slice numpy arrays in
from_array
(GH#6605) Deepak Cherian - Restore ability to pickle dask arrays (GH#6594) Noah D Brenowitz
- Add SVD support for short-and-fat arrays (GH#6591) Eric Czech
- Add simple chunk type registry and defer as appropriate to upcast types (GH#6393) Jon Thielen
- Align coarsen chunks by default (GH#6580) Deepak Cherian
- Fixup reshape on unknown dimensions and other testing fixes (GH#6578) Ryan Williams
- Add validation and fixes for
HighLevelGraph
dependencies (GH#6588) Mads R. B. Kristensen - Fix linting issue (GH#6598) Tom Augspurger
- Skip
bokeh
version 2.0.0 (GH#6572) John A Kirkham
- Added bytes/row calculation when using meta (GH#6585) McToel
- Handle
min_count
inSeries.sum
/prod
(GH#6618) Daniel Saxton - Update
DataFrame.set_index
docstring (GH#6549) Timost - Always compute 0 and 1 quantiles during quantile calculations (GH#6564) Erik Welch
- Fix wrong path when reading empty csv file (GH#6573) Abdulelah Bin Mahfoodh
- Doc: Troubleshooting dashboard 404 (GH#6215) Kilian Lieret
- Fixup
extraConfig
example (GH#6625) Tom Augspurger - Update supported Python versions (GH#6609) Julia Signell
- Document dask/daskhub helm chart (GH#6560) Tom Augspurger
2.25.0 / 2020-08-28¶
- Compare key hashes in
subs()
(GH#6559) Mads R. B. Kristensen - Rerun with latest
black
release (GH#6568) James Bourbeau - License update (GH#6554) Tom Augspurger
- Remove version from documentation page names (GH#6558) James Bourbeau
- Update
kubernetes-helm.rst
(GH#6523) David Sheldon - Stop 2020 survey (GH#6547) Tom Augspurger
2.24.0 / 2020-08-22¶
- Fix setting random seed in tests. (GH#6518) Elliott Sales de Andrade
- Support meta in apply gufunc (GH#6521) joshreback
- Replace cupy.sparse with cupyx.scipy.sparse (GH#6530) John A Kirkham
- Bump up tolerance for rolling tests (GH#6502) Julia Signell
- Implement DatFrame.__len__ (GH#6515) Tom Augspurger
- Infer arrow schema in to_parquet (for ArrowEngine`) (GH#6490) Richard (Rick) Zamora
- Fix parquet test when no pyarrow (GH#6524) Martin Durant
- Remove problematic
filter
arguments in ArrowEngine (GH#6527) Richard (Rick) Zamora - Avoid schema validation by default in ArrowEngine (GH#6536) Richard (Rick) Zamora
- Use unpack_collections in make_blockwise_graph (GH#6517) Thomas J. Fan
- Move key_split() from optimization.py to utils.py (GH#6529) Mads R. B. Kristensen
- Make tests run on moto server (GH#6528) Martin Durant
2.23.0 / 2020-08-14¶
- Reduce
np.zeros
,ones
, andfull
array size with broadcasting (GH#6491) Matthias Bussonnier - Add missing
meta=
fortrim
inmap_overlap
(GH#6494) Peter Andreas Entschev
- Bag repartition partition size (GH#6371) joshreback
Scalar.__dask_layers__()
to returnself._name
instead ofself.key
(GH#6507) Mads R. B. Kristensen- Update dependencies correctly in
fuse_root
optimization (GH#6508) Mads R. B. Kristensen
- Adds
items
to dataframe (GH#6503) Thomas J. Fan - Include compression in
write_table
call (GH#6499) Julia Signell - Fixed warning in
nonempty_series
(GH#6485) Tom Augspurger - Intelligently determine partitions based on type of first arg (GH#6479) Matthew Rocklin
- Fix pyarrow
mkdirs
(GH#6475) Julia Signell - Fix duplicate parquet output in
to_parquet
(GH#6451) michaelnarodovitch
- Fix documentation
da.histogram
(GH#6439) Roberto Panai - Add
agg
nunique
example (GH#6404) Ray Bell - Fixed a few typos in the SQL docs (GH#6489) Mike McCarty
- Docs for SQLing (GH#6453) Martin Durant
2.22.0 / 2020-07-31¶
- Compatibility for NumPy dtype deprecation (GH#6430) Tom Augspurger
- Implement
sizeof
for somebytes
-like objects (GH#6457) John A Kirkham - HTTP error for new
fsspec
(GH#6446) Martin Durant - When
RecursionError
is raised, return uuid fromtokenize
function (GH#6437) Julia Signell - Install deps of upstream-dev packages (GH#6431) Tom Augspurger
- Use updated link in
setup.cfg
(GH#6426) Zhengnan
- Add single quotes around column names if strings (GH#6471) Gil Forsyth
- Refactor
ArrowEngine
for betterread_parquet
performance (GH#6346) Richard (Rick) Zamora - Add
tolist
dispatch (GH#6444) GALI PREM SAGAR - Compatibility with pandas 1.1.0rc0 (GH#6429) Tom Augspurger
- Multi value pivot table (GH#6428) joshreback
- Duplicate argument definitions in
to_csv
docstring (GH#6411) Jun Han (Johnson) Ooi
- Add utility to docs to convert YAML config to env vars and back (GH#6472) Jacob Tomlinson
- Fix parameter server rendering (GH#6466) Scott Sievert
- Fixes broken links (GH#6403) Jim Circadian
- Complete parameter server implementation in docs (GH#6449) Scott Sievert
- Fix typo (GH#6436) Jack Xiaosong Xu
2.21.0 / 2020-07-17¶
- Correct error message in
array.routines.gradient()
(GH#6417) johnomotani - Fix blockwise concatenate for array with some
dimension=1
(GH#6342) Matthias Bussonnier
- Fix
bag.take
example (GH#6418) Roberto Panai
- Groups values in optimization pass should only be graph and keys – not an optimization + keys (GH#6409) Ben Zaitlen
- Call custom optimizations once, with
kwargs
provided (GH#6382) Clark Zinzow - Include
pickle5
for testing on Python 3.7 (GH#6379) John A Kirkham
- Correct typo in error message (GH#6422) Tom McTiernan
- Use
pytest.warns
to check forUserWarning
(GH#6378) Richard (Rick) Zamora - Parse
bytes_per_chunk keyword
from string (GH#6370) Matthew Rocklin
- Numpydoc formatting (GH#6421) Matthias Bussonnier
- Unpin
numpydoc
following 1.1 release (GH#6407) Gil Forsyth - Numpydoc formatting (GH#6402) Matthias Bussonnier
- Add instructions for using conda when installing code for development (GH#6399) Ray Bell
- Update
visualize
docstrings (GH#6383) Zhengnan
2.20.0 / 2020-07-02¶
- Register
sizeof
for numpy zero-strided arrays (GH#6343) Matthias Bussonnier - Use
concatenate_lookup
inconcatenate
(GH#6339) John A Kirkham - Fix rechunking of arrays with some zero-length dimensions (GH#6335) Matthias Bussonnier
- Dispatch
iloc`
calls togetitem
(GH#6355) Gil Forsyth - Handle unnamed pandas
RangeIndex
in fastparquet engine (GH#6350) Richard (Rick) Zamora - Preserve index when writing partitioned parquet datasets with pyarrow (GH#6282) Richard (Rick) Zamora
- Use
ignore_index
for pandas’group_split_dispatch
(GH#6251) Richard (Rick) Zamora
2.19.0 / 2020-06-19¶
- Cast chunk sizes to python int
dtype
(GH#6326) Gil Forsyth - Add
shape=None
to*_like()
array creation functions (GH#6064) Anderson Banihirwe
- Update expected error msg for protocol difference in fsspec (GH#6331) Gil Forsyth
- Fix for floats < 1 in
parse_bytes
(GH#6311) Gil Forsyth - Fix exception causes all over the codebase (GH#6308) Ram Rachum
- Fix duplicated tests (GH#6303) James Lamb
- Remove unused testing function (GH#6304) James Lamb
- Add high-level CSV Subgraph (GH#6262) Gil Forsyth
- Fix
ValueError
when merging an index-only 1-partition dataframe (GH#6309) Krishan Bhasin - Make
index.map
clear divisions. (GH#6285) Julia Signell
- Add link to 2020 survey (GH#6328) Tom Augspurger
- Update
bag.rst
(GH#6317) Ben Shaver
2.18.1 / 2020-06-09¶
- Don’t try to set name on
full
(GH#6299) Julia Signell - Histogram: support lazy values for range/bins (another way) (GH#6252) Gabe Joseph
- Fix exception causes in
utils.py
(GH#6302) Ram Rachum - Improve performance of
HighLevelGraph
construction (GH#6293) Julia Signell
- Now readthedocs builds unrelased features’ docstrings (GH#6295) Antonio Ercole De Luca
- Add
asyncssh
intersphinx mappings (GH#6298) Jacob Tomlinson
2.18.0 / 2020-06-05¶
- Cast slicing index to dask array if same shape as original (GH#6273) Julia Signell
- Fix
stack
error message (GH#6268) Stephanie Gott full
&full_like
: error on non-scalarfill_value
(GH#6129) Huite- Support for multiple arrays in
map_overlap
(GH#6165) Eric Czech - Pad resample divisions so that edges are counted (GH#6255) Julia Signell
- Random sampling of k elements from a dask bag #4799 (GH#6239) Antonio Ercole De Luca
- Add
dropna
,sort
, andascending
tosort_values
(GH#5880) Julia Signell - Generalize
from_dask_array
(GH#6263) GALI PREM SAGAR - Add derived docstring for
SeriesGroupby.nunique
(GH#6284) Julia Signell - Remove
NotImplementedError
in resample with rule (GH#6274) Abdulelah Bin Mahfoodh - Add
dd.to_sql
(GH#6038) Ryan Williams
2.17.2 / 2020-05-28¶
- Re-add the
complete
extra (GH#6257) Jim Crist-Harif
- Raise error if
resample
isn’t going to give right answer (GH#6244) Julia Signell
2.17.1 / 2020-05-28¶
- Empty array rechunk (GH#6233) Andrew Fulton
- Make
pyyaml
required (GH#6250) Jim Crist-Harif - Fix install commands from
ImportError
(GH#6238) Gaurav Sheni - Remove issue template (GH#6249) Jacob Tomlinson
- Pass
ignore_index
todd_shuffle
fromDataFrame.shuffle
(GH#6247) Richard (Rick) Zamora - Cope with missing HDF keys (GH#6204) Martin Durant
- Generalize
describe
&quantile
apis (GH#5137) GALI PREM SAGAR
2.17.0 / 2020-05-26¶
- Small improvements to
da.pad
(GH#6213) Mark Boer - Return
tuple
if multiple outputs indask.array.apply_gufunc
, add test to check for tuple (GH#6207) Kai Mühlbauer - Support
stack
with unknown chunksizes (GH#6195) swapna
- Random Choice on Bags (GH#6208) Antonio Ercole De Luca
- Raise warning
delayed.visualise()
(GH#6216) Amol Umbarkar - Ensure other pickle arguments work (GH#6229) John A Kirkham
- Overhaul
fuse()
config (GH#6198) Guido Imperiale - Update
dask.order.order
to consider “next” nodes using both FIFO and LIFO (GH#5872) Erik Welch
- Use 0 as
fill_value
for more agg methods (GH#6245) Julia Signell - Generalize
rearrange_by_column_tasks
and addDataFrame.shuffle
(GH#6066) Richard (Rick) Zamora - Xfail
test_rolling_numba_engine
for newer numba and older pandas (GH#6236) James Bourbeau - Generalize
fix_overlap
(GH#6240) GALI PREM SAGAR - Fix
DataFrame.shape
with no columns (GH#6237) noreentry - Avoid shuffle when setting a presorted index with overlapping divisions (GH#6226) Krishan Bhasin
- Adjust the Parquet engine classes to allow more easily subclassing (GH#6211) Marius van Niekerk
- Fix
dd.merge_asof
withleft_on='col'
&right_index=True
(GH#6192) noreentry - Disable warning for
concat
(GH#6210) Tung Dang - Move
AUTO_BLOCKSIZE
out ofread_csv
signature (GH#6214) Jim Crist-Harif .loc
indexing with callable (GH#6185) Endre Mark Borza- Avoid apply in
_compute_sum_of_squares
for groupby std agg (GH#6186) Richard (Rick) Zamora - Minor correction to
test_parquet
(GH#6190) Brian Larsen - Adhering to the passed pat for delimeter join and fix error message (GH#6194) GALI PREM SAGAR
- Skip
test_to_parquet_with_get
if no parquet libs available (GH#6188) Scott Sanderson
- Added documentation for
distributed.Event
class (GH#6231) Nils Braun - Doc write to remote (GH#6124) Ray Bell
2.16.0 / 2020-05-08¶
- Fix array general-reduction name (GH#6176) Nick Evans
- Replace
dim
withshape
inunravel_index
(GH#6155) Julia Signell - Moment: handle all elements being masked (GH#5339) Gabe Joseph
- Remove Redundant string concatenations in dask code-base (GH#6137) GALI PREM SAGAR
- Upstream compat (GH#6159) Tom Augspurger
- Ensure
sizeof
of dict and sequences returns an integer (GH#6179) James Bourbeau - Estimate python collection sizes with random sampling (GH#6154) Florian Jetter
- Update test upstream (GH#6146) Tom Augspurger
- Skip test for mindeps build (GH#6144) Tom Augspurger
- Switch default multiprocessing context to “spawn” (GH#4003) Itamar Turner-Trauring
- Update manifest to include dask-schema (GH#6140) Ben Zaitlen
- Harden inconsistent-schema handling in pyarrow-based
read_parquet
(GH#6160) Richard (Rick) Zamora - Add compute
kwargs
to methods that write data to disk (GH#6056) Krishan Bhasin - Fix issue where
unique
returns an index like result from backends (GH#6153) GALI PREM SAGAR - Fix internal error in
map_partitions
with collections (GH#6103) Tom Augspurger
- Add phase of computation to index TOC (GH#6157) Ben Zaitlen
- Remove unused imports in scheduling script (GH#6138) James Lamb
- Fix indent (GH#6147) Martin Durant
- Add Tom’s log config example (GH#6143) Martin Durant
2.15.0 / 2020-04-24¶
- Update
dask.array.from_array
to warn when passed a Dask collection (GH#6122) James Bourbeau - Un-numpy like behaviour in
dask.array.pad
(GH#6042) Mark Boer - Add support for
repeats=0
inda.repeat
(GH#6080) James Bourbeau
- Fix yaml layout for schema (GH#6132) Ben Zaitlen
- Configuration Reference (GH#6069) Ben Zaitlen
- Add configuration option to turn off task fusion (GH#6087) Matthew Rocklin
- Skip pyarrow on windows (GH#6094) Tom Augspurger
- Set limit to maximum length of fused key (GH#6057) Lucas Rademaker
- Add test against #6062 (GH#6072) Martin Durant
- Bump checkout action to v2 (GH#6065) James Bourbeau
- Generalize categorical calls to support cudf
Categorical
(GH#6113) GALI PREM SAGAR - Avoid reading
_metadata
on every worker (GH#6017) Richard (Rick) Zamora - Use
group_split_dispatch
andignore_index
inapply_concat_apply
(GH#6119) Richard (Rick) Zamora - Handle new (dtype) pandas metadata with pyarrow (GH#6090) Richard (Rick) Zamora
- Skip
test_partition_on_cats_pyarrow
if pyarrow is not installed (GH#6112) James Bourbeau - Update DataFrame len to handle columns with the same name (GH#6111) James Bourbeau
ArrowEngine
bug fixes and test coverage (GH#6047) Richard (Rick) Zamora- Added mode (GH#5958) Adam Lewis
- Update “helm install” for helm 3 usage (GH#6130) JulianWgs
- Extend preload documentation (GH#6077) Matthew Rocklin
- Fixed small typo in DataFrame
map_partitions()
docstring (GH#6115) Eugene Huang - Fix typo: “double” should be times, not plus (GH#6091) David Chudzicki
- Fix first line of
array.random.*
docs (GH#6063) Martin Durant - Add section about
Semaphore
in distributed (GH#6053) Florian Jetter
2.14.0 / 2020-04-03¶
- Added
np.iscomplexobj
implementation (GH#6045) Tom Augspurger
- Update
test_rearrange_disk_cleanup_with_exception
to pass without cloudpickle installed (GH#6052) James Bourbeau - Fixed flaky
test-rearrange
(GH#5977) Tom Augspurger
- Use
_meta_nonempty
for dtype casting instack_partitions
(GH#6061) mlondschien - Fix bugs in
_metadata
creation and filtering in parquetArrowEngine
(GH#6023) Richard (Rick) Zamora
- DOC: Add name caveats (GH#6040) Tom Augspurger
2.13.0 / 2020-03-25¶
- Support
dtype
and other keyword arguments inda.random
(GH#6030) Matthew Rocklin - Register support for
cupy
sparsehstack
/vstack
(GH#5735) Corey J. Nolet - Force
self.name
tostr
indask.array
(GH#6002) Chuanzhu Xu
- Set
rename_fused_keys
toNone
by default inbag.optimize
(GH#6000) Lucas Rademaker
- Copy dict in
to_graphviz
to prevent overwriting (GH#5996) JulianWgs - Stricter pandas
xfail
(GH#6024) Tom Augspurger - Fix CI failures (GH#6013) James Bourbeau
- Update
toolz
to 0.8.2 and usetlz
(GH#5997) Ryan Grout - Move Windows CI builds to GitHub Actions (GH#5862) James Bourbeau
- Improve path-related exceptions in
read_hdf
(GH#6032) psimaj - Fix
dtype
handling indd.concat
(GH#6006) mlondschien - Handle cudf’s leftsemi and leftanti joins (GH#6025) Richard J Zamora
- Remove unused
npartitions
variable indd.from_pandas
(GH#6019) Daniel Saxton - Added shuffle to
DataFrame.random_split
(GH#5980) petiop
- Fix indentation in scheduler-overview docs (GH#6022) Matthew Rocklin
- Update task graphs in optimize docs (GH#5928) Julia Signell
- Optionally get rid of intermediary boxes in visualize, and add more labels (GH#5976) Julia Signell
2.12.0 / 2020-03-06¶
- Improve reuse of temporaries with numpy (GH#5933) Bruce Merry
- Make
map_blocks
withblock_info
produce aBlockwise
(GH#5896) Bruce Merry - Optimize
make_blockwise_graph
(GH#5940) Bruce Merry - Fix axes ordering in
da.tensordot
(GH#5975) Gil Forsyth - Adds empty mode to
array.pad
(GH#5931) Thomas J. Fan
- Remove
toolz.memoize
dependency indask.utils
(GH#5978) Ryan Grout - Close pool leaking subprocess (GH#5979) Tom Augspurger
- Pin
numpydoc
to0.8.0
(fix double autoescape) (GH#5961) Gil Forsyth - Register deterministic tokenization for
range
objects (GH#5947) James Bourbeau - Unpin
msgpack
in CI (GH#5930) JAmes Bourbeau - Ensure dot results are placed in unique files. (GH#5937) Elliott Sales de Andrade
- Add remaining optional dependencies to Travis 3.8 CI build environment (GH#5920) James Bourbeau
- Skip parquet
getitem
optimization for some keys (GH#5917) Tom Augspurger - Add
ignore_index
argument torearrange_by_column
code path (GH#5973) Richard J Zamora - Add DataFrame and Series
memory_usage_per_partition
methods (GH#5971) James Bourbeau xfail
test_describe when using Pandas 0.24.2 (GH#5948) James Bourbeau- Implement
dask.dataframe.to_numeric
(GH#5929) Julia Signell - Add new error message content when columns are in a different order (GH#5927) Julia Signell
- Use shallow copy for assign operations when possible (GH#5740) Richard J Zamora
- Changed above to below in
dask.array.triu
docs (GH#5984) Henrik Andersson - Array slicing: fix typo in
slice_with_int_dask_array
error message (GH#5981) Gabe Joseph - Grammar and formatting updates to docstrings (GH#5963) James Lamb
- Update develop doc with conda option (GH#5939) Ray Bell
- Update title of DataFrame extension docs (GH#5954) James Bourbeau
- Fixed typos in documentation (GH#5962) James Lamb
- Add original class or module as a
kwarg
on_bind_*
methods (GH#5946) Julia Signell - Add collect list example (GH#5938) Ray Bell
- Update optimization doc for python 3 (GH#5926) Julia Signell
2.11.0 / 2020-02-19¶
- Cache result of
Array.shape
(GH#5916) Bruce Merry - Improve accuracy of
estimate_graph_size
forrechunk
(GH#5907) Bruce Merry - Skip rechunk steps that do not alter chunking (GH#5909) Bruce Merry
- Support
dtype
and otherkwargs
incoarsen
(GH#5903) Matthew Rocklin - Push chunk override from
map_blocks
into blockwise (GH#5895) Bruce Merry - Avoid using
rewrite_blockwise
for a singleton (GH#5890) Bruce Merry - Optimize
slices_from_chunks
(GH#5891) Bruce Merry - Avoid unnecessary
__getitem__
inblock()
when chunks have correct dimensionality (GH#5884) Thomas Robitaille
- Add
include_path
option fordask.bag.read_text
(GH#5836) Yifan Gu - Fixes
ValueError
in delayed execution of bagged NumPy array (GH#5828) Surya Avala
- CI: Pin
msgpack
(GH#5923) Tom Augspurger - Rename
test_inner
totest_outer
(GH#5922) Shiva Raisinghani quote
should quote dicts too (GH#5905) Bruce Merry- Register a normalizer for literal (GH#5898) Bruce Merry
- Improve layer name synthesis for non-HLGs (GH#5888) Bruce Merry
- Replace flake8 pre-commit-hook with upstream (GH#5892) Julia Signell
- Call pip as a module to avoid warnings (GH#5861) Cyril Shcherbin
- Close
ThreadPool
at exit (GH#5852) Tom Augspurger - Remove
dask.dataframe
import in tokenization code (GH#5855) James Bourbeau
- Require
pandas>=0.23
(GH#5883) Tom Augspurger - Remove lambda from dataframe aggregation (GH#5901) Matthew Rocklin
- Fix exception chaining in
dataframe/__init__.py
(GH#5882) Ram Rachum - Add support for reductions on empty dataframes (GH#5804) Shiva Raisinghani
- Expose
sort=
argument for groupby (GH#5801) Richard J Zamora - Add
df.empty
property (GH#5711) rockwellw - Use parquet read speed-ups from
fastparquet.api.paths_to_cats
. (GH#5821) Igor Gotlibovych
- Deprecate
doc_wraps
(GH#5912) Tom Augspurger - Update array internal design docs for HighLevelGraph era (GH#5889) Bruce Merry
- Move over dashboard connection docs (GH#5877) Matthew Rocklin
- Move prometheus docs from distributed.dask.org (GH#5876) Matthew Rocklin
- Removing duplicated DO block at the end (GH#5878) K.-Michael Aye
map_blocks
see also (GH#5874) Tom Augspurger- More derived from (GH#5871) Julia Signell
- Fix typo (GH#5866) Yetunde Dada
- Fix typo in
cloud.rst
(GH#5860) Andrew Thomas - Add note pointing to code of conduct and diversity statement (GH#5844) Matthew Rocklin
2.10.1 / 2020-01-30¶
- Fix Pandas 1.0 version comparison (GH#5851) Tom Augspurger
- Fix typo in distributed diagnostics documentation (GH#5841) Gerrit Holl
2.10.0 / 2020-01-28¶
- Support for pandas 1.0’s new
BooleanDtype
andStringDtype
(GH#5815) Tom Augspurger - Compatibility with pandas 1.0’s API breaking changes and deprecations (GH#5792) Tom Augspurger
- Fixed non-deterministic tokenization of some extension-array backed pandas objects (GH#5813) Tom Augspurger
- Fixed handling of dataclass class objects in collections (GH#5812) Matteo De Wint
- Fixed resampling with tz-aware dates when one of the endpoints fell in a non-existent time (GH#5807) dfonnegra
- Delay initial Zarr dataset creation until the computation occurs (GH#5797) Chris Roat
- Use parquet dataset statistics in more cases with the
pyarrow
engine (GH#5799) Richard J Zamora - Fixed exception in
groupby.std()
when some of the keys were large integers (GH#5737) H. Thomson Comer
2.9.2 / 2020-01-16¶
- Unify chunks in
broadcast_arrays
(GH#5765) Matthew Rocklin
xfail
CSV encoding tests (GH#5791) Tom Augspurger- Update order to handle empty dask graph (GH#5789) James Bourbeau
- Redo
dask.order.order
(GH#5646) Erik Welch
- Add transparent compression for on-disk shuffle with
partd
(GH#5786) Christian Wesp - Fix
repr
for empty dataframes (GH#5781) Shiva Raisinghani - Pandas 1.0.0RC0 compat (GH#5784) Tom Augspurger
- Remove buggy assertions (GH#5783) Tom Augspurger
- Pandas 1.0 compat (GH#5782) Tom Augspurger
- Fix bug in pyarrow-based
read_parquet
on partitioned datasets (GH#5777) Richard J Zamora - Compat for pandas 1.0 (GH#5779) Tom Augspurger
- Fix groupby/mean error with with categorical index (GH#5776) Richard J Zamora
- Support empty partitions when performing cumulative aggregation (GH#5730) Matthew Rocklin
set_index
accepts single-item unnested list (GH#5760) Wes Roach- Fixed partitioning in set index for ordered
Categorical
(GH#5715) Tom Augspurger
- Note additional use case for
normalize_token.register
(GH#5766) Thomas A Caswell - Update bag
repartition
docstring (GH#5772) Timost - Small typos (GH#5771) Maarten Breddels
- Fix typo in Task Expectations docs (GH#5767) James Bourbeau
- Add docs section on task expectations to graph page (GH#5764) Devin Petersohn
2.9.1 / 2019-12-27¶
- Support Array.view with dtype=None (GH#5736) Anderson Banihirwe
- Add dask.array.nanmedian (GH#5684) Deepak Cherian
- xfail test_temporary_directory on Python 3.8 (GH#5734) James Bourbeau
- Add support for Python 3.8 (GH#5603) James Bourbeau
- Use id to dedupe constants in rewrite_blockwise (GH#5696) Jim Crist
- Raise error when converting a dask dataframe scalar to a boolean (GH#5743) James Bourbeau
- Ensure dataframe groupby-variance is greater than zero (GH#5728) Matthew Rocklin
- Fix DataFrame.__iter__ (GH#5719) Tom Augspurger
- Support Parquet filters in disjunctive normal form, like PyArrow (GH#5656) Matteo De Wint
- Auto-detect categorical columns in ArrowEngine-based read_parquet (GH#5690) Richard J Zamora
- Skip parquet getitem optimization tests if no engine found (GH#5697) James Bourbeau
- Fix independent optimization of parquet-getitem (GH#5613) Tom Augspurger
- Update helm config doc (GH#5750) Ray Bell
- Link to examples.dask.org in several places (GH#5733) Tom Augspurger
- Add missing ” in performance report example (GH#5724) James Bourbeau
- Resolve several documentation build warnings (GH#5685) James Bourbeau
- add info on performance_report (GH#5713) Ben Zaitlen
- Add more docs disclaimers (GH#5710) Julia Signell
- Fix simple typo: wihout -> without (GH#5708) Tim Gates
- Update numpydoc dependency (GH#5694) James Bourbeau
2.9.0 / 2019-12-06¶
- Fix
da.std
to work with NumPy arrays (GH#5681) James Bourbeau
- Register
sizeof
functions for Numba and RMM (GH#5668) John A Kirkham - Update meeting time (GH#5682) Tom Augspurger
- Modify
dd.DataFrame.drop
to use shallow copy (GH#5675) Richard J Zamora - Fix bug in
_get_md_row_groups
(GH#5673) Richard J Zamora - Close sqlalchemy engine after querying DB (GH#5629) Krishan Bhasin
- Allow
dd.map_partitions
to not enforce meta (GH#5660) Matthew Rocklin - Generalize
concat_unindexed_dataframes
to support cudf-backend (GH#5659) Richard J Zamora - Add dataframe resample methods (GH#5636) Ben Zaitlen
- Compute length of dataframe as length of first column (GH#5635) Matthew Rocklin
- Doc fixup (GH#5665) James Bourbeau
- Update doc build instructions (GH#5640) James Bourbeau
- Fix ADL link (GH#5639) Ray Bell
- Add documentation build (GH#5617) James Bourbeau
2.8.1 / 2019-11-22¶
- Use auto rechunking in
da.rechunk
if no value given (GH#5605) Matthew Rocklin
- Add simple action to activate GH actions (GH#5619) James Bourbeau
- Fix “file_path_0” bug in
aggregate_row_groups
(GH#5627) Richard J Zamora - Add
chunksize
argument toread_parquet
(GH#5607) Richard J Zamora - Change
test_repartition_npartitions
to support arch64 architecture (GH#5620) ossdev07 - Categories lost after groupby + agg (GH#5423) Oliver Hofkens
- Fixed relative path issue with parquet metadata file (GH#5608) Nuno Gomes Silva
- Enable gpu-backed covariance/correlation in dataframes (GH#5597) Richard J Zamora
- Fix institutional faq and unknown doc warnings (GH#5616) James Bourbeau
- Add doc for some utils (GH#5609) Tom Augspurger
- Removes
html_extra_path
(GH#5614) James Bourbeau - Fixed See Also referencence (GH#5612) Tom Augspurger
2.8.0 / 2019-11-14¶
- Implement complete dask.array.tile function (GH#5574) Bouwe Andela
- Add median along an axis with automatic rechunking (GH#5575) Matthew Rocklin
- Allow da.asarray to chunk inputs (GH#5586) Matthew Rocklin
- Use key_split in Bag name (GH#5571) Matthew Rocklin
- Switch Doctests to Py3.7 (GH#5573) Ryan Nazareth
- Relax get_colors test to adapt to new Bokeh release (GH#5576) Matthew Rocklin
- Add dask.blockwise.fuse_roots optimization (GH#5451) Matthew Rocklin
- Add sizeof implementation for small dicts (GH#5578) Matthew Rocklin
- Update fsspec, gcsfs, s3fs (GH#5588) Tom Augspurger
- Add dropna argument to groupby (GH#5579) Richard J Zamora
- Revert “Remove import of dask_cudf, which is now a part of cudf (GH#5568)” (GH#5590) Matthew Rocklin
- Add best practice for dask.compute function (GH#5583) Matthew Rocklin
- Create FUNDING.yml (GH#5587) Gina Helfrich
- Add screencast for coordination primitives (GH#5593) Matthew Rocklin
- Move funding to .github repo (GH#5589) Tom Augspurger
- Update calendar link (GH#5569) Tom Augspurger
2.7.0 / 2019-11-08¶
This release drops support for Python 3.5
- Reuse code for assert_eq util method (GH#5496) Vijayant
- Update da.array to always return a dask array (GH#5510) James Bourbeau
- Skip transpose on trivial inputs (GH#5523) Ryan Abernathey
- Avoid NumPy scalar string representation in tokenize (GH#5527) James Bourbeau
- Remove unnecessary tiledb shape constraint (GH#5545) Norman Barker
- Removes bytes from sparse array HTML repr (GH#5556) James Bourbeau
- Drop Python 3.5 (GH#5528) James Bourbeau
- Update the use of fixtures in distributed tests (GH#5497) Matthew Rocklin
- Changed deprecated bokeh-port to dashboard-address (GH#5507) darindf
- Avoid updating with identical dicts in ensure_dict (GH#5501) James Bourbeau
- Test Upstream (GH#5516) Tom Augspurger
- Accelerate reverse_dict (GH#5479) Ryan Grout
- Update test_imports.sh (GH#5534) James Bourbeau
- Support cgroups limits on cpu count in multiprocess and threaded schedulers (GH#5499) Albert DeFusco
- Update minimum pyarrow version on CI (GH#5562) James Bourbeau
- Make cloudpickle optional (GH#5511) Guido Imperiale
- Add an example of index_col usage (GH#3072) Bruno Bonfils
- Explicitly use iloc for row indexing (GH#5500) Krishan Bhasin
- Accept dask arrays on columns assignemnt (GH#5224) Henrique Ribeiro-
- Implement unique and value_counts for SeriesGroupBy (GH#5358) Scott Sievert
- Add sizeof definition for pyarrow tables and columns (GH#5522) Richard J Zamora
- Enable row-group task partitioning in pyarrow-based read_parquet (GH#5508) Richard J Zamora
- Removes npartitions=’auto’ from dd.merge docstring (GH#5531) James Bourbeau
- Apply enforce error message shows non-overlapping columns. (GH#5530) Tom Augspurger
- Optimize meta_nonempty for repetitive dtypes (GH#5553) Petio Petrov
- Remove import of dask_cudf, which is now a part of cudf (GH#5568) Mads R. B. Kristensen
- Make capitalization more consistent in FAQ docs (GH#5512) Matthew Rocklin
- Add CONTRIBUTING.md (GH#5513) Jacob Tomlinson
- Document optional dependencies (GH#5456) Prithvi MK
- Update helm chart docs to reflect new chart repo (GH#5539) Jacob Tomlinson
- Add Resampler to API docs (GH#5551) James Bourbeau
- Fix typo in read_sql_table (GH#5554) Eric Dill
- Add adaptive deployments screencast [skip ci] (GH#5566) Matthew Rocklin
2.6.0 / 2019-10-15¶
- Call
ensure_dict
on graphs before enteringtoolz.merge
(GH#5486) Matthew Rocklin - Consolidating hash dispatch functions (GH#5476) Richard J Zamora
- Support Python 3.5 in Parquet code (GH#5491) Ben Zaitlen
- Avoid identity check in
warn_dtype_mismatch
(GH#5489) Tom Augspurger - Enable unused groupby tests (GH#3480) Jörg Dietrich
- Remove old parquet and bcolz dataframe optimizations (GH#5484) Matthew Rocklin
- Add getitem optimization for
read_parquet
(GH#5453) Tom Augspurger - Use
_constructor_sliced
method to determine Series type (GH#5480) Richard J Zamora - Fix map(series) for unsorted base series index (GH#5459) Justin Waugh
- Fix
KeyError
with Groupby label (GH#5467) Ryan Nazareth
- Use Zoom meeting instead of appear.in (GH#5494) Matthew Rocklin
- Added curated list of resources (GH#5460) Javad
- Update SSH docs to include
SSHCluster
(GH#5482) Matthew Rocklin - Update “Why Dask?” page (GH#5473) Matthew Rocklin
- Fix typos in docstrings (GH#5469) garanews
2.5.2 / 2019-10-04¶
- Correct chunk size logic for asymmetric overlaps (GH#5449) Ben Jeffery
- Make da.unify_chunks public API (GH#5443) Matthew Rocklin
- Fix dask.dataframe.fillna handling of Scalar object (GH#5463) Zhenqing Li
- Remove boxes in Spark comparison page (GH#5445) Matthew Rocklin
- Add latest presentations (GH#5446) Javad
- Update cloud documentation (GH#5444) Matthew Rocklin
2.5.0 / 2019-09-27¶
- Add sentinel no_default to get_dependencies task (GH#5420) James Bourbeau
- Update fsspec version (GH#5415) Matthew Rocklin
- Remove PY2 checks (GH#5400) Jim Crist
- Add option to not check meta in dd.from_delayed (GH#5436) Christopher J. Wright
- Fix test_timeseries_nulls_in_schema failures with pyarrow master (GH#5421) Richard J Zamora
- Reduce read_metadata output size in pyarrow/parquet (GH#5391) Richard J Zamora
- Test numeric edge case for repartition with npartitions. (GH#5433) amerkel2
- Unxfail pandas-datareader test (GH#5430) Tom Augspurger
- Add DataFrame.pop implementation (GH#5422) Matthew Rocklin
- Enable merge/set_index for cudf-based dataframes with cupy
values
(GH#5322) Richard J Zamora - drop_duplicates support for positional subset parameter (GH#5410) Wes Roach
- Add screencasts to array, bag, dataframe, delayed, futures and setup (GH#5429) (GH#5424) Matthew Rocklin
- Fix delimeter parsing documentation (GH#5428) Mahmut Bulut
- Update overview image (GH#5404) James Bourbeau
2.4.0 / 2019-09-13¶
- Adds explicit
h5py.File
mode (GH#5390) James Bourbeau - Provides method to compute unknown array chunks sizes (GH#5312) Scott Sievert
- Ignore runtime warning in Array
compute_meta
(GH#5356) estebanag - Add
_meta
toArray.__dask_postpersist__
(GH#5353) Benoit Bovy - Fixup
da.asarray
andda.asanyarray
for datetime64 dtype and xarray objects (GH#5334) Stephan Hoyer - Add shape implementation (GH#5293) Tom Augspurger
- Add chunktype to array text repr (GH#5289) James Bourbeau
- Array.random.choice: handle array-like non-arrays (GH#5283) Gabe Joseph
- Remove deprecated code (GH#5401) Jim Crist
- Fix
funcname
when vectorized func has no__name__
(GH#5399) James Bourbeau - Truncate
funcname
to avoid long key names (GH#5383) Matthew Rocklin - Add support for
numpy.vectorize
infuncname
(GH#5396) James Bourbeau - Fixed HDFS upstream test (GH#5395) Tom Augspurger
- Support numbers and None in
parse_bytes
/timedelta
(GH#5384) Matthew Rocklin - Fix tokenizing of subindexes on memmapped numpy arrays (GH#5351) Henry Pinkard
- Upstream fixups (GH#5300) Tom Augspurger
- Allow pandas to cast type of statistics (GH#5402) Richard J Zamora
- Preserve index dtype after applying
dd.pivot_table
(GH#5385) therhaag - Implement explode for Series and DataFrame (GH#5381) Arpit Solanki
set_index
on categorical fails with less categories than partitions (GH#5354) Oliver Hofkens- Support output to a single CSV file (GH#5304) Hongjiu Zhang
- Add
groupby().transform()
(GH#5327) Oliver Hofkens - Adding filter kwarg to pyarrow dataset call (GH#5348) Richard J Zamora
- Implement and check compression defaults for parquet (GH#5335) Sarah Bird
- Pass sqlalchemy params to delayed objects (GH#5332) Arpit Solanki
- Fixing schema handling in arrow-parquet (GH#5307) Richard J Zamora
- Add support for DF and Series
groupby().idxmin/max()
(GH#5273) Oliver Hofkens - Add correlation calculation and add test (GH#5296) Ben Zaitlen
- Numpy docstring standard has moved (GH#5405) Wes Roach
- Reference correct NumPy array name (GH#5403) Wes Roach
- Minor edits to Array chunk documentation (GH#5372) Scott Sievert
- Add methods to API docs (GH#5387) Tom Augspurger
- Add namespacing to configuration example (GH#5374) Matthew Rocklin
- Add get_task_stream and profile to the diagnostics page (GH#5375) Matthew Rocklin
- Add best practice to load data with Dask (GH#5369) Matthew Rocklin
- Update
institutional-faq.rst
(GH#5345) DomHudson - Add threads and processes note to the best practices (GH#5340) Matthew Rocklin
- Update cuDF links (GH#5328) James Bourbeau
- Fixed small typo with parentheses placement (GH#5311) Eugene Huang
- Update link in reshape docstring (GH#5297) James Bourbeau
2.3.0 / 2019-08-16¶
- Raise exception when
from_array
is given a dask array (GH#5280) David Hoese - Avoid adjusting gufunc’s meta dtype twice (GH#5274) Peter Andreas Entschev
- Add
meta=
keyword to map_blocks and add test with sparse (GH#5269) Matthew Rocklin - Add rollaxis and moveaxis (GH#4822) Tobias de Jong
- Always increment old chunk index (GH#5256) James Bourbeau
- Shuffle dask array (GH#3901) Tom Augspurger
- Fix ordering when indexing a dask array with a bool dask array (GH#5151) James Bourbeau
- Add workaround for memory leaks in bag generators (GH#5208) Marco Neumann
- Set strict xfail option (GH#5220) James Bourbeau
- test-upstream (GH#5267) Tom Augspurger
- Fixed HDFS CI failure (GH#5234) Tom Augspurger
- Error nicely if no file size inferred (GH#5231) Jim Crist
- A few changes to
config.set
(GH#5226) Jim Crist - Fixup black string normalization (GH#5227) Jim Crist
- Pin NumPy in windows tests (GH#5228) Jim Crist
- Ensure parquet tests are skipped if fastparquet and pyarrow not installed (GH#5217) James Bourbeau
- Add fsspec to readthedocs (GH#5207) Matthew Rocklin
- Bump NumPy and Pandas to 1.17 and 0.25 in CI test (GH#5179) John A Kirkham
- Fix
DataFrame.query
docstring (incorrect numexpr API) (GH#5271) Doug Davis - Parquet metadata-handling improvements (GH#5218) Richard J Zamora
- Improve messaging around sorted parquet columns for index (GH#5265) Martin Durant
- Add
rearrange_by_divisions
andset_index
support for cudf (GH#5205) Richard J Zamora - Fix
groupby.std()
with integer colum names (GH#5096) Nicolas Hug - Add
Series.__iter__
(GH#5071) Blane - Generalize
hash_pandas_object
to work for non-pandas backends (GH#5184) GALI PREM SAGAR - Add rolling cov (GH#5154) Ivars Geidans
- Add columns argument in drop function (GH#5223) Henrique Ribeiro
- Update institutional FAQ doc (GH#5277) Matthew Rocklin
- Add draft of institutional FAQ (GH#5214) Matthew Rocklin
- Make boxes for dask-spark page (GH#5249) Martin Durant
- Add motivation for shuffle docs (GH#5213) Matthew Rocklin
- Fix links and API entries for best-practices (GH#5246) Martin Durant
- Remove “bytes” (internal data ingestion) doc page (GH#5242) Martin Durant
- Redirect from our local distributed page to distributed.dask.org (GH#5248) Matthew Rocklin
- Cleanup API page (GH#5247) Matthew Rocklin
- Remove excess endlines from install docs (GH#5243) Matthew Rocklin
- Remove item list in phases of computation doc (GH#5245) Martin Durant
- Remove custom graphs from the TOC sidebar (GH#5241) Matthew Rocklin
- Remove experimental status of custom collections (GH#5236) James Bourbeau
- Adds table of contents to Why Dask? (GH#5244) James Bourbeau
- Moves bag overview to top-level bag page (GH#5240) James Bourbeau
- Remove use-cases in favor of stories.dask.org (GH#5238) Matthew Rocklin
- Removes redundant TOC information in index.rst (GH#5235) James Bourbeau
- Elevate dashboard in distributed diagnostics documentation (GH#5239) Martin Durant
- Updates “add” layer in HLG docs example (GH#5237) James Bourbeau
- Update GUFunc documentation (GH#5232) Matthew Rocklin
2.2.0 / 2019-08-01¶
- Use da.from_array(…, asarray=False) if input follows NEP-18 (GH#5074) Matthew Rocklin
- Add missing attributes to from_array documentation (GH#5108) Peter Andreas Entschev
- Fix meta computation for some reduction functions (GH#5035) Peter Andreas Entschev
- Raise informative error in to_zarr if unknown chunks (GH#5148) James Bourbeau
- Remove invalid pad tests (GH#5122) Tom Augspurger
- Ignore NumPy warnings in compute_meta (GH#5103) Peter Andreas Entschev
- Fix kurtosis calc for single dimension input array (GH#5177) @andrethrill
- Support Numpy 1.17 in tests (GH#5192) Matthew Rocklin
- Supply pool to bag test to resolve intermittent failure (GH#5172) Tom Augspurger
- Base dask on fsspec (GH#5064) (GH#5121) Martin Durant
- Various upstream compatibility fixes (GH#5056) Tom Augspurger
- Make distributed tests optional again. (GH#5128) Elliott Sales de Andrade
- Fix HDFS in dask (GH#5130) Martin Durant
- Ignore some more invalid value warnings. (GH#5140) Elliott Sales de Andrade
- Fix pd.MultiIndex size estimate (GH#5066) Brett Naul
- Generalizing has_known_categories (GH#5090) GALI PREM SAGAR
- Refactor Parquet engine (GH#4995) Richard J Zamora
- Add divide method to series and dataframe (GH#5094) msbrown47
- fix flaky partd test (GH#5111) Tom Augspurger
- Adjust is_dataframe_like to adjust for value_counts change (GH#5143) Tom Augspurger
- Generalize rolling windows to support non-Pandas dataframes (GH#5149) Nick Becker
- Avoid unnecessary aggregation in pivot_table (GH#5173) Daniel Saxton
- Add column names to apply_and_enforce error message (GH#5180) Matthew Rocklin
- Add schema keyword argument to to_parquet (GH#5150) Sarah Bird
- Remove recursion error in accessors (GH#5182) Jim Crist
- Allow fastparquet to handle gather_statistics=False for file lists (GH#5157) Richard J Zamora
- Adds NumFOCUS badge to the README (GH#5086) James Bourbeau
- Update developer docs [ci skip] (GH#5093) Jim Crist
- Document DataFrame.set_index computataion behavior Natalya Rapstine
- Use pip install . instead of calling setup.py (GH#5139) Matthias Bussonier
- Close user survey (GH#5147) Tom Augspurger
- Fix Google Calendar meeting link (GH#5155) Loïc Estève
- Add docker image customization example (GH#5171) James Bourbeau
- Update remote-data-services after fsspec (GH#5170) Martin Durant
- Fix typo in spark.rst (GH#5164) Xavier Holt
- Update setup/python docs for async/await API (GH#5163) Matthew Rocklin
- Update Local Storage HPC documentation (GH#5165) Matthew Rocklin
2.1.0 / 2019-07-08¶
- Add
recompute=
keyword tosvd_compressed
for lower-memory use (GH#5041) Matthew Rocklin - Change
__array_function__
implementation for backwards compatibility (GH#5043) Ralf Gommers - Added
dtype
andshape
kwargs toapply_along_axis
(GH#3742) Davis Bennett - Fix reduction with empty tuple axis (GH#5025) Peter Andreas Entschev
- Drop size 0 arrays in
stack
(GH#4978) John A Kirkham
- Removes index keyword from pandas
to_parquet
call (GH#5075) James Bourbeau - Fixes upstream dev CI build installation (GH#5072) James Bourbeau
- Ensure scalar arrays are not rendered to SVG (GH#5058) Willi Rath
- Environment creation overhaul (GH#5038) Tom Augspurger
- s3fs, moto compatibility (GH#5033) Tom Augspurger
- pytest 5.0 compat (GH#5027) Tom Augspurger
- Fix
compute_meta
recursion in blockwise (GH#5048) Peter Andreas Entschev - Remove hard dependency on pandas in
get_dummies
(GH#5057) GALI PREM SAGAR - Check dtypes unchanged when using
DataFrame.assign
(GH#5047) asmith26 - Fix cumulative functions on tables with more than 1 partition (GH#5034) tshatrov
- Handle non-divisible sizes in repartition (GH#5013) George Sakkis
- Handles timestamp and
preserve_index
changes in pyarrow (GH#5018) Richard J Zamora - Fix undefined
meta
forstr.split(expand=False)
(GH#5022) Brett Naul - Removed checks used for debugging
merge_asof
(GH#5011) Cody Johnson - Don’t use type when getting accessor in dataframes (GH#4992) Matthew Rocklin
- Add
melt
as a method of Dask DataFrame (GH#4984) Dustin Tindall - Adds path-like support to
to_hdf
(GH#5003) James Bourbeau
- Point to latest K8s setup article in JupyterHub docs (GH#5065) Sean McKenna
- Changes vizualize to visualize (GH#5061) David Brochart
- Fix
from_sequence
typo in delayed best practices (GH#5045) James Bourbeau - Add user survey link to docs (GH#5026) James Bourbeau
- Fixes typo in optimization docs (GH#5015) James Bourbeau
- Update community meeting information (GH#5006) Tom Augspurger
2.0.0 / 2019-06-25¶
- Support automatic chunking in da.indices (GH#4981) James Bourbeau
- Err if there are no arrays to stack (GH#4975) John A Kirkham
- Asymmetrical Array Overlap (GH#4863) Michael Eaton
- Dispatch concatenate where possible within dask array (GH#4669) Hameer Abbasi
- Fix tokenization of memmapped numpy arrays on different part of same file (GH#4931) Henry Pinkard
- Preserve NumPy condition in da.asarray to preserve output shape (GH#4945) Alistair Miles
- Expand foo_like_safe usage (GH#4946) Peter Andreas Entschev
- Defer order/casting einsum parameters to NumPy implementation (GH#4914) Peter Andreas Entschev
- Remove numpy warning in moment calculation (GH#4921) Matthew Rocklin
- Fix meta_from_array to support Xarray test suite (GH#4938) Matthew Rocklin
- Cache chunk boundaries for integer slicing (GH#4923) Bruce Merry
- Drop size 0 arrays in concatenate (GH#4167) John A Kirkham
- Raise ValueError if concatenate is given no arrays (GH#4927) John A Kirkham
- Promote types in concatenate using _meta (GH#4925) John A Kirkham
- Add chunk type to html repr in Dask array (GH#4895) Matthew Rocklin
- Add Dask Array._meta attribute (GH#4543) Peter Andreas Entschev
- Fix _meta slicing of flexible types (GH#4912) Peter Andreas Entschev
- Minor meta construction cleanup in concatenate (GH#4937) Peter Andreas Entschev
- Further relax Array meta checks for Xarray (GH#4944) Matthew Rocklin
- Support meta= keyword in da.from_delayed (GH#4972) Matthew Rocklin
- Concatenate meta along axis (GH#4977) John A Kirkham
- Use meta in stack (GH#4976) John A Kirkham
- Move blockwise_meta to more general compute_meta function (GH#4954) Matthew Rocklin
- Alias .partitions to .blocks attribute of dask arrays (GH#4853) Genevieve Buckley
- Drop outdated numpy_compat functions (GH#4850) John A Kirkham
- Allow da.eye to support arbitrary chunking sizes with chunks=’auto’ (GH#4834) Anderson Banihirwe
- Fix CI warnings in dask.array tests (GH#4805) Tom Augspurger
- Make map_blocks work with drop_axis + block_info (GH#4831) Bruce Merry
- Add SVG image and table in Array._repr_html_ (GH#4794) Matthew Rocklin
- ufunc: avoid __array_wrap__ in favor of __array_function__ (GH#4708) Peter Andreas Entschev
- Ensure trivial padding returns the original array (GH#4990) John A Kirkham
- Test
da.block
with 0-size arrays (GH#4991) John A Kirkham
- Drop Python 2.7 (GH#4919) Jim Crist
- Quiet dependency installs in CI (GH#4960) Tom Augspurger
- Raise on warnings in tests (GH#4916) Tom Augspurger
- Add a diagnostics extra to setup.py (includes bokeh) (GH#4924) John A Kirkham
- Add newline delimter keyword to OpenFile (GH#4935) btw08
- Overload HighLevelGraphs values method (GH#4918) James Bourbeau
- Add __await__ method to Dask collections (GH#4901) Matthew Rocklin
- Also ignore AttributeErrors which may occur if snappy (not python-snappy) is installed (GH#4908) Mark Bell
- Canonicalize key names in config.rename (GH#4903) Ian Bolliger
- Bump minimum partd to 0.3.10 (GH#4890) Tom Augspurger
- Catch async def SyntaxError (GH#4836) James Bourbeau
- catch IOError in ensure_file (GH#4806) Justin Poehnelt
- Cleanup CI warnings (GH#4798) Tom Augspurger
- Move distributed’s parse and format functions to dask.utils (GH#4793) Matthew Rocklin
- Apply black formatting (GH#4983) James Bourbeau
- Package license file in wheels (GH#4988) John A Kirkham
- Add an optional partition_size parameter to repartition (GH#4416) George Sakkis
- merge_asof and prefix_reduction (GH#4877) Cody Johnson
- Allow dataframes to be indexed by dask arrays (GH#4882) Endre Mark Borza
- Avoid deprecated message parameter in pytest.raises (GH#4962) James Bourbeau
- Update test_to_records to test with lengths argument(GH#4515) asmith26
- Remove pandas pinning in Dataframe accessors (GH#4955) Matthew Rocklin
- Fix correlation of series with same names (GH#4934) Philipp S. Sommer
- Map Dask Series to Dask Series (GH#4872) Justin Waugh
- Warn in dd.merge on dtype warning (GH#4917) mcsoini
- Add groupby Covariance/Correlation (GH#4889) Ben Zaitlen
- keep index name with to_datetime (GH#4905) Ian Bolliger
- Add Parallel variance computation for dataframes (GH#4865) Ksenia Bobrova
- Add divmod implementation to arrays and dataframes (GH#4884) Henrique Ribeiro
- Add documentation for dataframe reshape methods (GH#4896) tpanza
- Avoid use of pandas.compat (GH#4881) Tom Augspurger
- Added accessor registration for Series, DataFrame, and Index (GH#4829) Tom Augspurger
- Add read_function keyword to read_json (GH#4810) Richard J Zamora
- Provide full type name in check_meta (GH#4819) Matthew Rocklin
- Correctly estimate bytes per row in read_sql_table (GH#4807) Lijo Jose
- Adding support of non-numeric data to describe() (GH#4791) Ksenia Bobrova
- Scalars for extension dtypes. (GH#4459) Tom Augspurger
- Call head before compute in dd.from_delayed (GH#4802) Matthew Rocklin
- Add support for rolling operations with larger window that partition size in DataFrames with Time-based index (GH#4796) Jorge Pessoa
- Update groupby-apply doc with warning (GH#4800) Tom Augspurger
- Change groupby-ness tests in _maybe_slice (GH#4786) Ben Zaitlen
- Add master best practices document (GH#4745) Matthew Rocklin
- Add document for how Dask works with GPUs (GH#4792) Matthew Rocklin
- Add cli API docs (GH#4788) James Bourbeau
- Ensure concat output has coherent dtypes (GH#4692) Guillaume Lemaitre
- Fixes pandas_datareader dependencies installation (GH#4989) James Bourbeau
- Accept pathlib.Path as pattern in read_hdf (GH#3335) Jörg Dietrich
- Move CLI API docs to relavant pages (GH#4980) James Bourbeau
- Add to_datetime function to dataframe API docs Matthew Rocklin
- Add documentation entry for dask.array.ma.average (GH#4970) Bouwe Andela
- Add bag.read_avro to bag API docs (GH#4969) James Bourbeau
- Fix typo (GH#4968) mbarkhau
- Docs: Drop support for Python 2.7 (GH#4932) Hugo
- Remove requirement to modify changelog (GH#4915) Matthew Rocklin
- Add documentation about meta column order (GH#4887) Tom Augspurger
- Add documentation note in DataFrame.shift (GH#4886) Tom Augspurger
- Docs: Fix typo (GH#4868) Paweł Kordek
- Put do/don’t into boxes for delayed best practice docs (GH#3821) Martin Durant
- Doc fixups (GH#2528) Tom Augspurger
- Add quansight to paid support doc section (GH#4838) Martin Durant
- Add document for custom startup (GH#4833) Matthew Rocklin
- Allow utils.derive_from to accept functions, apply across array (GH#4804) Martin Durant
- Add “Avoid Large Partitions” section to best practices (GH#4808) Matthew Rocklin
- Update URL for joblib to new website hosting their doc (GH#4816) Christian Hudon
1.2.2 / 2019-05-08¶
- Clarify regions kwarg to array.store (GH#4759) Martin Durant
- Add dtype= parameter to da.random.randint (GH#4753) Matthew Rocklin
- Use “row major” rather than “C order” in docstring (GH#4452) @asmith26
- Normalize Xarray datasets to Dask arrays (GH#4756) Matthew Rocklin
- Remove normed keyword in da.histogram (GH#4755) Matthew Rocklin
- Add key argument to Bag.distinct (GH#4423) Daniel Severo
- Add core dask config file (GH#4774) Matthew Rocklin
- Add core dask config file to MANIFEST.in (GH#4780) James Bourbeau
- Enabling glob with HTTP file-system (GH#3926) Martin Durant
- HTTPFile.seek with whence=1 (GH#4751) Martin Durant
- Remove config key normalization (GH#4742) Jim Crist
- Remove explicit references to Pandas in dask.dataframe.groupby (GH#4778) Matthew Rocklin
- Add support for group_keys kwarg in DataFrame.groupby() (GH#4771) Brian Chu
- Describe doc (GH#4762) Martin Durant
- Remove explicit pandas check in cumulative aggregations (GH#4765) Nick Becker
- Added meta for read_json and test (GH#4588) Abhinav Ralhan
- Add test for dtype casting (GH#4760) Martin Durant
- Document alignment in map_partitions (GH#4757) Jim Crist
- Implement Series.str.split(expand=True) (GH#4744) Matthew Rocklin
- Tweaks to develop.rst from trying to run tests (GH#4772) Christian Hudon
- Add document describing phases of computation (GH#4766) Matthew Rocklin
- Point users to Dask-Yarn from spark documentation (GH#4770) Matthew Rocklin
- Update images in delayed doc to remove labels (GH#4768) Martin Durant
- Explain intermediate storage for dask arrays (GH#4025) John A Kirkham
- Specify bash code-block in array best practices (GH#4764) James Bourbeau
- Add array best practices doc (GH#4705) Matthew Rocklin
- Update optimization docs now that cull is not automatic (GH#4752) Matthew Rocklin
1.2.1 / 2019-04-29¶
- Fix map_blocks with block_info and broadcasting (GH#4737) Bruce Merry
- Make ‘minlength’ keyword argument optional in da.bincount (GH#4684) Genevieve Buckley
- Add support for map_blocks with no array arguments (GH#4713) Bruce Merry
- Add dask.array.trace (GH#4717) Danilo Horta
- Add sizeof support for cupy.ndarray (GH#4715) Peter Andreas Entschev
- Add name kwarg to from_zarr (GH#4663) Michael Eaton
- Add chunks=’auto’ to from_array (GH#4704) Matthew Rocklin
- Raise TypeError if dask array is given as shape for da.ones, zeros, empty or full (GH#4707) Genevieve Buckley
- Add TileDB backend (GH#4679) Isaiah Norton
- Delay long list arguments (GH#4735) Matthew Rocklin
- Bump to numpy >= 1.13, pandas >= 0.21.0 (GH#4720) Jim Crist
- Remove file “test” (GH#4710) James Bourbeau
- Reenable development build, uses upstream libraries (GH#4696) Peter Andreas Entschev
- Remove assertion in HighLevelGraph constructor (GH#4699) Matthew Rocklin
- Change cum-aggregation last-nonnull-value algorithm (GH#4736) Nick Becker
- Fixup series-groupby-apply (GH#4738) Jim Crist
- Refactor array.percentile and dataframe.quantile to use t-digest (GH#4677) Janne Vuorela
- Allow naive concatenation of sorted dataframes (GH#4725) Matthew Rocklin
- Fix perf issue in dd.Series.isin (GH#4727) Jim Crist
- Remove hard pandas dependency for melt by using methodcaller (GH#4719) Nick Becker
- A few dataframe metadata fixes (GH#4695) Jim Crist
- Add Dataframe.replace (GH#4714) Matthew Rocklin
- Add ‘threshold’ parameter to pd.DataFrame.dropna (GH#4625) Nathan Matare
- Add warning about derived docstrings early in the docstring (GH#4716) Matthew Rocklin
- Create dataframe best practices doc (GH#4703) Matthew Rocklin
- Uncomment dask_sphinx_theme (GH#4728) James Bourbeau
- Fix minor typo fix in a Queue/fire_and_forget example (GH#4709) Matthew Rocklin
- Update from_pandas docstring to match signature (GH#4698) James Bourbeau
1.2.0 / 2019-04-12¶
- Fixed mean() and moment() on sparse arrays (GH#4525) Peter Andreas Entschev
- Add test for NEP-18. (GH#4675) Hameer Abbasi
- Allow None to say “no chunking” in normalize_chunks (GH#4656) Matthew Rocklin
- Fix limit value in auto_chunks (GH#4645) Matthew Rocklin
- Updated diagnostic bokeh test for compatibility with bokeh>=1.1.0 (GH#4680) Philipp Rudiger
- Adjusts codecov’s target/threshold, disable patch (GH#4671) Peter Andreas Entschev
- Always start with empty http buffer, not None (GH#4673) Martin Durant
- Propagate index dtype and name when create dask dataframe from array (GH#4686) Henrique Ribeiro
- Fix ordering of quantiles in describe (GH#4647) gregrf
- Clean up and document rearrange_column_by_tasks (GH#4674) Matthew Rocklin
- Mark some parquet tests xfail (GH#4667) Peter Andreas Entschev
- Fix parquet breakages with arrow 0.13.0 (GH#4668) Martin Durant
- Allow sample to be False when reading CSV from a remote URL (GH#4634) Ian Rose
- Fix timezone metadata inference on parquet load (GH#4655) Martin Durant
- Use is_dataframe/index_like in dd.utils (GH#4657) Matthew Rocklin
- Add min_count parameter to groupby sum method (GH#4648) Henrique Ribeiro
- Correct quantile to handle unsorted quantiles (GH#4650) gregrf
- Add delayed extra dependencies to install docs (GH#4660) James Bourbeau
1.1.5 / 2019-03-29¶
- Ensure that we use the dtype keyword in normalize_chunks (GH#4646) Matthew Rocklin
- Use recursive glob in LocalFileSystem (GH#4186) Brett Naul
- Avoid YAML deprecation (GH#4603)
- Fix CI and add set -e (GH#4605) James Bourbeau
- Support builtin sequence types in dask.visualize (GH#4602)
- unpack/repack orderedDict (GH#4623) Justin Poehnelt
- Add da.random.randint to API docs (GH#4628) James Bourbeau
- Add zarr to CI environment (GH#4604) James Bourbeau
- Enable codecov (GH#4631) Peter Andreas Entschev
- Support setting the index (GH#4565)
- DataFrame.itertuples accepts index, name kwargs (GH#4593) Dan O’Donovan
- Support non-Pandas series in dd.Series.unique (GH#4599) Ben Zaitlen
- Replace use of explicit type check with ._is_partition_type predicate (GH#4533)
- Remove additional pandas warnings in tests (GH#4576)
- Check object for name/dtype attributes rather than type (GH#4606)
- Fix comparison against pd.Series (GH#4613) amerkel2
- Fixing warning from setting categorical codes to floats (GH#4624) Julia Signell
- Fix renaming on index to_frame method (GH#4498) Henrique Ribeiro
- Fix divisions when joining two single-partition dataframes (GH#4636) Justin Waugh
- Warn if partitions overlap in compute_divisions (GH#4600) Brian Chu
- Give informative meta= warning (GH#4637) Matthew Rocklin
- Add informative error message to Series.__getitem__ (GH#4638) Matthew Rocklin
- Add clear exception message when using index or index_col in read_csv (GH#4651) Álvaro Abella Bascarán
- Add documentation for custom groupby aggregations (GH#4571)
- Docs dataframe joins (GH#4569)
- Specify fork-based contributions (GH#4619) James Bourbeau
- correct to_parquet example in docs (GH#4641) Aaron Fowles
- Update and secure several references (GH#4649) Søren Fuglede Jørgensen
1.1.4 / 2019-03-08¶
- Use mask selection in compress (GH#4548) John A Kirkham
- Use asarray in extract (GH#4549) John A Kirkham
- Use correct dtype when test concatenation. (GH#4539) Elliott Sales de Andrade
- Fix CuPy tests or properly marks as xfail (GH#4564) Peter Andreas Entschev
- Fix local scheduler callback to deal with custom caching (GH#4542) Yu Feng
- Use parse_bytes in read_bytes(sample=…) (GH#4554) Matthew Rocklin
- Fix up groupby-standard deviation again on object dtype keys (GH#4541) Matthew Rocklin
- TST/CI: Updates for pandas 0.24.1 (GH#4551) Tom Augspurger
- Add ability to control number of unique elements in timeseries (GH#4557) Matthew Rocklin
- Add support in read_csv for parameter skiprows for other iterables (GH#4560) @JulianWgs
- DataFrame to Array conversion and unknown chunks (GH#4516) Scott Sievert
- Add docs for random array creation (GH#4566) Matthew Rocklin
- Fix typo in docstring (GH#4572) Shyam Saladi
1.1.3 / 2019-03-01¶
- Modify mean chunk functions to return dicts rather than arrays (GH#4513) Matthew Rocklin
- Change sparse installation in CI for NumPy/Python2 compatibility (GH#4537) Matthew Rocklin
- Make merge dispatchable on pandas/other dataframe types (GH#4522) Matthew Rocklin
- read_sql_table - datetime index fix and index type checking (GH#4474) Joe Corbett
- Use generalized form of index checking (is_index_like) (GH#4531) Ben Zaitlen
- Add tests for groupby reductions with object dtypes (GH#4535) Matthew Rocklin
- Fixes #4467 : Updates time_series for pandas deprecation (GH#4530) @HSR05
- Add missing method to documentation index (GH#4528) Bart Broere
1.1.2 / 2019-02-25¶
- Fix another unicode/mixed-type edge case in normalize_array (GH#4489) Marco Neumann
- Add dask.array.diagonal (GH#4431) Danilo Horta
- Call asanyarray in unify_chunks (GH#4506) Jim Crist
- Modify moment chunk functions to return dicts (GH#4519) Peter Andreas Entschev
- Don’t inline output keys in dask.bag (GH#4464) Jim Crist
- Ensure that bag.from_sequence always includes at least one partition (GH#4475) Anderson Banihirwe
- Implement out_type for bag.fold (GH#4502) Matthew Rocklin
- Remove map from bag keynames (GH#4500) Matthew Rocklin
- Avoid itertools.repeat in map_partitions (GH#4507) Matthew Rocklin
- Fix relative path parsing on windows when using fastparquet (GH#4445) Janne Vuorela
- Fix bug in pyarrow and hdfs (GH#4453) (GH#4455) Michał Jastrzębski
- df getitem with integer slices is not implemented (GH#4466) Jim Crist
- Replace cudf-specific code with dask-cudf import (GH#4470) Matthew Rocklin
- Avoid groupby.agg(callable) in groupby-var (GH#4482) Matthew Rocklin
- Consider uint types as numerical in check_meta (GH#4485) Marco Neumann
- Fix some typos in groupby comments (GH#4494) Daniel Saxton
- Add error message around set_index(inplace=True) (GH#4501) Matthew Rocklin
- meta_nonempty works with categorical index (GH#4505) Jim Crist
- Add module name to expected meta error message (GH#4499) Matthew Rocklin
- groupby-nunique works on empty chunk (GH#4504) Jim Crist
- Propagate index metadata if not specified (GH#4509) Jim Crist
- Update docs to use
from_zarr
(GH#4472) John A Kirkham - DOC: add section of Using Other S3-Compatible Services for remote-data-services (GH#4405) Aploium
- Fix header level of section in changelog (GH#4483) Bruce Merry
- Add quotes to pip install [skip-ci] (GH#4508) James Bourbeau
- Extend started_cbs AFTER state is initialized (GH#4460) Marco Neumann
- Fix bug in HTTPFile._fetch_range with headers (GH#4479) (GH#4480) Ross Petchler
- Repeat optimize_blockwise for diamond fusion (GH#4492) Matthew Rocklin
1.1.1 / 2019-01-31¶
- Add support for cupy.einsum (GH#4402) Johnnie Gray
- Provide byte size in chunks keyword (GH#4434) Adam Beberg
- Raise more informative error for histogram bins and range (GH#4430) James Bourbeau
- Lazily register more cudf functions and move to backends file (GH#4396) Matthew Rocklin
- Fix ORC tests for pyarrow 0.12.0 (GH#4413) Jim Crist
- rearrange_by_column: ensure that shuffle arg defaults to ‘disk’ if it’s None in dask.config (GH#4414) George Sakkis
- Implement filters for _read_pyarrow (GH#4415) George Sakkis
- Avoid checking against types in is_dataframe_like (GH#4418) Matthew Rocklin
- Pass username as ‘user’ when using pyarrow (GH#4438) Roma Sokolov
- Fix DelayedAttr return value (GH#4440) Matthew Rocklin
- Use SVG for pipeline graphic (GH#4406) John A Kirkham
- Add doctest-modules to py.test documentation (GH#4427) Daniel Severo
- Work around psutil 5.5.0 not allowing pickling Process objects Janne Vuorela
1.1.0 / 2019-01-18¶
- Fix the average function when there is a masked array (GH#4236) Damien Garaud
- Add allow_unknown_chunksizes to hstack and vstack (GH#4287) Paul Vecchio
- Fix tensordot for 27+ dimensions (GH#4304) Johnnie Gray
- Fixed block_info with axes. (GH#4301) Tom Augspurger
- Use safe_wraps for matmul (GH#4346) Mark Harfouche
- Use chunks=”auto” in array creation routines (GH#4354) Matthew Rocklin
- Fix np.matmul in dask.array.Array.__array_ufunc__ (GH#4363) Stephan Hoyer
- COMPAT: Re-enable multifield copy->view change (GH#4357) Diane Trout
- Calling np.dtype on a delayed object works (GH#4387) Jim Crist
- Rework normalize_array for numpy data (GH#4312) Marco Neumann
- Add fill_value support for series comparisons (GH#4250) James Bourbeau
- Add schema name in read_sql_table for empty tables (GH#4268) Mina Farid
- Adjust check for bad chunks in map_blocks (GH#4308) Tom Augspurger
- Add dask.dataframe.read_fwf (GH#4316) @slnguyen
- Use atop fusion in dask dataframe (GH#4229) Matthew Rocklin
- Use parallel_types() in from_pandas (GH#4331) Matthew Rocklin
- Change DataFrame._repr_data to method (GH#4330) Matthew Rocklin
- Install pyarrow fastparquet for Appveyor (GH#4338) Gábor Lipták
- Remove explicit pandas checks and provide cudf lazy registration (GH#4359) Matthew Rocklin
- Replace isinstance(…, pandas) with is_dataframe_like (GH#4375) Matthew Rocklin
- ENH: Support 3rd-party ExtensionArrays (GH#4379) Tom Augspurger
- Pandas 0.24.0 compat (GH#4374) Tom Augspurger
- Fix link to ‘map_blocks’ function in array api docs (GH#4258) David Hoese
- Add a paragraph on Dask-Yarn in the cloud docs (GH#4260) Jim Crist
- Copy edit documentation (GH#4267), (GH#4263), (GH#4262), (GH#4277), (GH#4271), (GH#4279), (GH#4265), (GH#4295), (GH#4293), (GH#4296), (GH#4302), (GH#4306), (GH#4318), (GH#4314), (GH#4309), (GH#4317), (GH#4326), (GH#4325), (GH#4322), (GH#4332), (GH#4333), Miguel Farrajota
- Fix typo in code example (GH#4272) Daniel Li
- Doc: Update array-api.rst (GH#4259) (GH#4282) Prabakaran Kumaresshan
- Update hpc doc (GH#4266) Guillaume Eynard-Bontemps
- Doc: Replace from_avro with read_avro in documents (GH#4313) Prabakaran Kumaresshan
- Remove reference to “get” scheduler functions in docs (GH#4350) Matthew Rocklin
- Fix typo in docstring (GH#4376) Daniel Saxton
- Added documentation for dask.dataframe.merge (GH#4382) Jendrik Jördening
- Avoid recursion in dask.core.get (GH#4219) Matthew Rocklin
- Remove verbose flag from pytest setup.cfg (GH#4281) Matthew Rocklin
- Support Pytest 4.0 by specifying marks explicitly (GH#4280) Takahiro Kojima
- Add High Level Graphs (GH#4092) Matthew Rocklin
- Fix SerializableLock locked and acquire methods (GH#4294) Stephan Hoyer
- Pin boto3 to earlier version in tests to avoid moto conflict (GH#4276) Martin Durant
- Treat None as missing in config when updating (GH#4324) Matthew Rocklin
- Update Appveyor to Python 3.6 (GH#4337) Gábor Lipták
- Use parse_bytes more liberally in dask.dataframe/bytes/bag (GH#4339) Matthew Rocklin
- Add a better error message when cloudpickle is missing (GH#4342) Mark Harfouche
- Support pool= keyword argument in threaded/multiprocessing get functions (GH#4351) Matthew Rocklin
- Allow updates from arbitrary Mappings in config.update, not only dicts. (GH#4356) Stuart Berg
- Move dask/array/top.py code to dask/blockwise.py (GH#4348) Matthew Rocklin
- Add has_parallel_type (GH#4395) Matthew Rocklin
- CI: Update Appveyor (GH#4381) Tom Augspurger
- Ignore non-readable config files (GH#4388) Jim Crist
1.0.0 / 2018-11-28¶
- Add nancumsum/nancumprod unit tests (GH#4215) Guido Imperiale
- Add index to to_dask_dataframe docstring (GH#4232) James Bourbeau
- Text and fix when appending categoricals with fastparquet (GH#4245) Martin Durant
- Don’t reread metadata when passing ParquetFile to read_parquet (GH#4247) Martin Durant
- Copy edit documentation (GH#4222) (GH#4224) (GH#4228) (GH#4231) (GH#4230) (GH#4234) (GH#4235) (GH#4254) Miguel Farrajota
- Updated doc for the new scheduler keyword (GH#4251) @milesial
- Avoid a few warnings (GH#4223) Matthew Rocklin
- Remove dask.store module (GH#4221) Matthew Rocklin
- Remove AUTHORS.md Jim Crist
0.20.2 / 2018-11-15¶
- Avoid fusing dependencies of atop reductions (GH#4207) Matthew Rocklin
- Improve memory footprint for dataframe correlation (GH#4193) Damien Garaud
- Add empty DataFrame check to boundary_slice (GH#4212) James Bourbeau
- Copy edit documentation (GH#4197) (GH#4204) (GH#4198) (GH#4199) (GH#4200) (GH#4202) (GH#4209) Miguel Farrajota
- Add stats module namespace (GH#4206) James Bourbeau
- Fix link in dataframe documentation (GH#4208) James Bourbeau
0.20.1 / 2018-11-09¶
- Only allocate the result space in wrapped_pad_func (GH#4153) John A Kirkham
- Generalize expand_pad_width to expand_pad_value (GH#4150) John A Kirkham
- Test da.pad with 2D linear_ramp case (GH#4162) John A Kirkham
- Fix import for broadcast_to. (GH#4168) samc0de
- Rewrite Dask Array’s pad to add only new chunks (GH#4152) John A Kirkham
- Validate index inputs to atop (GH#4182) Matthew Rocklin
- Dask.config set and get normalize underscores and hyphens (GH#4143) James Bourbeau
- Only subs on core collections, not subclasses (GH#4159) Matthew Rocklin
- Add block_size=0 option to HTTPFileSystem. (GH#4171) Martin Durant
- Add traverse support for dataclasses (GH#4165) Armin Berres
- Avoid optimization on sharedicts without dependencies (GH#4181) Matthew Rocklin
- Update the pytest version for TravisCI (GH#4189) Damien Garaud
- Use key_split rather than funcname in visualize names (GH#4160) Matthew Rocklin
- Add fix for DataFrame.__setitem__ for index (GH#4151) Anderson Banihirwe
- Fix column choice when passing list of files to fastparquet (GH#4174) Martin Durant
- Pass engine_kwargs from read_sql_table to sqlalchemy (GH#4187) Damien Garaud
- Fix documentation in Delayed best practices example that returned an empty list (GH#4147) Jonathan Fraine
- Copy edit documentation (GH#4164) (GH#4175) (GH#4185) (GH#4192) (GH#4191) (GH#4190) (GH#4180) Miguel Farrajota
- Fix typo in docstring (GH#4183) Carlos Valiente
0.20.0 / 2018-10-26¶
- Fuse Atop operations (GH#3998), (GH#4081) Matthew Rocklin
- Support da.asanyarray on dask dataframes (GH#4080) Matthew Rocklin
- Remove unnecessary endianness check in datetime test (GH#4113) Elliott Sales de Andrade
- Set name=False in array foo_like functions (GH#4116) Matthew Rocklin
- Remove dask.array.ghost module (GH#4121) Matthew Rocklin
- Fix use of getargspec in dask array (GH#4125) Stephan Hoyer
- Adds dask.array.invert (GH#4127), (GH#4131) Anderson Banihirwe
- Raise informative error on arg-reduction on unknown chunksize (GH#4128), (GH#4135) Matthew Rocklin
- Normalize reversed slices in dask array (GH#4126) Matthew Rocklin
- Add bag.to_avro (GH#4076) Martin Durant
- Pull num_workers from config.get (GH#4086), (GH#4093) James Bourbeau
- Fix invalid escape sequences with raw strings (GH#4112) Elliott Sales de Andrade
- Raise an error on the use of the get= keyword and set_options (GH#4077) Matthew Rocklin
- Add import for Azure DataLake storage, and add docs (GH#4132) Martin Durant
- Avoid collections.Mapping/Sequence (GH#4138) Matthew Rocklin
- Include index keyword in to_dask_dataframe (GH#4071) Matthew Rocklin
- add support for duplicate column names (GH#4087) Jan Koch
- Implement min_count for the DataFrame methods sum and prod (GH#4090) Bart Broere
- Remove pandas warnings in concat (GH#4095) Matthew Rocklin
- DataFrame.to_csv header option to only output headers in the first chunk (GH#3909) Rahul Vaidya
- Remove Series.to_parquet (GH#4104) Justin Dennison
- Avoid warnings and deprecated pandas methods (GH#4115) Matthew Rocklin
- Swap ‘old’ and ‘previous’ when reporting append error (GH#4130) Martin Durant
- Copy edit documentation (GH#4073), (GH#4074), (GH#4094), (GH#4097), (GH#4107), (GH#4124), (GH#4133), (GH#4139) Miguel Farrajota
- Fix typo in code example (GH#4089) Antonino Ingargiola
- Add pycon 2018 presentation (GH#4102) Javad
- Quick description for gcsfs (GH#4109) Martin Durant
- Fixed typo in docstrings of read_sql_table method (GH#4114) TakaakiFuruse
- Make target directories in redirects if they don’t exist (GH#4136) Matthew Rocklin
0.19.4 / 2018-10-09¶
- Implement
apply_gufunc(..., axes=..., keepdims=...)
(GH#3985) Markus Gonser
- Fix typo in datasets.make_people (GH#4069) Matthew Rocklin
- Added percentiles options for dask.dataframe.describe method (GH#4067) Zhenqing Li
- Add DataFrame.partitions accessor similar to Array.blocks (GH#4066) Matthew Rocklin
- Pass get functions and Clients through scheduler keyword (GH#4062) Matthew Rocklin
- Fix Typo on hpc example. (missing = in kwarg). (GH#4068) Matthias Bussonier
- Extensive copy-editing: (GH#4065), (GH#4064), (GH#4063) Miguel Farrajota
0.19.3 / 2018-10-05¶
- Make da.RandomState extensible to other modules (GH#4041) Matthew Rocklin
- Support unknown dims in ravel no-op case (GH#4055) Jim Crist
- Add basic infrastructure for cupy (GH#4019) Matthew Rocklin
- Avoid asarray and lock arguments for from_array(getitem) (GH#4044) Matthew Rocklin
- Move local imports in corrcoef to global imports (GH#4030) John A Kirkham
- Move local indices import to global import (GH#4029) John A Kirkham
- Fix-up Dask Array’s fromfunction w.r.t. dtype and kwargs (GH#4028) John A Kirkham
- Don’t use dummy expansion for trim_internal in overlapped (GH#3964) Mark Harfouche
- Add unravel_index (GH#3958) John A Kirkham
- Sort result in Bag.frequencies (GH#4033) Matthew Rocklin
- Add support for npartitions=1 edge case in groupby (GH#4050) James Bourbeau
- Add new random dataset for people (GH#4018) Matthew Rocklin
- Improve performance of bag.read_text on small files (GH#4013) Eric Wolak
- Add bag.read_avro (GH#4000) (GH#4007) Martin Durant
- Added an
index
parameter todask.dataframe.from_dask_array()
for creating a dask DataFrame from a dask Array with a given index. (GH#3991) Tom Augspurger - Improve sub-classability of dask dataframe (GH#4015) Matthew Rocklin
- Fix failing hdfs test [test-hdfs] (GH#4046) Jim Crist
- fuse_subgraphs works without normal fuse (GH#4042) Jim Crist
- Make path for reading many parquet files without prescan (GH#3978) Martin Durant
- Index in dd.from_dask_array (GH#3991) Tom Augspurger
- Making skiprows accept lists (GH#3975) Julia Signell
- Fail early in fastparquet read for nonexistent column (GH#3989) Martin Durant
- Add support for npartitions=1 edge case in groupby (GH#4050) James Bourbeau
- Automatically wrap large arguments with dask.delayed in map_blocks/partitions (GH#4002) Matthew Rocklin
- Fuse linear chains of subgraphs (GH#3979) Jim Crist
- Make multiprocessing context configurable (GH#3763) Itamar Turner-Trauring
- Extensive copy-editing (GH#4049), (GH#4034), (GH#4031), (GH#4020), (GH#4021), (GH#4022), (GH#4023), (GH#4016), (GH#4017), (GH#4010), (GH#3997), (GH#3996), Miguel Farrajota
- Update shuffle method selection docs (GH#4048) James Bourbeau
- Remove docs/source/examples, point to examples.dask.org (GH#4014) Matthew Rocklin
- Replace readthedocs links with dask.org (GH#4008) Matthew Rocklin
- Updates DataFrame.to_hdf docstring for returned values (GH#3992) James Bourbeau
0.19.2 / 2018-09-17¶
apply_gufunc
implements automatic infer of functions output dtypes (GH#3936) Markus Gonser- Fix array histogram range error when array has nans (GH#3980) James Bourbeau
- Issue 3937 follow up, int type checks. (GH#3956) Yu Feng
- from_array: add @martindurant’s explaining of how hashing is done for an array. (GH#3965) Mark Harfouche
- Support gradient with coordinate (GH#3949) Keisuke Fujii
- Fix use of has_keyword with partial in Python 2.7 (GH#3966) Mark Harfouche
- Set pyarrow as default for HDFS (GH#3957) Matthew Rocklin
- Use dask_sphinx_theme (GH#3963) Matthew Rocklin
- Use JupyterLab in Binder links from main page Matthew Rocklin
- DOC: fixed sphinx syntax (GH#3960) Tom Augspurger
0.19.1 / 2018-09-06¶
- Don’t enforce dtype if result has no dtype (GH#3928) Matthew Rocklin
- Fix NumPy issubtype deprecation warning (GH#3939) Bruce Merry
- Fix arg reduction tokens to be unique with different arguments (GH#3955) Tobias de Jong
- Coerce numpy integers to ints in slicing code (GH#3944) Yu Feng
- Linalg.norm ndim along axis partial fix (GH#3933) Tobias de Jong
- Deterministic DataFrame.set_index (GH#3867) George Sakkis
- Fix divisions in read_parquet when dealing with filters #3831 #3930 (GH#3923) (GH#3931) @andrethrill
- Fixing returning type in categorical.as_known (GH#3888) Sriharsha Hatwar
- Fix DataFrame.assign for callables (GH#3919) Tom Augspurger
- Include partitions with no width in repartition (GH#3941) Matthew Rocklin
- Don’t constrict stage/k dtype in dataframe shuffle (GH#3942) Matthew Rocklin
- DOC: Add hint on how to render task graphs horizontally (GH#3922) Uwe Korn
- Add try-now button to main landing page (GH#3924) Matthew Rocklin
0.19.0 / 2018-08-29¶
- Support coordinate in gradient (GH#3949) Keisuke Fujii
- Fix argtopk split_every bug (GH#3810) Guido Imperiale
- Ensure result computing dask.array.isnull() always gives a numpy array (GH#3825) Stephan Hoyer
- Support concatenate for scipy.sparse in dask array (GH#3836) Matthew Rocklin
- Fix argtopk on 32-bit systems. (GH#3823) Elliott Sales de Andrade
- Normalize keys in rechunk (GH#3820) Matthew Rocklin
- Allow shape of dask.array to be a numpy array (GH#3844) Mark Harfouche
- Fix numpy deprecation warning on tuple indexing (GH#3851) Tobias de Jong
- Rename ghost module to overlap (GH#3830) Robert Sare
- Re-add the ghost import to da __init__ (GH#3861) Jim Crist
- Ensure copy preserves masked arrays (GH#3852) Tobias de Jong
- Added
dtype
andsparse
keywords todask.dataframe.get_dummies()
(GH#3792) Tom Augspurger - Added
dask.dataframe.to_dask_array()
for converting a Dask Series or DataFrame to a Dask Array, possibly with known chunk sizes (GH#3884) Tom Augspurger - Changed the behavior for
dask.array.asarray()
for dask dataframe and series inputs. Previously, the series was eagerly converted to an in-memory NumPy array before creating a dask array with known chunks sizes. This caused unexpectedly high memory usage. Now, no intermediate NumPy array is created, and a Dask array with unknown chunk sizes is returned (GH#3884) Tom Augspurger - DataFrame.iloc (GH#3805) Tom Augspurger
- When reading multiple paths, expand globs. (GH#3828) Irina Truong
- Added index column name after resample (GH#3833) Eric Bonfadini
- Add (lazy) shape property to dataframe and series (GH#3212) Henrique Ribeiro
- Fix failing hdfs test [test-hdfs] (GH#3858) Jim Crist
- Fixes for pyarrow 0.10.0 release (GH#3860) Jim Crist
- Rename to_csv keys for diagnostics (GH#3890) Matthew Rocklin
- Match pandas warnings for concat sort (GH#3897) Tom Augspurger
- Include filename in read_csv (GH#3908) Julia Signell
- Better error message on import when missing common dependencies (GH#3771) Danilo Horta
- Drop Python 3.4 support (GH#3840) Jim Crist
- Remove expired deprecation warnings (GH#3841) Jim Crist
- Add DASK_ROOT_CONFIG environment variable (GH#3849) Joe Hamman
- Don’t cull in local scheduler, do cull in delayed (GH#3856) Jim Crist
- Increase conda download retries (GH#3857) Jim Crist
- Add python_requires and Trove classifiers (GH#3855) @hugovk
- Fix collections.abc deprecation warnings in Python 3.7.0 (GH#3876) Jan Margeta
- Allow dot jpeg to xfail in visualize tests (GH#3896) Matthew Rocklin
- Add Python 3.7 to travis.yml (GH#3894) Matthew Rocklin
- Add expand_environment_variables to dask.config (GH#3893) Joe Hamman
- Fix typo in import statement of diagnostics (GH#3826) John Mrziglod
- Add link to YARN docs (GH#3838) Jim Crist
- fix of minor typos in landing page index.html (GH#3746) Christoph Moehl
- Update delayed-custom.rst (GH#3850) Anderson Banihirwe
- DOC: clarify delayed docstring (GH#3709) Scott Sievert
- Add new presentations (GH#3880) Javad
- Add dask array normalize_chunks to documentation (GH#3878) Daniel Rothenberg
- Docs: Fix link to snakeviz (GH#3900) Hans Moritz Günther
- Add missing ` to docstring (GH#3915) @rtobar
0.18.2 / 2018-07-23¶
- Reimplemented
argtopk
to make it release the GIL (GH#3610) Guido Imperiale - Don’t overlap on non-overlapped dimensions in
map_overlap
(GH#3653) Matthew Rocklin - Fix
linalg.tsqr
for dimensions of uncertain length (GH#3662) Jeremy Chen - Break apart uneven array-of-int slicing to separate chunks (GH#3648) Matthew Rocklin
- Align auto chunks to provided chunks, rather than shape (GH#3679) Matthew Rocklin
- Adds endpoint and retstep support for linspace (GH#3675) James Bourbeau
- Implement
.blocks
accessor (GH#3689) Matthew Rocklin - Add
block_info
keyword tomap_blocks
functions (GH#3686) Matthew Rocklin - Slice by dask array of ints (GH#3407) Guido Imperiale
- Support
dtype
inarange
(GH#3722) Guido Imperiale - Fix
argtopk
with uneven chunks (GH#3720) Guido Imperiale - Raise error when
replace=False
inda.choice
(GH#3765) James Bourbeau - Update chunks in
Array.__setitem__
(GH#3767) Itamar Turner-Trauring - Add a
chunksize
convenience property (GH#3777) Jacob Tomlinson - Fix and simplify array slicing behavior when
step < 0
(GH#3702) Ziyao Wei - Ensure
to_zarr
withreturn_stored
True
returns a Dask Array (GH#3786) John A Kirkham
- Add
last_endline
optional parameter into_textfiles
(GH#3745) George Sakkis
- Add aggregate function for rolling objects (GH#3772) Gerome Pistre
- Properly tokenize cumulative groupby aggregations (GH#3799) Cloves Almeida
- Add the
@
operator to the delayed objects (GH#3691) Mark Harfouche - Add delayed best practices to documentation (GH#3737) Matthew Rocklin
- Fix
@delayed
decorator for methods and add tests (GH#3757) Ziyao Wei
- Fix extra progressbar (GH#3669) Mike Neish
- Allow tasks back onto ordering stack if they have one dependency (GH#3652) Matthew Rocklin
- Prefer end-tasks with low numbers of dependencies when ordering (GH#3588) Tom Augspurger
- Add
assert_eq
to top-level modules (GH#3726) Matthew Rocklin - Test that dask collections can hold
scipy.sparse
arrays (GH#3738) Matthew Rocklin - Fix setup of lz4 decompression functions (GH#3782) Elliott Sales de Andrade
- Add datasets module (GH#3780) Matthew Rocklin
0.18.1 / 2018-06-22¶
from_array
now supports scalar types and nested lists/tuples in input, just like all numpy functions do; it also produces a simpler graph when the input is a plain ndarray (GH#3568) Guido Imperiale- Fix slicing of big arrays due to cumsum dtype bug (GH#3620) Marco Rossi
- Add Dask Array implementation of pad (GH#3578) John A Kirkham
- Fix array random API examples (GH#3625) James Bourbeau
- Add average function to dask array (GH#3640) James Bourbeau
- Tokenize ghost_internal with axes (GH#3643) Matthew Rocklin
- Add outer for Dask Arrays (GH#3658) John A Kirkham
- Add Index.to_series method (GH#3613) Henrique Ribeiro
- Fix missing partition columns in pyarrow-parquet (GH#3636) Martin Durant
- Minor tweaks to CI (GH#3629) Guido Imperiale
- Add back dask.utils.effective_get (GH#3642) Matthew Rocklin
- DASK_CONFIG dictates config write location (GH#3621) Jim Crist
- Replace ‘collections’ key in unpack_collections with unique key (GH#3632) Yu Feng
- Avoid deepcopy in dask.config.set (GH#3649) Matthew Rocklin
0.18.0 / 2018-06-14¶
- Add to/from_zarr for Zarr-format datasets and arrays (GH#3460) Martin Durant
- Experimental addition of generalized ufunc support,
apply_gufunc
,gufunc
, andas_gufunc
(GH#3109) (GH#3526) (GH#3539) Markus Gonser - Avoid unnecessary rechunking tasks (GH#3529) Matthew Rocklin
- Compute dtypes at runtime for fft (GH#3511) Matthew Rocklin
- Generate UUIDs for all da.store operations (GH#3540) Martin Durant
- Correct internal dimension of Dask’s SVD (GH#3517) John A Kirkham
- BUG: do not raise IndexError for identity slice in array.vindex (GH#3559) Scott Sievert
- Adds isneginf and isposinf (GH#3581) John A Kirkham
- Drop Dask Array’s learn module (GH#3580) John A Kirkham
- added sfqr (short-and-fat) as a counterpart to tsqr… (GH#3575) Jeremy Chen
- Allow 0-width chunks in dask.array.rechunk (GH#3591) Marc Pfister
- Document Dask Array’s nan_to_num in public API (GH#3599) John A Kirkham
- Show block example (GH#3601) John A Kirkham
- Replace token= keyword with name= in map_blocks (GH#3597) Matthew Rocklin
- Disable locking in to_zarr (needed for using to_zarr in a distributed context) (GH#3607) John A Kirkham
- Support Zarr Arrays in to_zarr/from_zarr (GH#3561) John A Kirkham
- Added recursion to array/linalg/tsqr to better manage the single core bottleneck (GH#3586) Jeremy Chan (GH#3396) Guido Imperiale
- Add to/read_json (GH#3494) Martin Durant
- Adds
index
to unsupported arguments forDataFrame.rename
method (GH#3522) James Bourbeau - Adds support to subset Dask DataFrame columns using
numpy.ndarray
,pandas.Series
, andpandas.Index
objects (GH#3536) James Bourbeau - Raise error if meta columns do not match dataframe (GH#3485) Christopher Ren
- Add index to unsupprted argument for DataFrame.rename (GH#3522) James Bourbeau
- Adds support for subsetting DataFrames with pandas Index/Series and numpy ndarrays (GH#3536) James Bourbeau
- Dataframe sample method docstring fix (GH#3566) James Bourbeau
- fixes dd.read_json to infer file compression (GH#3594) Matt Lee
- Adds n to sample method (GH#3606) James Bourbeau
- Add fastparquet ParquetFile object support (GH#3573) @andrethrill
- Rename method= keyword to shuffle= in bag.groupby (GH#3470) Matthew Rocklin
- Replace get= keyword with scheduler= keyword (GH#3448) Matthew Rocklin
- Add centralized dask.config module to handle configuration for all Dask subprojects (GH#3432) (GH#3513) (GH#3520) Matthew Rocklin
- Add dask-ssh CLI Options and Description. (GH#3476) @beomi
- Read whole files fix regardless of header for HTTP (GH#3496) Martin Durant
- Adds synchronous scheduler syntax to debugging docs (GH#3509) James Bourbeau
- Replace dask.set_options with dask.config.set (GH#3502) Matthew Rocklin
- Update sphinx readthedocs-theme (GH#3516) Matthew Rocklin
- Introduce “auto” value for normalize_chunks (GH#3507) Matthew Rocklin
- Fix check in configuration with env=None (GH#3562) Simon Perkins
- Update sizeof definitions (GH#3582) Matthew Rocklin
- Remove –verbose flag from travis-ci (GH#3477) Matthew Rocklin
- Remove “da.random” from random array keys (GH#3604) Matthew Rocklin
0.17.5 / 2018-05-16¶
- Fix
rechunk
with chunksize of -1 in a dict (GH#3469) Stephan Hoyer einsum
now accepts thesplit_every
parameter (GH#3471) Guido Imperiale- Improved slicing performance (GH#3479) Yu Feng
- Compatibility with pandas 0.23.0 (GH#3499) Tom Augspurger
0.17.4 / 2018-05-03¶
- Add support for indexing Dask DataFrames with string subclasses (GH#3461) James Bourbeau
- Allow using both sorted_index and chunksize in read_hdf (GH#3463) Pierre Bartet
- Pass filesystem to arrow piece reader (GH#3466) Martin Durant
- Switches to using dask.compat string_types (GH#3462) James Bourbeau
0.17.3 / 2018-05-02¶
- Add
einsum
for Dask Arrays (GH#3412) Simon Perkins - Add
piecewise
for Dask Arrays (GH#3350) John A Kirkham - Fix handling of
nan
inbroadcast_shapes
(GH#3356) John A Kirkham - Add
isin
for dask arrays (GH#3363). Stephan Hoyer - Overhauled
topk
for Dask Arrays: faster algorithm, particularly for large k’s; added support for multiple axes, recursive aggregation, and an option to pick the bottom k elements instead. (GH#3395) Guido Imperiale - The
topk
API has changed from topk(k, array) to the more conventional topk(array, k). The legacy API still works but is now deprecated. (GH#2965) Guido Imperiale - New function
argtopk
for Dask Arrays (GH#3396) Guido Imperiale - Fix handling partial depth and boundary in
map_overlap
(GH#3445) John A Kirkham - Add
gradient
for Dask Arrays (GH#3434) John A Kirkham
- Allow t as shorthand for table in to_hdf for pandas compatibility (GH#3330) Jörg Dietrich
- Added top level isna method for Dask DataFrames (GH#3294) Christopher Ren
- Fix selection on partition column on
read_parquet
forengine="pyarrow"
(GH#3207) Uwe Korn - Added DataFrame.squeeze method (GH#3366) Christopher Ren
- Added infer_divisions option to
read_parquet
to specify whether read engines should compute divisions (GH#3387) Jon Mease - Added support for inferring division for
engine="pyarrow"
(GH#3387) Jon Mease - Provide more informative error message for meta= errors (GH#3343) Matthew Rocklin
- add orc reader (GH#3284) Martin Durant
- Default compression for parquet now always Snappy, in line with pandas (GH#3373) Martin Durant
- Fixed bug in Dask DataFrame and Series comparisons with NumPy scalars (GH#3436) James Bourbeau
- Remove outdated requirement from repartition docstring (GH#3440) Jörg Dietrich
- Fixed bug in aggregation when only a Series is selected (GH#3446) Jörg Dietrich
- Add default values to make_timeseries (GH#3421) Matthew Rocklin
- Support traversing collections in persist, visualize, and optimize (GH#3410) Jim Crist
- Add schedule= keyword to compute and persist. This replaces common use of the get= keyword (GH#3448) Matthew Rocklin
0.17.2 / 2018-03-21¶
- Add
broadcast_arrays
for Dask Arrays (GH#3217) John A Kirkham - Add
bitwise_*
ufuncs (GH#3219) John A Kirkham - Add optional
axis
argument tosqueeze
(GH#3261) John A Kirkham - Validate inputs to atop (GH#3307) Matthew Rocklin
- Avoid calls to astype in concatenate if all parts have the same dtype (GH#3301) Martin Durant
- Fixed bug in shuffle due to aggressive truncation (GH#3201) Matthew Rocklin
- Support specifying categorical columns on
read_parquet
withcategories=[…]
forengine="pyarrow"
(GH#3177) Uwe Korn - Add
dd.tseries.Resampler.agg
(GH#3202) Richard Postelnik - Support operations that mix dataframes and arrays (GH#3230) Matthew Rocklin
- Support extra Scalar and Delayed args in
dd.groupby._Groupby.apply
(GH#3256) Gabriele Lanaro
- Support joining against single-partitioned bags and delayed objects (GH#3254) Matthew Rocklin
- Fixed bug when using unexpected but hashable types for keys (GH#3238) Daniel Collins
- Fix bug in task ordering so that we break ties consistently with the key name (GH#3271) Matthew Rocklin
- Avoid sorting tasks in order when the number of tasks is very large (GH#3298) Matthew Rocklin
0.17.1 / 2018-02-22¶
- Corrected dimension chunking in indices (GH#3166, GH#3167) Simon Perkins
- Inline
store_chunk
calls forstore
’sreturn_stored
option (GH#3153) John A Kirkham - Compatibility with struct dtypes for NumPy 1.14.1 release (GH#3187) Matthew Rocklin
- Bugfix to allow column assignment of pandas datetimes(GH#3164) Max Epstein
- New file-system for HTTP(S), allowing direct loading from specific URLs (GH#3160) Martin Durant
- Fix bug when tokenizing partials with no keywords (GH#3191) Matthew Rocklin
- Use more recent LZ4 API (GH#3157) Thrasibule
- Introduce output stream parameter for progress bar (GH#3185) Dieter Weber
0.17.0 / 2018-02-09¶
- Added a support object-type arrays for nansum, nanmin, and nanmax (GH#3133) Keisuke Fujii
- Update error handling when len is called with empty chunks (GH#3058) Xander Johnson
- Fixes a metadata bug with
store
’sreturn_stored
option (GH#3064) John A Kirkham - Fix a bug in
optimization.fuse_slice
to properly handle when first input isNone
(GH#3076) James Bourbeau - Support arrays with unknown chunk sizes in percentile (GH#3107) Matthew Rocklin
- Tokenize scipy.sparse arrays and np.matrix (GH#3060) Roman Yurchak
- Support month timedeltas in repartition(freq=…) (GH#3110) Matthew Rocklin
- Avoid mutation in dataframe groupby tests (GH#3118) Matthew Rocklin
read_csv
,read_table
, andread_parquet
accept iterables of paths (GH#3124) Jim Crist- Deprecates the
dd.to_delayed
function in favor of the existing method (GH#3126) Jim Crist - Return dask.arrays from df.map_partitions calls when the UDF returns a numpy array (GH#3147) Matthew Rocklin
- Change handling of
columns
andindex
indd.read_parquet
to be more consistent, especially in handling of multi-indices (GH#3149) Jim Crist - fastparquet append=True allowed to create new dataset (GH#3097) Martin Durant
- dtype rationalization for sql queries (GH#3100) Martin Durant
- Change default task ordering to prefer nodes with few dependents and then many downstream dependencies (GH#3056) Matthew Rocklin
- Add color= option to visualize to color by task order (GH#3057) (GH#3122) Matthew Rocklin
- Deprecate
dask.bytes.open_text_files
(GH#3077) Jim Crist - Remove short-circuit hdfs reads handling due to maintenance costs. May be re-added in a more robust manner later (GH#3079) Jim Crist
- Add
dask.base.optimize
for optimizing multiple collections without computing. (GH#3071) Jim Crist - Rename
dask.optimize
module todask.optimization
(GH#3071) Jim Crist - Change task ordering to do a full traversal (GH#3066) Matthew Rocklin
- Adds an
optimize_graph
keyword to allto_delayed
methods to allow controlling whether optimizations occur on conversion. (GH#3126) Jim Crist - Support using
pyarrow
for hdfs integration (GH#3123) Jim Crist - Move HDFS integration and tests into dask repo (GH#3083) Jim Crist
- Remove write_bytes (GH#3116) Jim Crist
0.16.1 / 2018-01-09¶
- Fix handling of scalar percentile values in
percentile
(GH#3021) James Bourbeau - Prevent
bool()
coercion from calling compute (GH#2958) Albert DeFusco - Add
matmul
(GH#2904) John A Kirkham - Support N-D arrays with
matmul
(GH#2909) John A Kirkham - Add
vdot
(GH#2910) John A Kirkham - Explicit
chunks
argument forbroadcast_to
(GH#2943) Stephan Hoyer - Add
meshgrid
(GH#2938) John A Kirkham and (GH#3001) Markus Gonser - Preserve singleton chunks in
fftshift
/ifftshift
(GH#2733) John A Kirkham - Fix handling of negative indexes in
vindex
and raise errors for out of bounds indexes (GH#2967) Stephan Hoyer - Add
flip
,flipud
,fliplr
(GH#2954) John A Kirkham - Add
float_power
ufunc (GH#2962) (GH#2969) John A Kirkham - Compatibility for changes to structured arrays in the upcoming NumPy 1.14 release (GH#2964) Tom Augspurger
- Add
block
(GH#2650) John A Kirkham - Add
frompyfunc
(GH#3030) Jim Crist - Add the
return_stored
option tostore
for chaining stored results (GH#2980) John A Kirkham
- Fixed naming bug in cumulative aggregations (GH#3037) Martijn Arts
- Fixed
dd.read_csv
whennames
is given butheader
is not set toNone
(GH#2976) Martijn Arts - Fixed
dd.read_csv
so that passing instances ofCategoricalDtype
indtype
will result in known categoricals (GH#2997) Tom Augspurger - Prevent
bool()
coercion from calling compute (GH#2958) Albert DeFusco DataFrame.read_sql()
(GH#2928) to an empty database tables returns an empty dask dataframe Apostolos Vlachopoulos- Compatibility for reading Parquet files written by PyArrow 0.8.0 (GH#2973) Tom Augspurger
- Correctly handle the column name (df.columns.name) when reading in
dd.read_parquet
(GH#2973) Tom Augspurger - Fixed
dd.concat
losing the index dtype when the data contained a categorical (GH#2932) Tom Augspurger - Add
dd.Series.rename
(GH#3027) Jim Crist DataFrame.merge()
now supports merging on a combination of columns and the index (GH#2960) Jon Mease- Removed the deprecated
dd.rolling*
methods, in preparation for their removal in the next pandas release (GH#2995) Tom Augspurger - Fix metadata inference bug in which single-partition series were mistakenly special cased (GH#3035) Jim Crist
- Add support for
Series.str.cat
(GH#3028) Jim Crist
- Improve 32-bit compatibility (GH#2937) Matthew Rocklin
- Change task prioritization to avoid upwards branching (GH#3017) Matthew Rocklin
0.16.0 / 2017-11-17¶
This is a major release. It includes breaking changes, new protocols, and a large number of bug fixes.
- Add
atleast_1d
,atleast_2d
, andatleast_3d
(GH#2760) (GH#2765) John A Kirkham - Add
allclose
(GH#2771) by John A Kirkham - Remove
random.different_seeds
from Dask Array API docs (GH#2772) John A Kirkham - Deprecate
vnorm
in favor ofdask.array.linalg.norm
(GH#2773) John A Kirkham - Reimplement
unique
to be lazy (GH#2775) John A Kirkham - Support broadcasting of Dask Arrays with 0-length dimensions (GH#2784) John A Kirkham
- Add
asarray
andasanyarray
to Dask Array API docs (GH#2787) James Bourbeau - Support
unique
’sreturn_*
arguments (GH#2779) John A Kirkham - Simplify
_unique_internal
(GH#2850) (GH#2855) John A Kirkham - Avoid removing some getter calls in array optimizations (GH#2826) Jim Crist
- Support
pyarrow
indd.to_parquet
(GH#2868) Jim Crist - Fixed
DataFrame.quantile
andSeries.quantile
returningnan
when missing values are present (GH#2791) Tom Augspurger - Fixed
DataFrame.quantile
losing the result.name
whenq
is a scalar (GH#2791) Tom Augspurger - Fixed
dd.concat
return adask.Dataframe
when concatenating a single series along the columns, matching pandas’ behavior (GH#2800) James Munroe - Fixed default inplace parameter for
DataFrame.eval
to match the pandas defualt for pandas >= 0.21.0 (GH#2838) Tom Augspurger - Fix exception when calling
DataFrame.set_index
on text column where one of the partitions was empty (GH#2831) Jesse Vogt - Do not raise exception when calling
DataFrame.set_index
on empty dataframe (GH#2827) Jesse Vogt - Fixed bug in
Dataframe.fillna
when filling with aSeries
value (GH#2810) Tom Augspurger - Deprecate old argument ordering in
dd.to_parquet
to better match convention of putting the dataframe first (GH#2867) Jim Crist - df.astype(categorical_dtype -> known categoricals (GH#2835) Jim Crist
- Test against Pandas release candidate (GH#2814) Tom Augspurger
- Add more tests for read_parquet(engine=’pyarrow’) (GH#2822) Uwe Korn
- Remove unnecessary map_partitions in aggregate (GH#2712) Christopher Prohm
- Fix bug calling sample on empty partitions (GH#2818) @xwang777
- Error nicely when parsing dates in read_csv (GH#2863) Jim Crist
- Cleanup handling of passing filesystem objects to PyArrow readers (GH#2527) @fjetter
- Support repartitioning even if there are no divisions (GH#2873) @Ced4
- Support reading/writing to hdfs using
pyarrow
indd.to_parquet
(GH#2894, GH#2881) Jim Crist
- Allow tuples as sharedict keys (GH#2763) Matthew Rocklin
- Calling compute within a dask.distributed task defaults to distributed scheduler (GH#2762) Matthew Rocklin
- Auto-import gcsfs when gcs:// protocol is used (GH#2776) Matthew Rocklin
- Fully remove dask.async module, use dask.local instead (GH#2828) Thomas Caswell
- Compatibility with bokeh 0.12.10 (GH#2844) Tom Augspurger
- Reduce test memory usage (GH#2782) Jim Crist
- Add Dask collection interface (GH#2748) Jim Crist
- Update Dask collection interface during XArray integration (GH#2847) Matthew Rocklin
- Close resource profiler process on __exit__ (GH#2871) Jim Crist
- Fix S3 tests (GH#2875) Jim Crist
- Fix port for bokeh dashboard in docs (GH#2889) Ian Hopkinson
- Wrap Dask filesystems for PyArrow compatibility (GH#2881) Jim Crist
0.15.4 / 2017-10-06¶
da.random.choice
now works with array arguments (GH#2781)- Support indexing in arrays with np.int (fixes regression) (GH#2719)
- Handle zero dimension with rechunking (GH#2747)
- Support -1 as an alias for “size of the dimension” in
chunks
(GH#2749) - Call mkdir in array.to_npy_stack (GH#2709)
- Added the .str accessor to Categoricals with string categories (GH#2743)
- Support int96 (spark) datetimes in parquet writer (GH#2711)
- Pass on file scheme to fastparquet (GH#2714)
- Support Pandas 0.21 (GH#2737)
- Add tree reduction support for foldby (GH#2710)
- Drop s3fs from
pip install dask[complete]
(GH#2750)
0.15.3 / 2017-09-24¶
- Add masked arrays (GH#2301)
- Add
*_like array creation functions
(GH#2640) - Indexing with unsigned integer array (GH#2647)
- Improved slicing with boolean arrays of different dimensions (GH#2658)
- Support literals in
top
andatop
(GH#2661) - Optional axis argument in cumulative functions (GH#2664)
- Improve tests on scalars with
assert_eq
(GH#2681) - Fix norm keepdims (GH#2683)
- Add
ptp
(GH#2691) - Add apply_along_axis (GH#2690) and apply_over_axes (GH#2702)
- Added
Series.str[index]
(GH#2634) - Allow the groupby by param to handle columns and index levels (GH#2636)
DataFrame.to_csv
andBag.to_textfiles
now return the filenames to- which they have written (GH#2655)
- Fix combination of
partition_on
andappend
into_parquet
(GH#2645) - Fix for parquet file schemes (GH#2667)
- Repartition works with mixed categoricals (GH#2676)
0.15.2 / 2017-08-25¶
- Remove spurious keys from map_overlap graph (GH#2520)
- where works with non-bool condition and scalar values (GH#2543) (GH#2549)
- Improve compress (GH#2541) (GH#2545) (GH#2555)
- Add argwhere, _nonzero, and where(cond) (GH#2539)
- Generalize vindex in dask.array to handle multi-dimensional indices (GH#2573)
- Add choose method (GH#2584)
- Split code into reorganized files (GH#2595)
- Add linalg.norm (GH#2597)
- Add diff, ediff1d (GH#2607), (GH#2609)
- Improve dtype inference and reflection (GH#2571)
- Remove deprecated Bag behaviors (GH#2525)
- Support callables in assign (GH#2513)
- better error messages for read_csv (GH#2522)
- Add dd.to_timedelta (GH#2523)
- Verify metadata in from_delayed (GH#2534) (GH#2591)
- Add DataFrame.isin (GH#2558)
- Read_hdf supports iterables of files (GH#2547)
- Remove bare
except:
blocks everywhere (GH#2590)
0.15.1 / 2017-07-08¶
0.15.0 / 2017-06-09¶
- Add dask.array.stats submodule (GH#2269)
- Support
ufunc.outer
(GH#2345) - Optimize fancy indexing by reducing graph overhead (GH#2333) (GH#2394)
- Faster array tokenization using alternative hashes (GH#2377)
- Added the matmul
@
operator (GH#2349) - Improved coverage of the
numpy.fft
module (GH#2320) (GH#2322) (GH#2327) (GH#2323) - Support NumPy’s
__array_ufunc__
protocol (GH#2438)
0.14.3 / 2017-05-05¶
- Pandas 0.20.0 support
0.14.2 / 2017-05-03¶
- Add da.indices (GH#2268), da.tile (GH#2153), da.roll (GH#2135)
- Simultaneously support drop_axis and new_axis in da.map_blocks (GH#2264)
- Rechunk and concatenate work with unknown chunksizes (GH#2235) and (GH#2251)
- Support non-numpy container arrays, notably sparse arrays (GH#2234)
- Tensordot contracts over multiple axes (GH#2186)
- Allow delayed targets in da.store (GH#2181)
- Support interactions against lists and tuples (GH#2148)
- Constructor plugins for debugging (GH#2142)
- Multi-dimensional FFTs (single chunk) (GH#2116)
- to_dataframe enforces consistent types (GH#2199)
0.14.1 / 2017-03-22¶
- Micro-optimize optimizations (GH#2058)
- Change slicing optimizations to avoid fusing raw numpy arrays (GH#2075) (GH#2080)
- Dask.array operations now work on numpy arrays (GH#2079)
- Reshape now works in a much broader set of cases (GH#2089)
- Support deepcopy python protocol (GH#2090)
- Allow user-provided FFT implementations in
da.fft
(GH#2093)
- Fix to_parquet with empty partitions (GH#2020)
- Optional
npartitions='auto'
mode inset_index
(GH#2025) - Optimize shuffle performance (GH#2032)
- Support efficient repartitioning along time windows like
repartition(freq='12h')
(GH#2059) - Improve speed of categorize (GH#2010)
- Support single-row dataframe arithmetic (GH#2085)
- Automatically avoid shuffle when setting index with a sorted column (GH#2091)
- Improve handling of integer-na handling in read_csv (GH#2098)
- Repeated attribute access on delayed objects uses the same key (GH#2084)
0.14.0 / 2017-02-24¶
- Fix corner cases with zero shape and misaligned values in
arange
(GH#1902), (GH#1904), (GH#1935), (GH#1955), (GH#1956) - Improve concatenation efficiency (GH#1923)
- Avoid hashing in
from_array
if name is provided (GH#1972)
- Repartition can now increase number of partitions (GH#1934)
- Fix bugs in some reductions with empty partitions (GH#1939), (GH#1950), (GH#1953)
- Support non-uniform categoricals (GH#1877), (GH#1930)
- Groupby cumulative reductions (GH#1909)
- DataFrame.loc indexing now supports lists (GH#1913)
- Improve multi-level groupbys (GH#1914)
- Improved HTML and string repr for DataFrames (GH#1637)
- Parquet append (GH#1940)
- Add
dd.demo.daily_stock
function for teaching (GH#1992)
- Add
traverse=
keyword to delayed to optionally avoid traversing nested data structures (GH#1899) - Support Futures in from_delayed functions (GH#1961)
- Improve serialization of decorated delayed functions (GH#1969)
- Improve windows path parsing in corner cases (GH#1910)
- Rename tasks when fusing (GH#1919)
- Add top level
persist
function (GH#1927) - Propagate
errors=
keyword in byte handling (GH#1954) - Dask.compute traverses Python collections (GH#1975)
- Structural sharing between graphs in dask.array and dask.delayed (GH#1985)
0.13.0 / 2017-01-02¶
- Mandatory dtypes on dask.array. All operations maintain dtype information and UDF functions like map_blocks now require a dtype= keyword if it can not be inferred. (GH#1755)
- Support arrays without known shapes, such as arises when slicing arrays with arrays or converting dataframes to arrays (GH#1838)
- Support mutation by setting one array with another (GH#1840)
- Tree reductions for covariance and correlations. (GH#1758)
- Add SerializableLock for better use with distributed scheduling (GH#1766)
- Improved atop support (GH#1800)
- Rechunk optimization (GH#1737), (GH#1827)
- Avoid wrong results when recomputing the same groupby twice (GH#1867)
- Add
map_overlap
for custom rolling operations (GH#1769) - Add
shift
(GH#1773) - Add Parquet support (GH#1782) (GH#1792) (GH#1810), (GH#1843), (GH#1859), (GH#1863)
- Add missing methods combine, abs, autocorr, sem, nsmallest, first, last, prod, (GH#1787)
- Approximate nunique (GH#1807), (GH#1824)
- Reductions with multiple output partitions (for operations like drop_duplicates) (GH#1808), (GH#1823) (GH#1828)
- Add delitem and copy to DataFrames, increasing mutation support (GH#1858)
- Changed behaviour for
delayed(nout=0)
anddelayed(nout=1)
:delayed(nout=1)
does not default toout=None
anymore, anddelayed(nout=0)
is also enabled. I.e. functions with return tuples of length 1 or 0 can be handled correctly. This is especially handy, if functions with a variable amount of outputs are wrapped bydelayed
. E.g. a trivial example:delayed(lambda *args: args, nout=len(vals))(*vals)
0.12.0 / 2016-11-03¶
- Return a series when functions given to
dataframe.map_partitions
return scalars (GH#1515) - Fix type size inference for series (GH#1513)
dataframe.DataFrame.categorize
no longer includes missing values in thecategories
. This is for compatibility with a pandas change (GH#1565)- Fix head parser error in
dataframe.read_csv
when some lines have quotes (GH#1495) - Add
dataframe.reduction
andseries.reduction
methods to apply generic row-wise reduction to dataframes and series (GH#1483) - Add
dataframe.select_dtypes
, which mirrors the pandas method (GH#1556) dataframe.read_hdf
now supports readingSeries
(GH#1564)- Support Pandas 0.19.0 (GH#1540)
- Implement
select_dtypes
(GH#1556) - String accessor works with indexes (GH#1561)
- Add pipe method to dask.dataframe (GH#1567)
- Add
indicator
keyword to merge (GH#1575) - Support Series in
read_hdf
(GH#1575) - Support Categories with missing values (GH#1578)
- Support inplace operators like
df.x += 1
(GH#1585) - Str accessor passes through args and kwargs (GH#1621)
- Improved groupby support for single-machine multiprocessing scheduler (GH#1625)
- Tree reductions (GH#1663)
- Pivot tables (GH#1665)
- Add clip (GH#1667), align (GH#1668), combine_first (GH#1725), and any/all (GH#1724)
- Improved handling of divisions on dask-pandas merges (GH#1666)
- Add
groupby.aggregate
method (GH#1678) - Add
dd.read_table
function (GH#1682) - Improve support for multi-level columns (GH#1697) (GH#1712)
- Support 2d indexing in
loc
(GH#1726) - Extend
resample
to include DataFrames (GH#1741) - Support dask.array ufuncs on dask.dataframe objects (GH#1669)
- Add information about how
dask.array
chunks
argument work (GH#1504) - Fix field access with non-scalar fields in
dask.array
(GH#1484) - Add concatenate= keyword to atop to concatenate chunks of contracted dimensions
- Optimized slicing performance (GH#1539) (GH#1731)
- Extend
atop
with aconcatenate=
(GH#1609)new_axes=
(GH#1612) andadjust_chunks=
(GH#1716) keywords - Add clip (GH#1610) swapaxes (GH#1611) round (GH#1708) repeat
- Automatically align chunks in
atop
-backed operations (GH#1644) - Cull dask.arrays on slicing (GH#1709)
- Fix issue with callables in
bag.from_sequence
being interpreted as tasks (GH#1491) - Avoid non-lazy memory use in reductions (GH#1747)
- Added changelog (GH#1526)
- Create new threadpool when operating from thread (GH#1487)
- Unify example documentation pages into one (GH#1520)
- Add versioneer for git-commit based versions (GH#1569)
- Pass through node_attr and edge_attr keywords in dot visualization (GH#1614)
- Add continuous testing for Windows with Appveyor (GH#1648)
- Remove use of multiprocessing.Manager (GH#1653)
- Add global optimizations keyword to compute (GH#1675)
- Micro-optimize get_dependencies (GH#1722)
0.11.0 / 2016-08-24¶
DataFrames now enforce knowing full metadata (columns, dtypes) everywhere.
Previously we would operate in an ambiguous state when functions lost dtype
information (such as apply
). Now all dataframes always know their dtypes
and raise errors asking for information if they are unable to infer (which
they usually can). Some internal attributes like _pd
and
_pd_nonempty
have been moved.
The internals of the distributed scheduler have been refactored to transition tasks between explicit states. This improves resilience, reasoning about scheduling, plugin operation, and logging. It also makes the scheduler code easier to understand for newcomers.
- The
distributed.s3
anddistributed.hdfs
namespaces are gone. Use protocols in normal methods likeread_text('s3://...'
instead. Dask.array.reshape
now errs in some cases where previously it would have create a very large number of tasks
0.10.2 / 2016-07-27¶
- More Dataframe shuffles now work in distributed settings, ranging from setting-index to hash joins, to sorted joins and groupbys.
- Dask passes the full test suite when run when under in Python’s optimized-OO mode.
- On-disk shuffles were found to produce wrong results in some highly-concurrent situations, especially on Windows. This has been resolved by a fix to the partd library.
- Fixed a growth of open file descriptors that occurred under large data communications
- Support ports in the
--bokeh-whitelist
option ot dask-scheduler to better routing of web interface messages behind non-trivial network settings - Some improvements to resilience to worker failure (though other known failures persist)
- You can now start an IPython kernel on any worker for improved debugging and analysis
- Improvements to
dask.dataframe.read_hdf
, especially when reading from multiple files and docs
0.10.0 / 2016-06-13¶
- This version drops support for Python 2.6
- Conda packages are built and served from conda-forge
- The
dask.distributed
executables have been renamed from dfoo to dask-foo. For example dscheduler is renamed to dask-scheduler - Both Bag and DataFrame include a preliminary distributed shuffle.
- Add task-based shuffle for distributed groupbys
- Add accumulate for cumulative reductions
- Add a task-based shuffle suitable for distributed joins, groupby-applys, and set_index operations. The single-machine shuffle remains untouched (and much more efficient.)
- Add support for new Pandas rolling API with improved communication performance on distributed systems.
- Add
groupby.std/var
- Pass through S3/HDFS storage options in
read_csv
- Improve categorical partitioning
- Add eval, info, isnull, notnull for dataframes
- Rename executables like dscheduler to dask-scheduler
- Improve scheduler performance in the many-fast-tasks case (important for shuffling)
- Improve work stealing to be aware of expected function run-times and data sizes. The drastically increases the breadth of algorithms that can be efficiently run on the distributed scheduler without significant user expertise.
- Support maximum buffer sizes in streaming queues
- Improve Windows support when using the Bokeh diagnostic web interface
- Support compression of very-large-bytestrings in protocol
- Support clean cancellation of submitted futures in Joblib interface
- All dask-related projects (dask, distributed, s3fs, hdfs, partd) are now building conda packages on conda-forge.
- Change credential handling in s3fs to only pass around delegated credentials if explicitly given secret/key. The default now is to rely on managed environments. This can be changed back by explicitly providing a keyword argument. Anonymous mode must be explicitly declared if desired.
0.9.0 / 2016-05-11¶
dask.do
anddask.value
have been renamed todask.delayed
dask.bag.from_filenames
has been renamed todask.bag.read_text
- All S3/HDFS data ingest functions like
db.from_s3
ordistributed.s3.read_csv
have been moved into the plainread_text
,read_csv functions
, which now support protocols, likedd.read_csv('s3://bucket/keys*.csv')
- Add support for
scipy.LinearOperator
- Improve optional locking to on-disk data structures
- Change rechunk to expose the intermediate chunks
- Rename
from_filename
s toread_text
- Remove
from_s3
in favor ofread_text('s3://...')
- Fixed numerical stability issue for correlation and covariance
- Allow no-hash
from_pandas
for speedy round-trips to and from-pandas objects - Generally reengineered
read_csv
to be more in line with Pandas behavior - Support fast
set_index
operations for sorted columns
- Rename
do/value
todelayed
- Rename
to/from_imperative
toto/from_delayed
- Move s3 and hdfs functionality into the dask repository
- Adaptively oversubscribe workers for very fast tasks
- Improve PyPy support
- Improve work stealing for unbalanced workers
- Scatter data efficiently with tree-scatters
- Add lzma/xz compression support
- Raise a warning when trying to split unsplittable compression types, like gzip or bz2
- Improve hashing for single-machine shuffle operations
- Add new callback method for start state
- General performance tuning
0.8.1 / 2016-03-11¶
- Bugfix for range slicing that could periodically lead to incorrect results.
- Improved support and resiliency of
arg
reductions (argmin
,argmax
, etc.)
- Add
zip
function
- Add
corr
andcov
functions - Add
melt
function - Bugfixes for io to bcolz and hdf5
0.8.0 / 2016-02-20¶
- Changed default array reduction split from 32 to 4
- Linear algebra,
tril
,triu
,LU
,inv
,cholesky
,solve
,solve_triangular
,eye
,lstsq
,diag
,corrcoef
.
- Add tree reductions
- Add range function
- drop
from_hdfs
function (better functionality now exists in hdfs3 and distributed projects)
- Refactor
dask.dataframe
to include a full empty pandas dataframe as metadata. Drop the.columns
attribute on Series - Add Series categorical accessor, series.nunique, drop the
.columns
attribute for series. read_csv
fixes (multi-column parse_dates, integer column names, etc. )- Internal changes to improve graph serialization
- Documentation updates
- Add from_imperative and to_imperative functions for all collections
- Aesthetic changes to profiler plots
- Moved the dask project to a new dask organization
0.7.6 / 2016-01-05¶
- Improve thread safety
- Tree reductions
- Add
view
,compress
,hstack
,dstack
,vstack
methods map_blocks
can now remove and add dimensions
- Improve thread safety
- Extend sampling to include replacement options
- Removed optimization passes that fused results.
- Removed
dask.distributed
- Improved performance of blocked file reading
- Serialization improvements
- Test Python 3.5
0.7.4 / 2015-10-23¶
This was mostly a bugfix release. Some notable changes:
- Fix minor bugs associated with the release of numpy 1.10 and pandas 0.17
- Fixed a bug with random number generation that would cause repeated blocks due to the birthday paradox
- Use locks in
dask.dataframe.read_hdf
by default to avoid concurrency issues - Change
dask.get
to point todask.async.get_sync
by default - Allow visualization functions to accept general graphviz graph options like rankdir=’LR’
- Add reshape and ravel to
dask.array
- Support the creation of
dask.arrays
fromdask.imperative
objects
This release also includes a deprecation warning for dask.distributed
, which
will be removed in the next version.
Future development in distributed computing for dask is happening here: https://distributed.dask.org . General feedback on that project is most welcome from this community.
0.7.3 / 2015-09-25¶
- A utility for profiling memory and cpu usage has been added to the
dask.diagnostics
module.
This release improves coverage of the pandas API. Among other things
it includes nunique
, nlargest
, quantile
. Fixes encoding issues
with reading non-ascii csv files. Performance improvements and bug fixes
with resample. More flexible read_hdf with globbing. And many more. Various
bug fixes in dask.imperative
and dask.bag
.
0.7.0 / 2015-08-15¶
This release includes significant bugfixes and alignment with the Pandas API. This has resulted both from use and from recent involvement by Pandas core developers.
- New operations: query, rolling operations, drop
- Improved operations: quantiles, arithmetic on full dataframes, dropna, constructor logic, merge/join, elemwise operations, groupby aggregations
- Fixed a bug in fold where with a null default argument
- New operations: da.fft module, da.image.imread
- The array and dataframe collections create graphs with deterministic keys. These tend to be longer (hash strings) but should be consistent between computations. This will be useful for caching in the future.
- All collections (Array, Bag, DataFrame) inherit from common subclass
0.6.1 / 2015-07-23¶
- Improved (though not yet sufficient) resiliency for
dask.distributed
when workers die
- Improved writing to various formats, including to_hdf, to_castra, and to_csv
- Improved creation of dask DataFrames from dask Arrays and Bags
- Improved support for categoricals and various other methods
- Various bug fixes
- Histogram function
- Added tie-breaking ordering of tasks within parallel workloads to better handle and clear intermediate results
- Added the dask.do function for explicit construction of graphs with normal python code
- Traded pydot for graphviz library for graph printing to support Python3
- There is also a gitter chat room and a stackoverflow tag