Skip to article frontmatterSkip to article content

Datashader (Python)

Open In Colab

Datashader (Python)


Datashader turns even the largest datasets into images, faithfully preserving the data’s distribution. ~Datashader Official Website

Introduction

When working with large datasets, traditional visualization libraries like Matplotlib and Seaborn struggle with performance and clarity. These libraries plot individual data points, which can lead to over-plotting and slow rendering when handling millions or billions of points. Datashader, an open-source Python library, solves this problem by rasterizing data into a grid-based format, which is then converted into an image instead of plotting individual points. This ensures that all data is represented without over-plotting, making visualization clearer and more scalable.

Why Traditional Visualization Tools Fail

Most common plotting libraries, such as Matplotlib, Seaborn, and Plotly, render each data point individually. This works well for small datasets but quickly becomes inefficient for large datasets due to:

  • Over-plotting: When too many points overlap, important patterns become hidden.

  • Performance Issues: Processing millions of data points requires significant computational resources.

  • Cluttered Visuals: Traditional scatter plots become unreadable when dealing with large datasets.

To overcome these challenges, we need an alternative approach: Datashader.

Key Features of Datashader

  1. Rasterization of Large Datasets: Converts millions/billions of points into a summarized, readable image.

  2. Automatic Data Aggregation: Uses binning techniques to efficiently represent large-scale data. Automatically adjusts the visualization based on zoom level and density.

  3. Scalability & Compatibility: Handles datasets that traditional visualization tools cannot. Integration with Pandas, Dask, and GeoPandas: Works seamlessly with big data frameworks.

  4. Customizable Pipelines: Allows flexible transformations and styling.

Installing Datashader

To get started, you need to install Datashader along with holoviews, hvplot, and dask for better compatibility using pip.

You’ll also need NumPy, Pandas, and Matplotlib for working with data.

!pip install datashader
!pip install holoviews hvplot dask
!pip install jupyter_bokeh
!pip install numpy pandas matplotlib
!pip install bokeh
!pip install pyarrow fastparquet

Requirement already satisfied: datashader in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (0.17.0)
Requirement already satisfied: colorcet in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (3.1.0)
Requirement already satisfied: multipledispatch in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (1.0.0)
Requirement already satisfied: numba in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (0.61.0)
Requirement already satisfied: numpy in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (2.1.3)
Requirement already satisfied: packaging in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (24.2)
Requirement already satisfied: pandas in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (2.2.3)
Requirement already satisfied: param in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (2.2.0)

Requirement already satisfied: pyct in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (0.5.0)
Requirement already satisfied: requests in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (2.32.3)
Requirement already satisfied: scipy in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (1.15.2)
Requirement already satisfied: toolz in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (1.0.0)
Requirement already satisfied: xarray in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from datashader) (2025.1.2)
Requirement already satisfied: llvmlite<0.45,>=0.44.0dev0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from numba->datashader) (0.44.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas->datashader) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas->datashader) (2025.1)
Requirement already satisfied: tzdata>=2022.7 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas->datashader) (2025.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from requests->datashader) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from requests->datashader) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from requests->datashader) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from requests->datashader) (2025.1.31)
Requirement already satisfied: six>=1.5 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas->datashader) (1.17.0)

Requirement already satisfied: holoviews in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (1.20.1)
Requirement already satisfied: hvplot in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (0.11.2)
Requirement already satisfied: dask in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (2025.2.0)
Requirement already satisfied: bokeh>=3.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from holoviews) (3.6.3)
Requirement already satisfied: colorcet in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from holoviews) (3.1.0)
Requirement already satisfied: numpy>=1.21 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from holoviews) (2.1.3)
Requirement already satisfied: packaging in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from holoviews) (24.2)

Requirement already satisfied: pandas>=1.3 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from holoviews) (2.2.3)
Requirement already satisfied: panel>=1.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from holoviews) (1.6.1)
Requirement already satisfied: param<3.0,>=2.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from holoviews) (2.2.0)
Requirement already satisfied: pyviz-comms>=2.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from holoviews) (3.0.4)
Requirement already satisfied: click>=8.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from dask) (8.1.8)
Requirement already satisfied: cloudpickle>=3.0.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from dask) (3.1.1)
Requirement already satisfied: fsspec>=2021.09.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from dask) (2025.2.0)
Requirement already satisfied: partd>=1.4.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from dask) (1.4.2)
Requirement already satisfied: pyyaml>=5.3.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from dask) (6.0.2)
Requirement already satisfied: toolz>=0.10.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from dask) (1.0.0)
Requirement already satisfied: importlib_metadata>=4.13.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from dask) (8.6.1)
Requirement already satisfied: Jinja2>=2.9 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh>=3.1->holoviews) (3.1.5)
Requirement already satisfied: contourpy>=1.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh>=3.1->holoviews) (1.3.1)
Requirement already satisfied: pillow>=7.1.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh>=3.1->holoviews) (11.1.0)
Requirement already satisfied: tornado>=6.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh>=3.1->holoviews) (6.4.2)
Requirement already satisfied: xyzservices>=2021.09.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh>=3.1->holoviews) (2025.1.0)
Requirement already satisfied: zipp>=3.20 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from importlib_metadata>=4.13.0->dask) (3.21.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.3->holoviews) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.3->holoviews) (2025.1)

Requirement already satisfied: tzdata>=2022.7 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.3->holoviews) (2025.1)
Requirement already satisfied: bleach in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from panel>=1.0->holoviews) (6.2.0)
Requirement already satisfied: linkify-it-py in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from panel>=1.0->holoviews) (2.0.3)
Requirement already satisfied: markdown in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from panel>=1.0->holoviews) (3.7)
Requirement already satisfied: markdown-it-py in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from panel>=1.0->holoviews) (3.0.0)
Requirement already satisfied: mdit-py-plugins in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from panel>=1.0->holoviews) (0.4.2)
Requirement already satisfied: requests in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from panel>=1.0->holoviews) (2.32.3)
Requirement already satisfied: tqdm in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from panel>=1.0->holoviews) (4.67.1)
Requirement already satisfied: typing-extensions in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from panel>=1.0->holoviews) (4.12.2)
Requirement already satisfied: locket in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from partd>=1.4.0->dask) (1.0.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from Jinja2>=2.9->bokeh>=3.1->holoviews) (3.0.2)
Requirement already satisfied: six>=1.5 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas>=1.3->holoviews) (1.17.0)
Requirement already satisfied: webencodings in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bleach->panel>=1.0->holoviews) (0.5.1)
Requirement already satisfied: uc-micro-py in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from linkify-it-py->panel>=1.0->holoviews) (1.0.3)
Requirement already satisfied: mdurl~=0.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from markdown-it-py->panel>=1.0->holoviews) (0.1.2)

Requirement already satisfied: charset-normalizer<4,>=2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from requests->panel>=1.0->holoviews) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from requests->panel>=1.0->holoviews) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from requests->panel>=1.0->holoviews) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from requests->panel>=1.0->holoviews) (2025.1.31)

Requirement already satisfied: jupyter_bokeh in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (4.0.5)
Requirement already satisfied: bokeh==3.* in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from jupyter_bokeh) (3.6.3)
Requirement already satisfied: ipywidgets==8.* in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from jupyter_bokeh) (8.1.5)
Requirement already satisfied: Jinja2>=2.9 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh==3.*->jupyter_bokeh) (3.1.5)
Requirement already satisfied: contourpy>=1.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh==3.*->jupyter_bokeh) (1.3.1)
Requirement already satisfied: numpy>=1.16 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh==3.*->jupyter_bokeh) (2.1.3)
Requirement already satisfied: packaging>=16.8 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh==3.*->jupyter_bokeh) (24.2)
Requirement already satisfied: pandas>=1.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh==3.*->jupyter_bokeh) (2.2.3)
Requirement already satisfied: pillow>=7.1.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh==3.*->jupyter_bokeh) (11.1.0)
Requirement already satisfied: PyYAML>=3.10 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh==3.*->jupyter_bokeh) (6.0.2)
Requirement already satisfied: tornado>=6.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh==3.*->jupyter_bokeh) (6.4.2)
Requirement already satisfied: xyzservices>=2021.09.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh==3.*->jupyter_bokeh) (2025.1.0)
Requirement already satisfied: comm>=0.1.3 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipywidgets==8.*->jupyter_bokeh) (0.2.2)
Requirement already satisfied: ipython>=6.1.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipywidgets==8.*->jupyter_bokeh) (8.32.0)
Requirement already satisfied: traitlets>=4.3.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipywidgets==8.*->jupyter_bokeh) (5.14.3)
Requirement already satisfied: widgetsnbextension~=4.0.12 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipywidgets==8.*->jupyter_bokeh) (4.0.13)
Requirement already satisfied: jupyterlab-widgets~=3.0.12 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipywidgets==8.*->jupyter_bokeh) (3.0.13)

Requirement already satisfied: decorator in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (5.2.1)
Requirement already satisfied: jedi>=0.16 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (0.19.2)
Requirement already satisfied: matplotlib-inline in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (0.1.7)
Requirement already satisfied: pexpect>4.3 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (4.9.0)
Requirement already satisfied: prompt_toolkit<3.1.0,>=3.0.41 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (3.0.50)
Requirement already satisfied: pygments>=2.4.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (2.19.1)
Requirement already satisfied: stack_data in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (0.6.3)
Requirement already satisfied: typing_extensions>=4.6 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (4.12.2)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from Jinja2>=2.9->bokeh==3.*->jupyter_bokeh) (3.0.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.2->bokeh==3.*->jupyter_bokeh) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.2->bokeh==3.*->jupyter_bokeh) (2025.1)
Requirement already satisfied: tzdata>=2022.7 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.2->bokeh==3.*->jupyter_bokeh) (2025.1)
Requirement already satisfied: parso<0.9.0,>=0.8.4 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from jedi>=0.16->ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (0.8.4)
Requirement already satisfied: ptyprocess>=0.5 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pexpect>4.3->ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (0.7.0)
Requirement already satisfied: wcwidth in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from prompt_toolkit<3.1.0,>=3.0.41->ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (0.2.13)
Requirement already satisfied: six>=1.5 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas>=1.2->bokeh==3.*->jupyter_bokeh) (1.17.0)
Requirement already satisfied: executing>=1.2.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from stack_data->ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (2.2.0)
Requirement already satisfied: asttokens>=2.1.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from stack_data->ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (3.0.0)
Requirement already satisfied: pure-eval in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from stack_data->ipython>=6.1.0->ipywidgets==8.*->jupyter_bokeh) (0.2.3)

Requirement already satisfied: numpy in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (2.1.3)
Requirement already satisfied: pandas in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (2.2.3)
Requirement already satisfied: matplotlib in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (3.10.0)

Requirement already satisfied: python-dateutil>=2.8.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas) (2025.1)
Requirement already satisfied: tzdata>=2022.7 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas) (2025.1)
Requirement already satisfied: contourpy>=1.0.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from matplotlib) (1.3.1)
Requirement already satisfied: cycler>=0.10 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from matplotlib) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from matplotlib) (4.56.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from matplotlib) (1.4.8)
Requirement already satisfied: packaging>=20.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from matplotlib) (24.2)
Requirement already satisfied: pillow>=8 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from matplotlib) (11.1.0)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from matplotlib) (3.2.1)
Requirement already satisfied: six>=1.5 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)

Requirement already satisfied: bokeh in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (3.6.3)
Requirement already satisfied: Jinja2>=2.9 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh) (3.1.5)
Requirement already satisfied: contourpy>=1.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh) (1.3.1)
Requirement already satisfied: numpy>=1.16 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh) (2.1.3)
Requirement already satisfied: packaging>=16.8 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh) (24.2)
Requirement already satisfied: pandas>=1.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh) (2.2.3)
Requirement already satisfied: pillow>=7.1.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh) (11.1.0)
Requirement already satisfied: PyYAML>=3.10 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh) (6.0.2)
Requirement already satisfied: tornado>=6.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh) (6.4.2)
Requirement already satisfied: xyzservices>=2021.09.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from bokeh) (2025.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from Jinja2>=2.9->bokeh) (3.0.2)

Requirement already satisfied: python-dateutil>=2.8.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.2->bokeh) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.2->bokeh) (2025.1)
Requirement already satisfied: tzdata>=2022.7 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.2->bokeh) (2025.1)
Requirement already satisfied: six>=1.5 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas>=1.2->bokeh) (1.17.0)

Requirement already satisfied: pyarrow in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (19.0.1)
Requirement already satisfied: fastparquet in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (2024.11.0)
Requirement already satisfied: pandas>=1.5.0 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from fastparquet) (2.2.3)
Requirement already satisfied: numpy in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from fastparquet) (2.1.3)
Requirement already satisfied: cramjam>=2.3 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from fastparquet) (2.9.1)
Requirement already satisfied: fsspec in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from fastparquet) (2025.2.0)
Requirement already satisfied: packaging in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from fastparquet) (24.2)

Requirement already satisfied: python-dateutil>=2.8.2 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.5.0->fastparquet) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.5.0->fastparquet) (2025.1)
Requirement already satisfied: tzdata>=2022.7 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from pandas>=1.5.0->fastparquet) (2025.1)
Requirement already satisfied: six>=1.5 in /opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas>=1.5.0->fastparquet) (1.17.0)

import datashader as ds
import holoviews as hv, pandas as pd, numpy as np, colorcet as cc
import bokeh
import panel as pn
import hvplot.pandas
from holoviews.element.tiles import EsriImagery
import datashader.transfer_functions as tf
from holoviews.operation.datashader import rasterize
from holoviews.operation.datashader import datashade
from IPython.display import display, Image
hv.extension('bokeh', inline=True)
Loading...

Reading the Data

Let’s take a practical scenario of evaluating NYC Taxi dataset.

# Using a paraquet dataset for easy column extraction!
df = pd.read_parquet(
    'https://s3.amazonaws.com/datashader-data/nyc_taxi_wide.parq',
    columns = ['dropoff_x', 'dropoff_y']
)
print(len(df)) # the data set is pretty large
print("The Dataset has approx 12 million data points!")
df.head()
Loading...

Using python .plot fuction let’s plot a scatter plot of 2 columns from our dataset.

%%time
# it's so slow
# the plot is shabby and incomprehensible
# No concrete pattern can be drawn
df.plot.scatter('dropoff_x', 'dropoff_y')
CPU times: user 824 ms, sys: 107 ms, total: 931 ms
Wall time: 1.81 s
<Axes: xlabel='dropoff_x', ylabel='dropoff_y'>
<Figure size 640x480 with 1 Axes>

Plotting Using Datashader

# from google.colab import output
# output.enable_custom_widget_manager()
%%time
agg = ds.Canvas().points(df, 'dropoff_x', 'dropoff_y')
ds.tf.set_background(ds.tf.shade(agg, cmap=cc.fire), "black")
Loading...

Getting a plot like this with other approaches would take quite a bit of time and effort, but with Datashader it appears in milliseconds without trial and error.

The output above is just a bare image, which is all that Datashader knows how to generate directly. But Datashader can integrate closely with Bokeh, HoloViews, and GeoViews, which makes it simple to allow interactive zooming, axis labeling, overlays and layouts, and complex web apps. For example, making a zoomable interactive overlay on a geographic map requires just a few more lines of code:

%%time
map_tiles = EsriImagery().opts(alpha=0.5, width=900, height=480, bgcolor='black')
points = hv.Points(df, ['dropoff_x', 'dropoff_y'])
taxi_trips = datashade(points, cmap=cc.fire, width=900, height=480) # Datashade for Rasterized plot

# Display the visualization
final = map_tiles * taxi_trips
# display(map_tiles * taxi_trips)
hv.output(final)
Loading...

Datashader made it incredibly easy and efficient to create a detailed and comprehensible scatter plot with just four lines of code by significantly reducing processing time.

This powerful library streamlines data visualization by following a five-step pipeline:

  1. Projection
    Maps raw data points into a coordinate space suitable for visualization.
  2. Aggregation
    Groups and summarizes data into a fixed grid, reducing complexity.
  3. Transformation
    Applies scaling, filtering, or mathematical operations to refine the data.
  4. Colormapping
    Assigns colors based on data values to enhance interpretability.
  5. Embedding
    Renders the processed image into a final visual output for display.
Datashader (1).png

Projection

canvas = ds.Canvas(plot_width=900, plot_height=480)
canvas
<datashader.core.Canvas at 0x7fa0da5f1f90>

Aggregation

canvas.points(df, 'dropoff_x', 'dropoff_y', agg=ds.count())
Loading...

Transformation (optional)

Can perform various transformations using Datashader such as selection, masking, etc

# Here we are squaring the data points to enhance the density and reveal details in dense regions.
np.power(canvas.points(df, 'dropoff_x', 'dropoff_y', agg=ds.count()),2)
Loading...

Colormapping

tf.set_background(
    tf.shade(
        canvas.points(df, 'dropoff_x', 'dropoff_y', agg=ds.count()),
        cmap=cc.fire
    ),
    'black'
)
Loading...

Embedding

# from google.colab import output
# output.enable_custom_widget_manager()
map_tiles = EsriImagery().opts(alpha=0.5, width=900, height=480, bgcolor='black')
points = hv.Points(df, ['dropoff_x', 'dropoff_y'])
ropts = dict(tools=['hover'], colorbar=True, colorbar_position='bottom', cmap=cc.fire, cnorm='eq_hist')
taxi_trips = rasterize(points).opts(**ropts)
display(map_tiles * taxi_trips)
Loading...

Conclusion

When to use this library:

Use this when you have a very large data set. If you find this visualization aesthetically appealing then you can use this for smaller datasets as well, but the results will depend on the density distribution of your data. You won’t get high interactivity, because datashader prioritizes performance and clarity as it overlays images rather than vector-based plots. But you can still zoom and pan the visualization.

Practical Applications of Datashader

Datashader is widely used in fields that require high-performance visualization of large datasets. Here are some key applications:

1. Geospatial Data Visualization
  • Example: Mapping millions of GPS coordinates from taxi rides, earthquakes, or population data.
  • Why Datashader? Efficiently renders dense clusters without losing detail.
2. Finance and Market Analysis
  • Example: Visualizing high-frequency trading data, stock market trends, or large-scale financial transactions.
  • Why Datashader? Handles billions of data points quickly, revealing patterns in market fluctuations.
3. Scientific and Medical Research
  • Example: Processing and visualizing genomic data, brain imaging, or large-scale simulations.
  • Why Datashader? Helps analyze complex, high-dimensional datasets without information loss.
4. Social Media & Network Analysis
  • Example: Tracking user activity, sentiment analysis, or connection patterns in networks like Twitter.
  • Why Datashader? Helps visualize millions of interactions and detect trends in online behavior.
6. Machine Learning & AI Model Debugging
  • Example: Understanding high-dimensional feature distributions or anomaly detection in ML datasets.
  • Why Datashader? Enables fast exploration of large datasets without performance bottlenecks.

References & Further Reading:

Documentatiom & Articles

Yotube Resources

!jupyter nbconvert --to html "Datashader (Python).ipynb"
[NbConvertApp] WARNING | pattern 'Datashader (Python).ipynb' matched no files
This application is used to convert notebook files (*.ipynb)
        to various other formats.

        WARNING: THE COMMANDLINE INTERFACE MAY CHANGE IN FUTURE RELEASES.

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to logging.DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=10]
--show-config
    Show the application's configuration (human-readable format)
    Equivalent to: [--Application.show_config=True]
--show-config-json
    Show the application's configuration (json format)
    Equivalent to: [--Application.show_config_json=True]
--generate-config
    generate default config file
    Equivalent to: [--JupyterApp.generate_config=True]
-y
    Answer yes to any questions instead of prompting.
    Equivalent to: [--JupyterApp.answer_yes=True]
--execute
    Execute the notebook prior to export.
    Equivalent to: [--ExecutePreprocessor.enabled=True]
--allow-errors
    Continue notebook execution even if one of the cells throws an error and include the error message in the cell output (the default behaviour is to abort conversion). This flag is only relevant if '--execute' was specified, too.
    Equivalent to: [--ExecutePreprocessor.allow_errors=True]
--stdin
    read a single notebook file from stdin. Write the resulting notebook with default basename 'notebook.*'
    Equivalent to: [--NbConvertApp.from_stdin=True]
--stdout
    Write notebook output to stdout instead of files.
    Equivalent to: [--NbConvertApp.writer_class=StdoutWriter]
--inplace
    Run nbconvert in place, overwriting the existing notebook (only
            relevant when converting to notebook format)
    Equivalent to: [--NbConvertApp.use_output_suffix=False --NbConvertApp.export_format=notebook --FilesWriter.build_directory=]
--clear-output
    Clear output of current file and save in place,
            overwriting the existing notebook.
    Equivalent to: [--NbConvertApp.use_output_suffix=False --NbConvertApp.export_format=notebook --FilesWriter.build_directory= --ClearOutputPreprocessor.enabled=True]
--coalesce-streams
    Coalesce consecutive stdout and stderr outputs into one stream (within each cell).
    Equivalent to: [--NbConvertApp.use_output_suffix=False --NbConvertApp.export_format=notebook --FilesWriter.build_directory= --CoalesceStreamsPreprocessor.enabled=True]
--no-prompt
    Exclude input and output prompts from converted document.
    Equivalent to: [--TemplateExporter.exclude_input_prompt=True --TemplateExporter.exclude_output_prompt=True]
--no-input
    Exclude input cells and output prompts from converted document.
            This mode is ideal for generating code-free reports.
    Equivalent to: [--TemplateExporter.exclude_output_prompt=True --TemplateExporter.exclude_input=True --TemplateExporter.exclude_input_prompt=True]
--allow-chromium-download
    Whether to allow downloading chromium if no suitable version is found on the system.
    Equivalent to: [--WebPDFExporter.allow_chromium_download=True]
--disable-chromium-sandbox
    Disable chromium security sandbox when converting to PDF..
    Equivalent to: [--WebPDFExporter.disable_sandbox=True]
--show-input
    Shows code input. This flag is only useful for dejavu users.
    Equivalent to: [--TemplateExporter.exclude_input=False]
--embed-images
    Embed the images as base64 dataurls in the output. This flag is only useful for the HTML/WebPDF/Slides exports.
    Equivalent to: [--HTMLExporter.embed_images=True]
--sanitize-html
    Whether the HTML in Markdown cells and cell outputs should be sanitized..
    Equivalent to: [--HTMLExporter.sanitize_html=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--config=<Unicode>
    Full path of a config file.
    Default: ''
    Equivalent to: [--JupyterApp.config_file]
--to=<Unicode>
    The export format to be used, either one of the built-in formats
            ['asciidoc', 'custom', 'html', 'latex', 'markdown', 'notebook', 'pdf', 'python', 'qtpdf', 'qtpng', 'rst', 'script', 'slides', 'webpdf']
            or a dotted object name that represents the import path for an
            ``Exporter`` class
    Default: ''
    Equivalent to: [--NbConvertApp.export_format]
--template=<Unicode>
    Name of the template to use
    Default: ''
    Equivalent to: [--TemplateExporter.template_name]
--template-file=<Unicode>
    Name of the template file to use
    Default: None
    Equivalent to: [--TemplateExporter.template_file]
--theme=<Unicode>
    Template specific theme(e.g. the name of a JupyterLab CSS theme distributed
    as prebuilt extension for the lab template)
    Default: 'light'
    Equivalent to: [--HTMLExporter.theme]
--sanitize_html=<Bool>
    Whether the HTML in Markdown cells and cell outputs should be sanitized.This
    should be set to True by nbviewer or similar tools.
    Default: False
    Equivalent to: [--HTMLExporter.sanitize_html]
--writer=<DottedObjectName>
    Writer class used to write the
                                        results of the conversion
    Default: 'FilesWriter'
    Equivalent to: [--NbConvertApp.writer_class]
--post=<DottedOrNone>
    PostProcessor class used to write the
                                        results of the conversion
    Default: ''
    Equivalent to: [--NbConvertApp.postprocessor_class]
--output=<Unicode>
    Overwrite base name use for output files.
                Supports pattern replacements '{notebook_name}'.
    Default: '{notebook_name}'
    Equivalent to: [--NbConvertApp.output_base]
--output-dir=<Unicode>
    Directory to write output(s) to. Defaults
                                  to output to the directory of each notebook. To recover
                                  previous default behaviour (outputting to the current
                                  working directory) use . as the flag value.
    Default: ''
    Equivalent to: [--FilesWriter.build_directory]
--reveal-prefix=<Unicode>
    The URL prefix for reveal.js (version 3.x).
            This defaults to the reveal CDN, but can be any url pointing to a copy
            of reveal.js.
            For speaker notes to work, this must be a relative path to a local
            copy of reveal.js: e.g., "reveal.js".
            If a relative path is given, it must be a subdirectory of the
            current directory (from which the server is run).
            See the usage documentation
            (https://nbconvert.readthedocs.io/en/latest/usage.html#reveal-js-html-slideshow)
            for more details.
    Default: ''
    Equivalent to: [--SlidesExporter.reveal_url_prefix]
--nbformat=<Enum>
    The nbformat version to write.
            Use this to downgrade notebooks.
    Choices: any of [1, 2, 3, 4]
    Default: 4
    Equivalent to: [--NotebookExporter.nbformat_version]

Examples
--------

    The simplest way to use nbconvert is

            > jupyter nbconvert mynotebook.ipynb --to html

            Options include ['asciidoc', 'custom', 'html', 'latex', 'markdown', 'notebook', 'pdf', 'python', 'qtpdf', 'qtpng', 'rst', 'script', 'slides', 'webpdf'].

            > jupyter nbconvert --to latex mynotebook.ipynb

            Both HTML and LaTeX support multiple output templates. LaTeX includes
            'base', 'article' and 'report'.  HTML includes 'basic', 'lab' and
            'classic'. You can specify the flavor of the format used.

            > jupyter nbconvert --to html --template lab mynotebook.ipynb

            You can also pipe the output to stdout, rather than a file

            > jupyter nbconvert mynotebook.ipynb --stdout

            PDF is generated via latex

            > jupyter nbconvert mynotebook.ipynb --to pdf

            You can get (and serve) a Reveal.js-powered slideshow

            > jupyter nbconvert myslides.ipynb --to slides --post serve

            Multiple notebooks can be given at the command line in a couple of
            different ways:

            > jupyter nbconvert notebook*.ipynb
            > jupyter nbconvert notebook1.ipynb notebook2.ipynb

            or you can specify the notebooks list in a config file, containing::

                c.NbConvertApp.notebooks = ["my_notebook.ipynb"]

            > jupyter nbconvert --config mycfg.py

To see all available configurables, use `--help-all`.