Getting Started

Bulk CSV Downloads is in early beta. We'd love your feedback — please reach out with any questions or issues.

Bulk CSV Downloads provides our entire data catalog as compressed CSV flat files, delivered through AWS S3. The full export is approximately 1 TB compressed, and S3 gives you high-throughput parallel downloads to efficiently backfill your systems with our data.

We will share access to the export using AWS Security Token Service (STS), which grants you a temporary, limited-privilege credential to access the files in S3.

Prerequisites

Please send us:

• AWS Account ID

• Confirmation you can use sts:AssumeRole

You must use an IAM user or role when downloading files. A root account will not work (this is a limitation of AWS).

We will send you the following credentials so you can then access the data.

• RoleArn

• ExternalId

Configure the AWS CLI (recommended option for downloading data)

First install the AWS CLI.

In ~/.aws/config, add a gridstatus profile. Using a named profile will allow the CLI to handle credential refresh automatically.

[profile gridstatus]
role_arn = <RoleArn>
external_id = <ExternalId>
source_profile = default

s3 =
  max_concurrent_requests = 20

Verify it works by listing the available datasets:

aws s3 ls s3://gs-catalog-csv/run=2026-02-17/ --profile gridstatus

PRE aeso_daily_average_pool_price/
PRE aeso_fuel_mix/
PRE aeso_interchange/
...

If you see dataset folders listed, your credentials are working. See Example Usage for more commands.

Other Options

• Python with s3fs - Use S3FileSystem with assume_role_arn and assume_role_kwargs to download files.

• Python with boto3 - Use RefreshableCredentials via STS AssumeRole to list and download objects with concurrent transfers.