Publish Time

Overview

The Grid Status API provides sophisticated publish time filtering capabilities that allow you to control which data points are returned based on when they were published or reported. This is particularly useful for forecasting data, where multiple predictions may exist for the same operating time but were published at different times.

Publish time filtering is only available for datasets that have a publish_time_column. You can identify these datasets by checking the dataset metadata - datasets with publish time capabilities will have a non-null publish_time_column field.

Available Publish Time Parameters

1. publish_time - Primary Filtering Parameter

The publish_time parameter is the main way to filter data based on publication timing. It supports several different modes:

1.1. latest_report - Most Recent Report

Returns only data from the most recently published report across the entire dataset.

Use Case: Get the very latest forecast or report available

Example:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=latest_report&limit=5&api_key=YOUR_API_KEY"

Python

1.2. latest - Latest for Each Time Index

For any given operating timestamp, returns the most recently published data point.

Use Case: Get the most up-to-date forecast for each operating hour

Dataset Requirement: Requires the dataset to have both a publish_time_column and time_index_column

Example:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=latest&start_time=2024-01-01&end_time=2024-01-02&limit=10&api_key=YOUR_API_KEY"

Python

Response: For each interval_start_utc timestamp, returns the record with the most recent publish_time_utc.

1.3. Specific Timestamp

Provide an ISO 8601 timestamp to get only data published at that exact time.

Use Case: Retrieve data from a specific forecast time

Example:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=2024-01-01T12:30:00Z&start_time=2024-01-01&end_time=2024-01-02&limit=5&api_key=YOUR_API_KEY"

Python

1.4. latest_before - Latest Before Cutoff

Returns the most recent forecast for each operating time where the publish time is before or equal to a calculated cutoff time relative to each operating time.

Formats:

Format	Description
latest_before	Shorthand for latest_before:-0 hours. Returns the latest forecast published at or before each operating time
latest_before:<offset>	e.g., latest_before:-6 hours. Retrieves forecasts published at least 6 hours before each operating time
latest_before:<offset>@<HH:MM:SS>	e.g., latest_before:-1 day@10:30:00. Retrieves forecasts published at or before 10:30 AM on the day before each operating time

Use Case: Get forecasts published at least X time before the operating time, useful for day-ahead vs real-time analysis

Examples:

Basic offset:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=latest_before:-6%20hours&start_time=2024-01-01&end_time=2024-01-02&limit=5&api_key=YOUR_API_KEY"

Python

With specific time:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=latest_before:-1%20day@10:30:00&start_time=2024-01-01&end_time=2024-01-02&limit=5&api_key=YOUR_API_KEY"

Python

Logic:

• For latest_before:-6 hours: publish_time ≤ operating_time - 6 hours

• For latest_before:-1 day@10:30:00: publish_time ≤ 10:30:00 on (operating_time - 1 day)

1.5. window - Time Window

Returns all forecasts published within a specific time window relative to the operating time.

Formats:

Format	Description
window:<offset>	e.g., window:-6 hours. Returns all forecasts published in the 6 hours leading up to each operating time
window:<offset>@<HH:MM:SS>	e.g., window:-1 day@10:00:00. Returns all forecasts published from 10:00 AM the day before up to the operating time

Use Case: Analyze all forecasts published within a specific timeframe before the operating time

Example:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=window:-6%20hours&start_time=2024-01-01&end_time=2024-01-02&limit=10&api_key=YOUR_API_KEY"

Python

Logic: (operating_time + offset) ≤ publish_time ≤ operating_time

For window:-6 hours: (operating_time - 6 hours) ≤ publish_time ≤ operating_time

---

2. publish_time_start - Start Range Filter

Filter data to include only records published on or after this timestamp.

• Use Case: Get all data published since a specific date/time

• Cannot be used with: publish_time parameter

• Format: ISO 8601 datetime string

Example:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time_start=2024-01-01T00:00:00Z&start_time=2024-01-01&end_time=2024-01-02&limit=10&api_key=YOUR_API_KEY"

Python

---

3. publish_time_end - End Range Filter

Filter data to include only records published before this timestamp (exclusive).

• Use Case: Get historical data published before a specific cutoff

• Cannot be used with: publish_time parameter

• Format: ISO 8601 datetime string

Example:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time_end=2024-01-01T12:00:00Z&start_time=2024-01-01&end_time=2024-01-02&limit=10&api_key=YOUR_API_KEY"

Python

---

4. Combined Range Filtering

You can use both publish_time_start and publish_time_end together to create a specific publication time window.

Example:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time_start=2024-01-01T06:00:00Z&publish_time_end=2024-01-01T18:00:00Z&start_time=2024-01-01&end_time=2024-01-02&limit=10&api_key=YOUR_API_KEY"

Python

This returns all forecasts published between 6 AM and 6 PM on January 1st, 2024.

---

Offset Format Specification

For latest_before and window parameters, the offset format follows these rules:

Basic Offset Format

<amount> <unit> where:

• amount: Number (can be negative, positive, or decimal)

• unit: One of: seconds, minutes, hours, days, weeks, months, years

Examples:

• -6 hours

• 1.5 days

• -30 minutes

• 2 weeks

Time-Specific Format

<offset>@<HH:MM:SS> where:

• offset: Basic offset as above

• @: Separator

• HH:MM:SS: Specific time in 24-hour format

Examples:

• -1 day@10:30:00 - 10:30 AM on the previous day

• -2 days@14:15:30 - 2:15:30 PM two days prior

URL Encoding

When using offsets in URLs, remember to URL-encode spaces:

• -6 hours becomes -6%20hours

• -1 day@10:30:00 becomes -1%20day@10:30:00

---

Common Use Cases and Examples

1. Day-Ahead vs Real-Time Forecast Analysis

Compare day-ahead forecasts (published ~24 hours before) with near-real-time forecasts:

Day-ahead forecasts:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=latest_before:-20%20hours&start_time=2024-01-01&end_time=2024-01-02&api_key=YOUR_API_KEY"

Python

Near real-time forecasts:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=latest_before:-1%20hour&start_time=2024-01-01&end_time=2024-01-02&api_key=YOUR_API_KEY"

Python

2. Morning Forecast Analysis

Get forecasts published during morning hours (6 AM - 12 PM):

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time_start=2024-01-01T06:00:00Z&publish_time_end=2024-01-01T12:00:00Z&start_time=2024-01-01&end_time=2024-01-02&api_key=YOUR_API_KEY"

Python

3. Forecast Revision Analysis

Analyze how forecasts change over time by getting all forecasts in a 6-hour window before operating time:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=window:-6%20hours&start_time=2024-01-01T12:00:00Z&end_time=2024-01-01T18:00:00Z&api_key=YOUR_API_KEY"

Python

4. Historical Snapshot

Get the most recent data as it was available at a specific point in time:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=2024-07-01T00:30:00Z&start_time=2024-07-01&end_time=2024-07-02&api_key=YOUR_API_KEY"

Python

---

Response Format

All responses include the publish_time_utc column when available, allowing you to verify the publication timing:

{
  "status_code": 200,
  "data": [
    {
      "interval_start_utc": "2024-01-01T00:00:00+00:00",
      "interval_end_utc": "2024-01-01T01:00:00+00:00",
      "publish_time_utc": "2023-12-31T18:30:00+00:00",
      "north": 15543.961113146968,
      "south": 11921.243918188475,
      "west": 5854.474702258297,
      "houston": 11600.86030546875,
      "system_total": 44920.54003906249
    }
  ],
  "meta": {
    "page": 1,
    "limit": 10,
    "hasNextPage": false
  },
  "dataset_metadata": {
    "publish_time_column": "publish_time_utc",
    "time_index_column": "interval_start_utc"
  }
}

---

Error Handling and Validation

Common Errors

Invalid Offset Format

{
  "status_code": 400,
  "detail": "Invalid offset format: -6hour. Expected format: '-6 hours' or '1 day'"
}

Invalid Time Format

{
  "status_code": 400,
  "detail": "Invalid time format: 25:00:00. Expected HH:MM:SS"
}

Dataset Without Publish Time

{
  "status_code": 400,
  "detail": "A dataset must have a time index to use 'latest' or 'latest_before:' options. Only 'latest_report' or a timestamp publish time are supported for datasets without a time index"
}

Conflicting Parameters

{
  "status_code": 400,
  "detail": "Cannot use publish_time with publish_time_start or publish_time_end"
}

---

Best Practices

1. Check Dataset Compatibility

Before using publish time filtering, verify that the dataset has a publish_time_column:

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone?api_key=YOUR_API_KEY"

Look for "publish_time_column": "publish_time_utc" in the response.

2. Use Appropriate Time Zones

Use timezone="market" when your publish-time filter should be interpreted in the dataset's market timezone. This is especially useful for filters like latest_before:-1 day@10:00:00, where the cutoff should follow the market's local clock and daylight saving time rules.

3. Limit Result Sets

Publish time filtering can return large datasets. Always use limit parameters appropriately:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=window:-24%20hours&limit=1000&start_time=2024-01-01&end_time=2024-01-02&api_key=YOUR_API_KEY"

Python

4. Combine with Other Filters

Publish time filters work well with other parameters like start_time, end_time, and column filters:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=latest&start_time=2024-01-01&end_time=2024-01-02&columns=interval_start_utc,publish_time_utc,system_total&api_key=YOUR_API_KEY"

Python

5. Performance Considerations

• latest_report is typically the fastest option as it returns the smallest dataset

• window operations can be slower for large time ranges

• Consider using pagination for large result sets

---

Timezone Handling

The API accepts a timezone parameter that affects how publish time calculations are performed:

Shell

curl -H "Accept-Encoding: br, gzip" "https://api.gridstatus.io/v1/datasets/ercot_load_forecast_by_forecast_zone/query?publish_time=latest_before:-1%20day@10:30:00&timezone=America/Chicago&start_time=2024-01-01&end_time=2024-01-02&api_key=YOUR_API_KEY"

Python

When a timezone is specified:

• The time calculations for @HH:MM:SS formats are performed in the specified timezone

• The input timestamps are converted to the specified timezone before processing

• Results are still returned in UTC

---

Summary

The Grid Status API's publish time filtering provides powerful capabilities for:

• Getting the latest available data (latest, latest_report)

• Historical analysis (latest_before with timestamps)

• Forecast revision tracking (window operations)

• Time-based publication filtering (publish_time_start, publish_time_end)

These features enable sophisticated analysis of how forecasts evolve over time and allow users to recreate historical views of data as it was available at specific points in time.