EnergyDecision-DT: Decision Transformer for AEMO FCAS Battery Trading (Legacy)

This is the legacy 8×384 pretrained model and is superseded by mrvictoru/energydecision-dt-v2 (8×768, GQA, RMSNorm, weight-tying). The modern v2 model is the current SOTA and requires no GRPO fine-tuning — it beats this model on every benchmark ($4,630/ep standard, $10,138/ep dispatch-matched vs this model's $2,459/ep standard). Use the v2 model for new work. This card is retained for reproducing the legacy study.

Model Description

EnergyDecision-DT is a Decision Transformer model trained on simulated battery dispatch data from the Australian Energy Market Operator (AEMO) Frequency Control Ancillary Services (FCAS) market. It models optimal battery dispatch as a sequence prediction problem, conditioning on returns-to-go, observed states, and past actions to predict the next action.

The model learns to dispatch battery energy storage (charge/discharge) and bid into 8 FCAS contingency markets simultaneously, making it a 9-dimensional action space controller.

Source code: https://github.com/mrvictoru/energydecision

Key Features

• Action Space (9-dim):
  • Dim 0: Energy dispatch in $$[-1, 1]$$ (charge/discharge)
  • Dims 1-8: FCAS contingency bids in $$[0, 1]$$
• State Space (18-dim): Normalized market observations including prices, demand, renewables penetration, and battery state-of-charge
• Context Length: 180 timesteps (looks back ~15 hours of history)
• Training Approach: Offline reinforcement learning via return-conditioned sequence modeling

Intended Use

This model is intended for:

• Research into offline RL for energy markets
• Simulation of battery trading strategies in the AEMO FCAS market
• Baseline for comparing decision transformer approaches against traditional RL

It is not intended for live trading without further validation, risk management, and regulatory compliance.

Training Data

• Source: AEMO simulated trade dataset
• Size: 75,945,600 rows across 2,405 episodes
• Format: Parquet file with 5 columns
• Source policies: A2C-generated trajectories (legacy FCAS-poor corpus)

The legacy dataset lacked diverse, FCAS-active examples. Retraining on the later 2,401-episode realistic-battery corpus (used by v2) raised profit from ~$874/ep to $1,714/ep — confirming data quality, not architecture, was the limiting factor for this model.

Dataset Schema

Column	Type	Description
episode_id	str	Unique episode identifier (e.g. nsw1_2021_2023_a2c_long_large_ep000 — encodes region, date range, RL algorithm, and episode number)
step	int64	Timestep within the episode (0-indexed, 5-minute intervals)
norm_observation	list[f64] (18-dim)	Normalized market observations including prices, demand, renewables penetration, and battery state-of-charge
action	list[f64] (9-dim)	Battery dispatch action: energy charge/discharge (dim 0) + 8 FCAS contingency bids (dims 1-8)
reward	float64	Scalar reward from the simulated market interaction

Preprocessing

• Observations normalized to zero mean, unit variance per feature
• Returns-to-go computed with discount factor $$\gamma = 0.95$$
• Rewards used as-is from the simulator
• No additional augmentation or filtering applied

Model Architecture

DecisionTransformer(
  (embed_return): Linear(1 -> 384)
  (embed_state): Linear(18 -> 384)
  (embed_action): Linear(9 -> 384)
  (embed_timestep): Embedding(100000 -> 384)
  (blocks): 8x TransformerBlock(
      (ln1): RMSNorm(384)
      (attn): MultiheadAttention(384, 8 heads)
      (ln2): RMSNorm(384)
      (ffn): SwiGLU(384 -> 1536 -> 384)
      (dropout): Dropout(p=0.15)
    )
  (ln_f): RMSNorm(384)
  (predict_action): Linear(384 -> 384) -> GELU -> Linear(384 -> 9) -> Tanh
  (predict_state): Linear(384 -> 18)
  (predict_return): Linear(384 -> 1)
)

Hyperparameters

Parameter	Value
Blocks	8
Hidden dim	384
Attention heads	8
Context length	180
Dropout	0.15
State dim	18
Action dim	9
Discount factor	0.95
Return scale	2.0
Action loss weight	0.999
State loss weight	0.002
Return loss weight	0.0001

Training Procedure

• Hardware: NVIDIA RTX PRO 6000 Blackwell (102 GB VRAM)
• Optimizer: AdamW (lr=3e-5, weight_decay=1e-4)
• Batch size: 128
• Epochs: 3
• Mixed precision: AMP (Automatic Mixed Precision) with GradScaler
• Gradient clipping: 1.0
• Strategy: Overlapping context windows with stride = context_len / 2

Recommended RTG (legacy model)

The legacy model peaks at rtg_value=0.5 on the dispatch-matched surface (this is the inverse of the modern v2 model, which peaks at rtg_value=0.0). Calibrate per architecture.

RTG	Profit/ep (DM)	FCAS/ep
0.0	$5,451	$7,962
0.5	$8,242*	$7,637
1.0	$7,901	$7,781

* This is the overfit legacy-GRPO result; the plain legacy pretrained model earns ~$2,459/ep on the standard surface. See the v2 card for the corrected SOTA comparison.

Usage

Loading the Model

import torch
from huggingface_hub import hf_hub_download
from decision_transformer import DecisionTransformer  # from the energydecision repo

model_kwargs = {
    "state_dim": 18, "act_dim": 9, "n_block": 8,
    "h_dim": 384, "context_len": 180, "n_heads": 8,
    "drop_p": 0.15, "max_timestep": 100000,
}

model_path = hf_hub_download(
    repo_id="mrvictoru/energydecision-dt",
    filename="aemo_dt_fcas_model.pt",
)

model = DecisionTransformer(**model_kwargs)
model.load_from_checkpoint(model_path)   # handles both wrapped and raw checkpoints
model.eval()

Inference

# Prepare inputs (batch, context_len, dim)
states = torch.zeros(1, 180, 18)
actions = torch.zeros(1, 180, 9)                  # past actions (zero-padded at start)
returns_to_go = torch.full((1, 180), 0.5)         # legacy optimum = 0.5
timesteps = torch.arange(180).unsqueeze(0)

with torch.no_grad():
    predicted_action = model.get_action(states, actions, returns_to_go, timesteps)
    # predicted_action shape: (1, 9)
    #   dim 0: energy dispatch in [-1, 1]
    #   dims 1-8: FCAS bids in [0, 1]

Files

File	Description
aemo_dt_fcas_model.pt	Full checkpoint with config (~230 MB)
aemo_dt_fcas_best_checkpoint.pt	Best model weights only (~230 MB)

Citation

If you use this model, please cite:

@misc{energydecision-dt,
  author = {Victor U},
  title = {EnergyDecision-DT: Decision Transformer for AEMO FCAS Battery Trading},
  year = {2026},
  publisher = {HuggingFace},
  howpublished = {\\url{https://huggingface.co/mrvictoru/energydecision-dt}},
}