How do I implement the old switch chunk_day using the new type of syntax sim.run()?
We use it since we sometime have memory issues in Windows env., or maybe this is not needed anymore?
Ok, found it.
“run_kwargs (dict) –
Note: Additional keyword arguments are currently not supported due to underlying function signature constraints. This parameter is reserved for future use.
In a future version, the following options may be supported: - save_state: bool - Save state at each timestep (planned) - chunk_day: int - Days per chunk for memory efficiency (planned)
For now, simulations use default settings: - save_state=False (states not saved at each step) - chunk_day=3660 (approximately 10 years per chunk)”
When do you plan to implement this again?
This topic has been tracked as a GitHub issue for the development team.
GitHub issue: How to implement chunk_day in new syntax · Issue #1195 · UMEP-dev/SUEWS · GitHub
The team will follow up there. Updates will be posted in the GitHub issue.
Hasn’t it been implemented yet? I’ve been using sim.run(chunk_day=31) with no error messages in 2025.11.20rc1.
Good question @Janka. In 2025.11.20rc1, sim.run() did accept chunk_day without raising an error, but it was actually silently ignored — the parameter got caught by the **run_kwargs catch-all and was never passed through to the simulation engine. So your runs were processing the full forcing period in a single pass regardless of the value you set.
This has been fixed (PR #1172, merged 30 Jan 2026) — chunk_day is now an explicit parameter on sim.run() and properly wired through to the backend. It’ll be available in the next public release.
Hi,
For some reason I’m not able to get chunk_days to work with the new scheme 2026.5. I have two years of forcing data. The log shows:
Rust backend: forcing split into 25 chunks of <= 30 days.
and then fails with RuntimeError: invalid state payload.
I originally encountered this with multiple grids (which is the reason I need chunking), but I also tested a yml file containing a single grid and get the same error.
If I run without specifying chunk_day:
output = sim.run(logging_level=20)
the simulation works fine (for a single grid).
Thanks in advance!
Thanks, @janka. I tested this with a public two-year case split into 25 chunks, and it completed successfully, so the trigger may depend on the configuration or model state. Could you share your YAML configuration and the full traceback, including which chunk fails first? We should not need the complete forcing dataset initially. In the meantime, omitting chunk_day is the safest workaround.
Thanks again, @Janka. I investigated this further and found that there is indeed a checkpoint/restart problem behind the reported error.
I can reproduce the same RuntimeError: invalid state payload failure mechanism when a non-finite model-state value reaches a chunk checkpoint. The current JSON serialisation turns that value into null, and the next chunk rejects it while restoring the state. This is configuration/state dependent, which explains why the public example completed while your case failed.
I have not yet reproduced your complete case without the YAML and forcing data, so I cannot identify which state variable becomes non-finite in your run. However, the checkpoint failure mechanism is real and needs to be fixed.
I also found a separate restart-continuity problem: the elapsed model-time counter is reset at each chunk, so a chunked run can differ numerically from an uninterrupted run even when both complete.
I have opened separate issues for the two problems:
For now, omitting chunk_day remains the safest workaround. If you can share the YAML, full traceback, and first failing chunk, that will let us locate the state variable responsible in your case.
Hi, many thanks for investigating this so thoroughly. I’m glad/sorry to hear that you’ve experienced the same issue 
Omitting chunk_day works for single grids, but my single-grid case was only for testing whether the problem was related to the number of grids (doesn’t seem to be). My main goal is to run a multigrid analysis with several scenarios (hence parameter_values etc. in the traceback). Without chunk_day, my laptop quickly runs out of memory.
I’m attaching my 15-grid .yml, a single-grid .yml containing only grid 1, and the forcing file. Copying the traceback below. Not sure how to identify the first failing chunk.
Note: I created the .yml files using SUEWS Database Prepare - I’ve been working on updating it to SUEWS 2026.5. If you can see any suspicious things in the files, version mismatch is probably the reason.
2026-08-12 13:09:46,877 - SuPy - INFO - Loading config with schema version: 2026.5
2026-08-12 13:09:50,851 - SuPy - INFO - SuPy is validating `df_forcing`...
2026-08-12 13:09:50,891 - SuPy - INFO - All checks for `df_forcing` passed!
2026-08-12 13:09:50,897 - SuPy - INFO - Rust backend: forcing split into 25 chunks of <= 30 days.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[15], line 22
18 exec(f"{ex['key']} = value")
19
20 sim_i.update_forcing(df_forcing)
21
---> 22 output = sim_i.run(chunk_day=30, logging_level=20)
23 # output = sim_i.run(logging_level=20)
24
25 list_outputs.append((parameter_value, output))
File C:\OSGeo4W\apps\Python312\Lib\site-packages\supy\suews_sim.py:575, in SUEWSSimulation.run(self, start_date, end_date, chunk_day, n_jobs, **run_kwargs)
571 # Run simulation via Rust bridge
572 initial_state_json_by_grid = (
573 self._checkpoint.grid_states if self._checkpoint is not None else None
574 )
--> 575 df_output, dict_state_json = run_suews_rust_chunked(
576 config=self._config,
577 df_forcing=df_forcing_slice,
578 chunk_day=chunk_day,
579 serial_mode=serial_mode,
580 max_workers=max_workers,
581 initial_state_json_by_grid=initial_state_json_by_grid,
582 )
...
567 )
568 results.append((list_grid_ids[idx], output_flat, state_json, len_sim))
570 list_df_output = []
RuntimeError: invalid state payload
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...
Thanks in advance for any tips!
kv_grid1.yml (97.6 KB)
kv_2017_data_60.txt (2.9 MB)
kv.yml (1.5 MB)
Thanks, @Janka — these files were very helpful.
I ran the attached forcing with both the single-grid and 15-grid YAML using the released supy 2026.6.5 build and chunk_day=30. The unmodified single-grid case completed all 25 chunks, and a memory-bounded diagnostic of the 15-grid case also crossed all 25 checkpoint boundaries without producing a null checkpoint state.
Therefore, the attached base YAML and forcing are not sufficient on their own to reproduce your particular failure. Your traceback shows that sim_i is modified immediately before the run through the parameter_values / ex / exec(...) loop. Could you please share that complete cell, including the keys and values being assigned? It would also help if you export the effective configuration immediately before the run:
sim_i.config.to_yaml("effective_config.yml")
The YAML files themselves do not contain NaN or infinite values. The raw forcing does contain several missing observations, which the normal forcing loader interpolates before the simulation; these did not trigger the checkpoint failure in my tests.
The two checkpoint fixes are now merged: one preserves NaN state markers across restart, and the other preserves elapsed model time across chunks. They will be included in the next public release. Once we have the effective post-mutation configuration, I can identify the first failing chunk and state variable in your exact scenario.
Hi, thanks again for checking this and sorry for replying so late, I was out of office.
Since you weren’t able to reproduce the error, I looked more closely into my ‘modify parameters’ cell and found the cause of the chunk_day error message. In some scenarios, I modify more than one parameter at a time - as when increasing tree cover I also reduce another surface fraction so that they sum to 1. The code was overly complicated, but it worked fine in previous versions (actually, it still works in all scenarios except when modifying surface fractions and setting tree cover to 0). After changing it to the recently introduced update_config, the chunk_day works fine. I’ll keep testing with more grids and come back if I need more help Thanks again!
1 Like