Skip to content

Fix: don't downgrade map/factor-file provider based only on USA equity data#653

Open
AlexCatarino wants to merge 2 commits into
QuantConnect:masterfrom
AlexCatarino:fix-futures-mapfile-provider-downgrade
Open

Fix: don't downgrade map/factor-file provider based only on USA equity data#653
AlexCatarino wants to merge 2 commits into
QuantConnect:masterfrom
AlexCatarino:fix-futures-mapfile-provider-downgrade

Conversation

@AlexCatarino

Copy link
Copy Markdown
Member

Problem

A CME-futures-only backtest (e.g. a continuous future /NQ) silently produces future.Mapped == None for every bar, selects no contract, makes no trades, and ends with Sequence contains no elements in result analysis — with no error raised.

Root cause is in lean-cli (not LEAN). LeanRunner._handle_data_providers downgrades the global map-file-provider (and factor-file-provider) from the Zip provider to the Disk provider based solely on the state of equity/usa/map_files:

self._force_disk_provider_if_necessary(lean_config, "map-file-provider",
    "...LocalZipMapFileProvider", "...LocalDiskMapFileProvider",
    data_dir / "equity" / "usa" / "map_files")   # gate is USA-equity only

The provider is a single global engine setting applied to every market, and LocalDiskMapFileProvider reads only loose .csv files — it silently ignores map_files_yyyyMMdd.zip archives. Futures map files (e.g. nq.csv) ship only inside that zip. So a futures-only user (fresh future/cme/map_files/*.zip, but no fresh equity/usa data) has the global provider swapped to Disk, and the continuous future never maps.

Fix

Gate the downgrade on every market's auxiliary folder (<type>/<market>/{map_files,factor_files}/) instead of just equity/usa. Only fall back to the Disk provider when there is no recent zip to lose for any market. Futures-only users keep LocalZipMapFileProvider; equity-sample users (loose csv, no zips) still downgrade as before.

Testing

  • Added regression tests in tests/components/docker/test_lean_runner.py: futures-only data keeps the Zip providers; loose-csv-only data downgrades to Disk; all-stale-zip data downgrades to Disk.
  • Verified end-to-end: a CME continuous-future backtest with no USA-equity map files now maps correctly (Mapped: NQ ...) with the stock engine image; previously Mapped: None.
  • Full test_lean_runner.py and test_backtest.py suites pass.

🤖 Generated with Claude Code

AlexCatarino and others added 2 commits June 20, 2026 20:56
…y data

`_handle_data_providers` downgraded the global `map-file-provider` (and
`factor-file-provider`) from the Zip provider to the Disk provider based
solely on the state of `equity/usa/map_files`. The provider is a single
global engine setting and the Disk provider silently ignores
`map_files_*.zip` archives, so a futures-only data folder (whose map
files ship only inside the zip) had its provider swapped out and
continuous futures never mapped (`Mapped: None`) with no error raised.

Gate the downgrade on every market's auxiliary folder instead of just
`equity/usa`: only fall back to the Disk provider when there is no recent
zip to lose for any market. Adds regression tests covering futures-only
data, loose-csv-only data, and stale-zip data.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Initialize newest_zip_date to datetime.min so the "no zip found" case
naturally falls through the >7-days-old downgrade check, removing both
explicit None checks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant