Skip to content

feat: support multiple filters on the same dimension field#22

Open
jpetey75 wants to merge 1 commit into
mainfrom
feat/multiple-filters-same-field
Open

feat: support multiple filters on the same dimension field#22
jpetey75 wants to merge 1 commit into
mainfrom
feat/multiple-filters-same-field

Conversation

@jpetey75

Copy link
Copy Markdown
Contributor

Closes #20

Problem

Applying more than one filter to the same field — e.g. constraining a date dimension to a range — raised:

NotImplementedError: Multiple filters for field <field_name> not implemented yet
.filter(
    (model.dimensions.datetime_day >= date_x) &
    (model.dimensions.datetime_day <= date_y)
)

Change

Removes the self-imposed processed_field_ids guard in CompositeFilter.to_dict(). That check was a placeholder ("not implemented yet"), not an API constraint — the Lightdash API accepts multiple rules per field, using the same {target, operator, values} shape already used for single dimension filters. We simply emit one entry per rule under filters.dimensions.

A date range now serializes as:

{"dimensions": {"and": [
  {"target": {"fieldId": "orders_datetime_day"}, "operator": "greaterThanOrEqual", "values": ["2026-01-01"]},
  {"target": {"fieldId": "orders_datetime_day"}, "operator": "lessThanOrEqual",    "values": ["2026-01-31"]}
]}}

Tests

Adds TestSameFieldFilters covering same-field AND/OR combinations, 3+ rules on one field, and the .filter().filter() chain path. All 54 filter/query-builder unit tests pass.

Scope

This is the first of three related issues (#19 row limit, #21 table-calculation filters) — split out on its own because it's a low-risk, self-contained removal of a client-side guard. #19 and #21 will follow in separate PRs.

🤖 Generated with Claude Code

Remove the self-imposed `NotImplementedError` guard in
`CompositeFilter.to_dict()` that rejected more than one filter rule
targeting the same field. The Lightdash API accepts multiple rules per
field (the same shape already used for single dimension filters), so a
range like `(dim >= start) & (dim <= end)` now serializes both rules
under `filters.dimensions`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jpetey75

Copy link
Copy Markdown
Contributor Author

Verified that removing the duplicate-field guard is safe — this was a self-imposed SDK limitation (note the original "not implemented yet" message), not an API constraint.

Live test against the API: ran a query with two filter rules on the same field — pages_date_day >= 2025-01-01 AND pages_date_day <= 2025-03-31, grouped under and (exactly the payload shape this PR produces). It succeeded and returned correctly range-bounded data:

Month Page views
2025-01 3,881,394
2025-02 5,564,346
2025-03 10,152,069

Three months, nothing outside the range → both same-field rules were applied.

Two independent lines of evidence:

  1. The live query above — the API accepts the payload and applies both conditions.
  2. The API's filter model: filters.dimensions is a flat array of {fieldId, operator, values} rules with no uniqueness constraint on fieldId. The array structure inherently permits multiple rules per field; the old guard was artificially narrower than the contract.

Also confirmed there's no per-rule id requirement for ad-hoc queries — single filters already worked without one, and multiple now do too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support multiple filters on the same dimension field

1 participant