All updates
QABackend

Grid Column Filters: Distinct "Null" Option, Count-Column Filtering Fix, and Rules Lookup Reliability

PR #78aisupport-datagainSep 4, 2025 · 13:35 UTC
QASep 4, 2025

Executive summary

A batch promotion to QA that improves how grid column filter dropdowns behave. Filters can now offer a distinct "Null" (empty) option separate from 0, count columns such as document and mail counts now populate their filter values correctly, and a rules-engine caching defect that could break lookups was fixed.

Why this was needed

Several grid filter dropdowns were not working as expected. NULL/empty values were silently dropped, so users could not filter for records that simply had no value (and an empty cell looked the same as a 0). Count columns (e.g. document count, mail count) failed to list any filter values because the configured column key (document_count) did not match the key actually present in the data payload (total_documents). Separately, the rules service was calling the cache manager with arguments in the wrong order, which could cause folder/label/priority/status/tag lookups to misbehave.

Client / user impact

  • Users can now explicitly filter grid columns for empty/missing values via a distinct "Null" choice, clearly separated from numeric 0.
  • Filter dropdowns for count columns (document count, mail count) now show real values instead of appearing empty.
  • Rule configuration screens that resolve folders, labels, priorities, statuses, and tags are more reliable, and rule action values can carry both legacy text and enriched id/name details.
  • No user-facing UI redesign; these are correctness improvements to existing filtering and rules behavior.

Technical scope

Backend (FastAPI) changes, promoted as a Dev -> QA batch ("Batch Document Count"):

  • column_values_service.py: stop filtering out NULLs when building filter values; track explicit nulls and append None to the result so the UI can render a distinct "Null" option; sort with null first for both numeric and non-numeric types; adjust has_more to account for the null entry.
  • Added a _synonym_keys helper mapping equivalent column keys: document_count<->total_documents, mail_count<->total_mails, status<->status_name; item lookups now try the requested key then its synonyms.
  • response_flattener.py: extract_value_by_column_key now falls back to the same synonym keys when config and payload key names differ.
  • rules_service.py: fixed cache_manager.get/set call signatures (key/namespace order, expire= -> ttl=) across folder, label, priority, status, and tag lookups.
  • rule_schemas.py: widened ActionValueResponse.value to accept legacy str/list[str] plus enriched id/name dict forms.
  • _format_values updated to preserve None instead of discarding it; doc comment in grid_columns.py updated to reflect null inclusion.

Risk & mitigation

Low to moderate. The largest behavioral change is that filter value lists now include NULL, which alters dropdown contents and sort order (null-first); the frontend must render None as a "Null" label rather than blank. The synonym key fallback is additive and only triggers when the primary key is absent, but could surface values for previously-empty count filters. The rules cache fix corrects a real argument-order bug; the main risk is exercising previously dead/broken cache paths. Mitigation: QA verification on QA before any production promotion.

QA validation focus

  • Open grid filter dropdowns and confirm a distinct "Null" option appears for columns that have empty values, and that it is separate from 0.
  • Verify count-column filters (document count, mail count) now list values instead of being empty; confirm sorting (null first, then ascending/descending) is correct.
  • Confirm search-within-filter still works and respects the row limit/has_more flag including the null entry.
  • Exercise rule configuration: verify folder, label, priority, status, and tag lookups load correctly and are cached (no errors on repeat loads).
  • Confirm rule action values render for both simple string/list values and enriched id/name objects.