fix: filter anonymous reports from related records in communication view #150
eliribble
ned
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "issue-149-related-records-anon-filter"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The SourceReportID path in CommunicationRelatedRecords was doing two unbounded org-wide queries (by reporter name and by address ID) and appending every result without any filtering. For address ID 68 on org 1 this returned 29 reports belonging to 29 distinct contacts — the majority sharing nothing more than a physical location.
Changes:
Address completeness check (new): Before using an address for related-report matching, check that the address record is complete. An address is complete if it has been geocoded (Gid populated) or if all core structural fields (number, street, locality, region, postal_code) are filled.
Phone filter: When collecting matching reports from either the name or address path, skip reports where reporter_phone is empty. Phone is a strong identity signal; empty phone = we cannot establish the submitter is the same person.
Deduplication: Matching reports now use a seen map keyed by public ID. If the same report matches both the name and address paths, it appears only once in the result.
No change to the name path — it already guards against empty reporter names before querying, which per Eli is sufficient.
Fixes #149
The SourceReportID path in CommunicationRelatedRecords was doing two unbounded org-wide queries (by reporter name and by address ID) and appending every result without any filtering. For address ID 68 this returned 29 reports belonging to 29 distinct contacts, the majority of which shared nothing more than a common physical location. Eli's rules for when to treat reports as truly related: - Name matching: already guarded by a reporter_name != '' check - Phone matching: skip matching reports that have an empty phone number, since phone is a strong identity signal and empty phone means we cannot establish identity - Address matching: only use address for relating records when the address record is complete — either geocoded (Gid is set) or all core structural fields (number, street, locality, region, postal_code) are populated Additionally, deduplicate results so a report matching both the name and address paths doesn't appear twice. Affects: issue #149Thanks, solid fix. Could you write up the body of an email response to our customer, Alicia, explaining what we found at a high level and the changes to the rules for matching reports? I want it to be clear to her what she should expect to see change without burdening her with understanding all the details, and make her aware that some things will be different. Also, as usual, if there's anything she sees that seems wrong to let us know.
Go ahead and post the explanation on this issue, I'll review and adjust and send it to her when we've deployed the changes.
oops, sorry, I thought this comment was on issue #149 - post there
Here is a draft of the email to Alicia explaining the changes in plain language:
Hi Alicia,
We tracked down the issue with the "Related Reports" section you saw on communication #546. What was happening is the system was pulling in every report tied to the same physical address — even if the person living there had nothing to do with your case. For that particular address, there were 29 separate reports from 29 different people over the years, all getting lumped together.
We have made three adjustments to how related reports are matched moving forward:
Address matching is now stricter. We only use an address to find related reports if we have enough detail to be sure about the location — for example, if the address has been geocoded or has a full street address with city and ZIP. Partial or vague addresses won't pull in unrelated reports anymore.
No more matching on empty phone numbers. If a report has no contact phone on file, we skip it when looking for related records. A phone number is a strong way to know two reports came from the same person, so leaving it empty tells us we can't make that connection.
Duplicate prevention. If the same report happens to match through more than one search path, it now only appears once in the results.
In practice, this should mean the "Related Reports" section shows fewer, more meaningful matches — reports that are genuinely connected to the people or location involved in your communication, not a grab-bag of everyone who has ever been associated with an address.
As always, if you see anything that looks off after these changes go in, please let us know and we will take a look.
Best,
Eli
Let me know if you want any adjustments to the tone or content before you send it.