Commit graph

58 commits

Author SHA1 Message Date
6945b9f9ed
Drop to a single worker in the geocode pool
I'm sharing transactions incorrectly, and until I fix that I need
correctness, not speed.
2026-04-16 23:50:19 +00:00
c0935c848b
Default required fields to empty strings
So the insertion doesn't fail
2026-04-16 22:32:48 +00:00
f490e4a1a4
Avoid duplicate rownumber calculation
We have the row number saved on the pool object itself, which is how we
gather errors against rows.
2026-04-16 04:20:28 +00:00
b2d8e3ba27
Move address list func to types so it can be shared with csv
And stop double-geocoding all the rows.
2026-04-16 03:06:18 +00:00
82dd5e8683
More debugging for CSV import 2026-04-16 02:46:55 +00:00
0e165b57d0
Default header to tag type 2026-04-16 00:18:42 +00:00
dfe7d3650f
Default header type to unknown
This is a subtle bug from the zero value of a header enum that's causing
overwriting in pool uploads
2026-04-16 00:14:35 +00:00
ac27c60e0c
Save address IDs when doing pool geocoding 2026-04-15 20:29:42 +00:00
6a8ae6d81a
Exit the geocode job if we hit an error 2026-04-15 19:31:55 +00:00
87c802fa90
Fix relationship for looking up whether the pool is in the district 2026-04-15 19:31:32 +00:00
b08582224a
Add missing required state header
I was incorrectly mapping "city" to "region" previously. A 'region'
actually is closer to a state. We need a locality, which is closer to a
city.
2026-04-15 19:30:50 +00:00
66d35428fa
Add error display to file upload 2026-04-15 19:02:25 +00:00
cadf6afb5f
Use embedded address location rather than external location on geocode 2026-04-14 01:42:53 +00:00
730f40956f
Store addresses on every geocode 2026-04-10 22:32:40 +00:00
786a6c16a3
Fix up upload by ID
Show the street number as well as the rest of the address, emit an event
when the upload is processed, actually check if pools are existing, etc.
2026-03-19 15:31:04 +00:00
97c9269215
Update file status to committed when commit completes 2026-03-19 05:53:43 +00:00
434746aa99
Allow the catch-all district to do uploads 2026-03-19 03:25:36 +00:00
f2ea1367e2
Allow the transaction to commit on failure in CSV processing 2026-03-19 03:19:17 +00:00
1e071d5ce5
Overhaul publicreport storage layer, create unified tables
This is a huge change. I was getting really sick of the split between
nuisance/water tables when more than half of the data they store is
common. I finally bit off the big work of switching it all.

This creates a single unified table, publicreport.report and copies the
existing report data into it. It also ports existing data from the
original tables into the new table.

Along with all of this I also overhauled the system for handling
asynchronous work to use a LISTEN/NOTIFY connection from the database
and a single cache table to avoid ever losing work.
2026-03-18 15:36:20 +00:00
2538638c9d
Create generic backend process, fix background interdependencies
This refactor was born out of the inter-dependency cycles developing
between the "background" module and just about every other module which
was caused by the background module becoming a dependency of every
module that needed to background work and the fact that the background
module was also supposedly responsible for the logic for processing
those tasks.

Instead the "background" module is now very, very shallow and relies
entirely on the Postgres NOTIFY logic for triggering jobs. There's a new
table, `job` which holds just a type and single row ID.

All told, this means that jobs can be added to the queue as part of the
API-level or platform-level transaction, ensuring atomicity, and
processing coordination is handled by the platform module, which can
depend on anything.
2026-03-16 19:52:29 +00:00
e2af49a323
Make lead creation and invalidation for public reports work
The only thing wrong at this point that I can tell is that address
aren't being correctly populated when I reverse geocode.
2026-03-14 01:14:30 +00:00
44c4f17f32
Massive rework of platform layer user/organization
The goal of this rework is to make it so I can pass around platform.User
instead of a pair of models.Organization and models.User. This is useful
for reason I kind of forget now, but it started with working on
notifications and ballooned massively from there into refactoring a
number of things that were bugging me.

This also includes a tiny amount of work on server-side events (SSE).

 * background stuff lives inside the platform now, which I need for
   having it push updates through SSE
 * userfile now lives in the platform, under file, so other platform
   functions can safely use it
 * oauth is broken into pieces and inside platform because other stuff
   was calling it already, but badly.
 * notifications go into the platform as well
2026-03-12 23:49:16 +00:00
182fd7d229
Fix quoting in insertFlyover
I didn't know I could do this when I wrote it.
2026-03-11 22:54:22 +00:00
e932c2c473
Rework publicreport addressing
This adds the ability to link a proper address in the database to the
report and harmonizes the field names with the address table. It also
migrates away from mapbox entirely.

And I fixed the "pool" naming for the publicreports, which are supposed
to be the more generic 'water'.
2026-03-09 18:02:22 +00:00
4494bd97cf
Rename address.geom to address.location
It's a better name.
2026-03-06 19:46:41 +00:00
3bc5b1c945
Add missing organization links in pool commit logic 2026-03-06 19:00:47 +00:00
527e82031e
Remove a bunch of generated bob, add feature and review tasks 2026-03-06 18:56:30 +00:00
1a22b9233d
Get much more data about signals to send to the planning dash 2026-03-05 03:17:45 +00:00
78a35e5d1f
Make parcels attached to addresses optional 2026-03-05 02:30:12 +00:00
31c6bf3a64
Make sure to commit the transaction when committing the CSV 2026-03-05 02:20:45 +00:00
60344e3c30
Relate compliance report requests through leads 2026-03-05 01:22:21 +00:00
9c3d2ba3df
Remove organization from fileupload.pool table, fix in district logic 2026-03-04 20:59:57 +00:00
8e00d3e04b
Fix setting the file to "parsed" when starting parse job 2026-03-04 19:27:33 +00:00
daa8cb1748
Push geocoding down a layer
This makes it possible to always save address information from our
geocoder.
2026-03-04 18:29:52 +00:00
6a5a59f8b8
Merge aerial flyover and pool CSV row datatypes
They are extremely similar, having both was just extra work.
2026-03-04 14:52:34 +00:00
ff2ec0ad14
Split out ability to upload flyover data from pool uploads
Tons of changes here, all in the name of quickly getting to where I can
create test compliance letters.
2026-03-02 18:49:02 +00:00
2bb4a134b2
Add centroid information when geocoding
I would use the boundary rect, but I'm getting a 500-level error from
stadia maps
2026-02-25 16:08:32 +00:00
8feabbc489
Add bulk geocoding by goroutine worker
This is nearly as fast, and doesn't require the corp-only license from
stadia map which is 10x the cost.
2026-02-25 14:57:28 +00:00
4e8cb0568e
Translate "empty" as "dry" in pool conditions 2026-02-24 20:34:46 +00:00
efbef12080
Only show CSV as parse after geocoding. 2026-02-24 20:34:32 +00:00
6180860ac0
Fix update of file upload statistics 2026-02-24 20:07:39 +00:00
c8f5408f27
Add link from root upload page to upload pool detail page. 2026-02-24 20:02:44 +00:00
a65f1e0776
Fix up attaching errors to rows 2026-02-16 17:59:18 +00:00
da7a687499
Default all new pools to being new 2026-02-16 16:49:24 +00:00
ef569aef18
Save tags on pool rows, show errors in summary table 2026-02-16 16:38:04 +00:00
f859e372c6
Fill in correct data about the district 2026-02-16 15:26:41 +00:00
ebc329fc5e
Actually set geom and h3cell for uploaded pools 2026-02-14 16:49:54 +00:00
0659b8993d
Bulk ignore empty CSV columns 2026-02-14 15:42:59 +00:00
5d0d75ebb1
Add initial integration with stadia maps for bulk geocoding 2026-02-14 15:41:38 +00:00
2bc0e18b9e
Actually finish uploaded files 2026-02-14 05:40:27 +00:00