The cloud portion of Nidus
The goal of this rework is to make it so I can pass around platform.User instead of a pair of models.Organization and models.User. This is useful for reason I kind of forget now, but it started with working on notifications and ballooned massively from there into refactoring a number of things that were bugging me. This also includes a tiny amount of work on server-side events (SSE). * background stuff lives inside the platform now, which I need for having it push updates through SSE * userfile now lives in the platform, under file, so other platform functions can safely use it * oauth is broken into pieces and inside platform because other stuff was calling it already, but badly. * notifications go into the platform as well |
||
|---|---|---|
| api | ||
| arcgis-go@f5ec5c75c1 | ||
| auth | ||
| cmd/passwordgen | ||
| comms | ||
| config | ||
| db | ||
| debug | ||
| go-geojson2h3@c2ff1a96ab | ||
| h3utils | ||
| html | ||
| http | ||
| label-studio | ||
| llm | ||
| minio | ||
| platform | ||
| rmo | ||
| scss | ||
| stadia | ||
| sync | ||
| tomtom | ||
| tools | ||
| .air.toml | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| .prettierrc | ||
| default.nix | ||
| flake.lock | ||
| flake.nix | ||
| go.mod | ||
| go.sum | ||
| lefthook.yml | ||
| LICENSE | ||
| main.go | ||
| query.go | ||
| README.md | ||
| start-air.sh | ||
| start-flogo.sh | ||
| start-nidus-sync.sh | ||
Nidus Sync
This is the software that powers Nidus Cloud Sync.
Building from source
First, you'll need Nix.
Then:
nix develop
go build .
Building Custom Theme
We're using a customized Bootstrap theme for this site. You'll need to build the SCSS into CSS:
nix develop
sass --style=compressed --trace "$SASS_SRC_DIR/custom.scss":"$CSS_OUTPUT_DIR/bootstrap.css"
Running
You'll need a number of environment variables for configuring things;
- ARCGIS_CLIENT_ID - The client ID for ArcGIS oauth, configured with esri.
- ARCGIS_CLIENT_SECRET - The client secret for ArcGIS oauth, configured with esri.
- BASE_URL - The URL the site is hosted at, used for forming callback URLs. Should be complete like 'https://foo.bar'
- BIND - The address and port to bind to. Use ':9001' for 'any address, port 9001'
- ENVIRONMENT - either 'PRODUCTION' or 'DEVELOPMENT'. It's used to set things like oauth token length.
- MAPBOX_TOKEN - The token to use with mapbox which is important for rendering maps.
- POSTGRES_DSN - The DSN for connecting to the postgres database.
- FIELDSEEKER_SCHEMA_DIRECTORY - The directory to write fieldseeker schema files for debugging.
- USER_FILES_DIRECTORY - The directory for writing uploaded user data files (audio, images).
> BASE_URL=https://sync.nidus.cloud ARCGIS_CLIENT_ID=foo ARCGIS_CLIENT_SECRET=bar POSTGRES_DSN='postgresql://?host=/var/run/postgresql&dbname=nidus-sync' ./nidus-sync
Districts
There's a table containing district information in the database, import.district. It was created with:
psql
CREATE SCHEMA import;
shp2pgsql -s 3857 -c -D -I CA_districts.shp import.district | psql -d nidus-sync
psql
ALTER TABLE import.district ADD COLUMN geom_4326 geometry(MultiPolygon,4326) GENERATED ALWAYS AS (ST_Transform(geom, 4326)) STORED;
Hacking
air
This project uses air for fast compile-and-test loops. You can run it with:
> BASE_URL=https://sync.nidus.cloud ARCGIS_CLIENT_ID=foo ARCGIS_CLIENT_SECRET=bar POSTGRES_DSN='postgresql://?host=/var/run/postgresql&dbname=nidus-sync' air
bob
This uses the bob query framework. You can regenerate the models for bob with:
PSQL_DSN="postgresql://dbname?host=/var/run/postgresql&sslmode=disable" go run github.com/stephenafamo/bob/gen/bobgen-psql@latest"
PSQL_DSN="postgresql://?host=/var/run/postgresql&sslmode=disable&dbname=nidus-sync" go run github.com/stephenafamo/bob/gen/bobgen-psql@latest
This will generate a bunch of files. They're already committed, you only need this if you change the database schema in some way.
goose
This uses goose. You can use the goose command line to check status and make changes
> cd migrations
> GOOSE_DRIVER=postgres GOOSE_DBSTRING="dbname=nidus-sync sslmode=disable" goose status
> GOOSE_DRIVER=postgres GOOSE_DBSTRING="dbname=nidus-sync sslmode=disable" goose down
> GOOSE_DRIVER=postgres GOOSE_DBSTRING="dbname=nidus-sync sslmode=disable" goose up
watchexec
For iterating on styles
watchexec -e *.scss sass scss/custom.scss:html/static/css/bootstrap.css