From fd1d93972f1a0a4d88f0e5a752075f5f6db774ba Mon Sep 17 00:00:00 2001 From: nq Date: Thu, 15 Oct 2026 23:53:12 +0300 Subject: [PATCH] v0.91 --- README_V07.md | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++ README_V09.md | 60 ++++++++++++++++++++++++ 2 files changed, 187 insertions(+) create mode 100644 README_V07.md create mode 100644 README_V09.md diff --git a/README_V07.md b/README_V07.md new file mode 100644 index 0000000..62705c5 --- /dev/null +++ b/README_V07.md @@ -0,0 +1,127 @@ +# BGMountains Garmin IMG → OsmAnd OBF (v0.7) + +**One command builds the C++ IMG converter, emits a reviewed OSM, generates complete custom OsmAnd POI and map type definitions, validates the intermediate source/OSM data, and invokes OsmAndMapCreator to build an OBF.** + +Unlike v0.6's optional `build_obf.sh` wrapper, the main script no longer expects you to create any missing XML files. First-time setup requires access to the official OsmAnd resource files and the OsmAndMapCreator Java tool; the script can download both on demand. Java, Python, CMake, libpng, SQLite3 and a C++ compiler must be installed. + +## One-command full Bulgarian map + +```bash +sudo apt update +sudo apt install -y build-essential cmake libpng-dev libsqlite3-dev python3 openjdk-21-jre-headless unzip + +python3 scripts/build_bgmountains.py \ + --img '/path/to/gmapsupp.img' \ + --out '/path/to/BGMountains-build' \ + --download-creator +``` + +If you **already have** OsmAndMapCreator, use `--creator /path/to/OsmAndMapCreator` in place of `--download-creator`. You can also `export OSMAND_MAPCREATOR=/path/to/OsmAndMapCreator` and omit `--creator`. + +OsmAndMapCreator is ~150 MB and requires extra disk space and Java RAM. By default, the script downloads it **only when** you request `--download-creator`; upstream OsmAnd `poi_types.xml` and `rendering_types.xml` are downloaded or read from your MapCreator directory automatically, then cached in the output folder. Requires Internet for first use unless you supply existing XML files with `--poi-base FILE --rendering-base FILE`. + +**Multithreading remains disabled by default (`--jobs 1`).** Pass `--jobs 4` explicitly for the IMG parsing stage. It does not control MapCreator's Java processing. + +If the Java process runs out of memory: + +```bash +python3 scripts/build_bgmountains.py --img gmapsupp.img --out ./BGMountains-build \ + --creator ~/OsmAndMapCreator --jobs 4 \ + --java-opts '-Xms1G -Xmx12G' +``` + +The full-country OSM can be large. Allocate substantial disk space for OSM XML, temporary indexing databases, and OBF output. The script retains intermediate files for auditing. + +## What is generated + +```text +BGMountains-build/ + BGMountains.osm # converted source, Garmin type and name kept + source-pois.csv # source point audit manifest + poi-types.csv # observed type counts and example coordinates + validation.json # source-to-OSM counts/mismatches + mismatches.csv + converter.log + validator.log + upstream/poi_types.xml # official OsmAnd definitions (if downloaded) + upstream/rendering_types.xml + osmand-config/ + poi_types.bgmountains.xml # FULL base OsmAnd POI rules + 1 custom type per observed Garmin code + rendering_types.bgmountains.xml # FULL base render encoding + map encoding for each Garmin code + BGMountains-custom.render.xml # OPTIONAL visual style for custom icons in OsmAnd + type-index-summary.json + mapcreator.log + *.obf # full map (created by OsmAndMapCreator) + obf-poi-index.txt # dump from inspector.sh, if available + build-manifest.json +``` + +For the v0.6 reviewed **national catalog**: 263 decoded types, 410,563 POIs, 87 types with **at least 100** occurrences, 86 with Garmin TYP bitmap extraction. Actual new runs produce their own observed counts. The default threshold is 100; use `--threshold 250` to change it. + +### Three different meanings of 'icon' + +1. **Native OsmAnd map icon.** Verified type receives proper OSM tags (`natural=spring`, `tourism=hotel`, etc.) from `config/poi-overrides-all-types.csv`. Existing OsmAnd styles choose their built-in symbols. +2. **Custom Garmin type in OBF.** Every observed POI receives `garmin:type` and `bgm_poi=type_0xHEX`; the generated POI and map type XML preserve those types as a dedicated category. Map-index objects at high-frequency types are stored from zoom **15** by default, rare types from zoom **17** (adjustable by `--frequent-zoom`, `--rare-zoom`). Native OsmAnd styles may not draw custom types without a matching style. +3. **Actual Garmin TYP bitmap.** Preserved separately as PNG in `data/poi-icons/`; an OBF **does not embed third-party bitmap resources into the app icon set**. To display these exact symbols requires changing OsmAnd application resources, not just editing XML. A different installed OsmAnd render style may refer to already available built-in symbols. + +## Assign additional built-in OsmAnd icons yourself + +Edit `config/osmand-icons.csv`: + +```csv +code,show_in_custom_style,minzoom,osmand_builtin_icon,... +0x650b,yes,16,information,... +``` + +For common unknowns (>=100 occurrences with no verified standard OSM semantic category) the sample config enables a generic **built-in** `information` icon in an optional style. For verified classes, `show_in_custom_style=no` lets the stock OsmAnd map style draw a more meaningful native icon. The file includes one entry for **all 263 source types**, occurrence counts and sample coordinates. Change the `osmand_builtin_icon` to a valid icon ID supported by your installed OsmAnd version and set `show_in_custom_style=yes` to override the default. + +After building, install `BGMountains-custom.render.xml` as a custom OsmAnd rendering style in the application's `rendering` directory, then select the corresponding map style. It depends on OsmAnd's `default` style. It is optional and **not automatically installed** by placing the OBF on the device. Its exact icon rendering has not been device-tested in this release; built-in IDs and application versions may differ. Avoid over-rendering at low zoom because 410k points can be slow. + +## Test a region first + +```bash +python3 scripts/build_bgmountains.py --img gmapsupp.img \ + --out ./rila-build --bbox 23.22,42.08,23.47,42.29 \ + --creator ~/OsmAndMapCreator +``` + +Alternatively use `--prepare-only` to stop before Java: this produces the OSM plus generated XML and validation reports. + +## Rebuild from existing OSM + +Only use an OSM generated by **v0.7 or later** containing `bgm_poi` tags: + +```bash +python3 scripts/build_bgmountains.py \ + --osm ./BGMountains-build/BGMountains.osm \ + --catalog ./BGMountains-build/poi-types.csv \ + --creator ~/OsmAndMapCreator \ + --out ./obf-rebuild +``` + +Original v0.6 OSM exports **lack** the `bgm_poi` classifier. The script rejects those for this path rather than quietly producing an incomplete custom POI index. To reuse old data, rerun the IMG extraction with v0.7. + +## How POIs are edited + +- `config/poi-overrides-all-types.csv`: reviewed original IMG type → semantic OSM tags (five-column CSV). You can override/add rules, preserving provenance. This includes the earlier 67 legend/icon matches. +- `config/poi-name-fixes.csv`: coordinate-specific correction of verified names, never guessed universally. +- `config/poi-additions.csv`: manually add personally verified POIs. +- `config/osmand-icons.csv`: optional on-screen **stock OsmAnd** symbol selections for custom raw types. +- `data/BGMountains-unique-poi-types.csv`: counts, names, five real sample coordinates per Garmin hex type. +- `data/poi-icons`: 96 extracted TYP bitmaps for reference, not automatically installed into OsmAnd. + +### Validation + +The C++/Python roundtrip test checks extracted source POI manifest against OSM coordinates, names and tags. `ctest --test-dir build --output-on-failure` runs 5 bundled tests, including checks of full upstream XML augmentation. On the tested source bbox (23.22,42.08,23.47,42.29) a v0.7 extraction produced 3,986 POIs across 60 types and zero source-versus-OSM mismatches. **These tests do not assert OBF persistence** until MapCreator runs on your machine and `obf-poi-index.txt` is reviewed. + +OsmAnd has separate indexed data for map rendering and POI search, and an additional app-side style/icons component. A successful OBF build does not prove every icon will be visible. To debug OBF use `~/OsmAndMapCreator/inspector.sh -vpoi /path/to/map.obf` and compare the categories to `osmand-config/type-index-summary.json`. + +## Upstream and redistribution + +The repository's C++ converter has its own `LICENSE`; BGMountains-derived SVG/PNG icons, data, examples and Garmin IMG/OBF remain subject to map licensors' conditions. Do not redistribute the full data release or generated OBF as unrestricted ODbL/CC0 without additional rights clearance. The script downloads official OsmAnd XML into a private build directory rather than repackaging those resource files in this source ZIP. + +Official references: +- https://osmand.net/docs/technical/map-creation/create-offline-maps-yourself/ +- https://github.com/osmandapp/OsmAnd-resources/tree/master/poi +- https://github.com/osmandapp/OsmAnd-resources/tree/master/obf_creation +- https://osmand.net/docs/technical/osmand-file-formats/osmand-rendering-style/ diff --git a/README_V09.md b/README_V09.md new file mode 100644 index 0000000..bcbd4bb --- /dev/null +++ b/README_V09.md @@ -0,0 +1,60 @@ +# BGMountains v0.9: meaningful POIs, distinct icons and Garmin TYP route colours + +## Why v0.8 displayed 'Marina', information boards and orange anchors + +The old dataset used unrelated standard tags to force OsmAnd to index symbols. In v0.9 the default `poi-overrides-all-types.csv` no longer injects synthetic `tourism=information;information=board` for arbitrary places. The C++ area type 0x09 was also incorrectly tagged `leisure=marina`, although the BGMountains legend identifies it as shopping area. It is now `landuse=retail`. We found 0x650b winter waymarks clustering on the Vihren route (not marinas). The output is not for public OpenStreetMap imports without source license clearance. + +## Build from original IMG to OBF (Linux) + +```bash +sudo apt install -y build-essential cmake libpng-dev libsqlite3-dev python3 openjdk-21-jre-headless unzip +python3 scripts/build_bgmountains.py \ + --img '/path/to/gmapsupp.img' \ + --out './BGMountains-output' --download-creator +``` + +Or use an already installed OsmAndMapCreator: + +```bash +python3 scripts/build_bgmountains.py \ + --img '/path/to/gmapsupp.img' --creator /path/to/OsmAndMapCreator \ + --out ./BGMountains-output +``` + +CLI uses **1 worker by default**, add `--jobs 4` to opt in. + +Important: the MapCreator utility is not bundled (large official upstream binary); script downloads it if you specify `--download-creator`. The online upstream OsmAnd `poi_types.xml` and `rendering_types.xml` are fetched too. To work offline provide `--poi-base ... --rendering-base ... --creator ... --offline`. + +Generated files: +- `BGMountains.osm` geographic source, custom `bgm_poi` tags, raw Garmin IDs and hashes +- `osmand-config/poi_types.bgmountains.xml`, `rendering_types.bgmountains.xml` for indexing +- `osmand-config/BGMountains-outdoor.render.xml` **MUST BE INSTALLED/SELECTED IN OSMAND** for custom icons and trail colors +- `source-pois.csv`, `poi-types.csv`, `validation.json`, `poi-coverage.csv` for checks +- `.obf`, `mapcreator.log`, `obf-poi-index.txt` **only if MapCreator completed successfully** + +### Install on your phone + +1. Copy the output `.obf` to the OsmAnd offline maps directory or import through OsmAnd. +2. Import the `.render.xml` style in OsmAnd. On Android: *Menu → Maps & Resources → Local → Rendering styles → Import*, or *Settings → Import from file*. Some versions require placing the style in the app's `rendering/` directory and restarting. +3. Select `BGMountains outdoor v0.9` in *Menu → Configure map → Map rendering → Map style*. +4. Test at zoom 16-18. Very dense points such as winter waymarks and reservoir/catchment markers intentionally begin at zoom 16-17 to prevent the map from being solid with symbols. +5. For search: OsmAnd's POI database/index and on-screen symbols are different. Custom `bgm_poi` entries are added to the build's POI index, but custom POI *categories* and 'Nearby POI' behavior may require app-compatible POI definitions. Search by generated fallback name if Nearby doesn't expose them. We cannot guarantee this until the final OBF is inspected on the device. +6. Remove the older test OBF before loading this one, otherwise the app can display overlapping categories/data. + +### How to customise + +- `config/poi-overrides-all-types.csv`: **meaning**, stable source Garmin hex → OSM tags. **Do not use false `leisure=marina` or fake boards for icons.** +- `config/osmand-icons.csv`: **appearance**, code → built-in OsmAnd icon and minzoom. Some obscure icon IDs may vary between app builds: validate with your installed version. +- `config/trail-styles.csv`: **hiking overlay appearance**, extended Garmin line type → color/zoom/width. Colors recovered from TYP; 50 line types audited. Exact Garmin pixel/band patterns aren't reproducible from plain OsmAnd XML alone; the extracted original source TYP remains in the IMG. +- `config/poi-labels-bg.csv`: fallback **Bulgarian prefix + deterministic hash** for unnamed POIs; valid existing names untouched. +- `config/poi-name-fixes.csv`: coordinate- and type-specific manual corrections, no fuzzy changes to location names. +- `data/BGMountains-v09-POI-icon-crosswalk.csv`: verified legend, all 263 types, default style icon, counted occurrences and sample location. + +### Known limitations + +- IMG → OSM regional export fidelity validated, not mobile screen rendering. +- MapCreator couldn't run in the author's offline environment; a new OBF was NOT generated or inspected in these tests. +- Not every Garmin TYP icon has a built-in OsmAnd equivalent. The native application cannot load arbitrary TYP PNGs from an ordinary OBF/style; exact icon artwork requires a custom OsmAnd application build. +- Road/trail colored overlays are not necessarily independently routable trail geometry. **v0.9 no longer creates false routable paths from the decorative extended overlays.** Underlying real road/path features retain their physical geometry. +- Many obscure Garmin types still share the generic built-in info icon, not because they mean 'information board', but because their semantics and replacement art are unverified. Users can edit `config/osmand-icons.csv`; filenames in OsmAnd icon resources can differ between app versions. +- Use this dataset for private testing pending rights clearance from BGMountains and third-party sources.