opthq/cpp-garmin-img-to-osm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

v08

Browse Source
This commit is contained in:
nq
2026-10-14 17:57:55 +00:00
parent fe9e9b593e
commit 3211170ca5
21 changed files with 3529 additions and 635 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+75 -99
View File
@@ -1,127 +1,103 @@
# BGMountains Garmin IMG → OsmAnd OBF (v0.7)
# BGMountains Garmin IMG → OSM → OsmAnd OBF v0.8
**One command builds the C++ IMG converter, emits a reviewed OSM, generates complete custom OsmAnd POI and map type definitions, validates the intermediate source/OSM data, and invokes OsmAndMapCreator to build an OBF.**
**This release fixes the v0.7 icon-visibility blind spot and anonymous POIs.**
Unlike v0.6's optional `build_obf.sh` wrapper, the main script no longer expects you to create any missing XML files. First-time setup requires access to the official OsmAnd resource files and the OsmAndMapCreator Java tool; the script can download both on demand. Java, Python, CMake, libpng, SQLite3 and a C++ compiler must be installed.
- Every one of the **263 validated retained Garmin POI types** has at least one conventional OSM semantic/display tag in `config/poi-overrides-all-types.csv`, in addition to raw `bgm_poi`/`garmin:*` provenance tags.
- **87 codes** occur at least 100 times in the source catalog (409,154 source POIs). **46** use conventional feature categories recognized by stock OsmAnd in principle; **41** use an explicitly marked, synthetic `tourism=information;information=board` **local-OsmAnd display fallback**. Some additional lower-frequency codes also need fallback. This DOES NOT assert that the physical object is an information board.
- Missing original IMG names become stable Bulgarian descriptive names, e.g. `Вода - Каптаж - 1THF1SDNTKJ`, with `bgm:generated_name=yes`. Original nonempty names are preserved.
- `--jobs N` enables parallel extraction, **default one thread**.
- New `tools/audit_icon_coverage.py` verifies source OSM classification coverage, names and produces per-type coordinate report. It **does not** prove OBF index presence or visual display; use an OBF inspector and OsmAnd for that.
- Map-derived icon art and datasets remain governed by BGMountains' applicable copyright conditions. These synthetic POI tags are **not appropriate for uploading unchanged to the real OpenStreetMap database**.
## One-command full Bulgarian map
## Quick build (Ubuntu/Debian)
```bash
```sh
sudo apt update
sudo apt install -y build-essential cmake libpng-dev libsqlite3-dev python3 openjdk-21-jre-headless unzip
python3 scripts/build_bgmountains.py \
--img '/path/to/gmapsupp.img' \
--out '/path/to/BGMountains-build' \
--img /absolute/path/to/gmapsupp.img \
--out ./BGMountains-out \
--download-creator
```
If you **already have** OsmAndMapCreator, use `--creator /path/to/OsmAndMapCreator` in place of `--download-creator`. You can also `export OSMAND_MAPCREATOR=/path/to/OsmAndMapCreator` and omit `--creator`.
`--download-creator` is for the official OsmAndMapCreator, and the pipeline downloads the original official `poi_types.xml` and `rendering_types.xml` the first time as needed. Network access is required for these unless you already supply the files. **This runtime did not have MapCreator, so full OBF compilation was not executed here.**
OsmAndMapCreator is ~150 MB and requires extra disk space and Java RAM. By default, the script downloads it **only when** you request `--download-creator`; upstream OsmAnd `poi_types.xml` and `rendering_types.xml` are downloaded or read from your MapCreator directory automatically, then cached in the output folder. Requires Internet for first use unless you supply existing XML files with `--poi-base FILE --rendering-base FILE`.
Already installed OsmAndMapCreator:
**Multithreading remains disabled by default (`--jobs 1`).** Pass `--jobs 4` explicitly for the IMG parsing stage. It does not control MapCreator's Java processing.
If the Java process runs out of memory:
```bash
python3 scripts/build_bgmountains.py --img gmapsupp.img --out ./BGMountains-build \
--creator ~/OsmAndMapCreator --jobs 4 \
--java-opts '-Xms1G -Xmx12G'
```sh
python3 scripts/build_bgmountains.py --img ./gmapsupp.img --out ./BGMountains-out \
--creator /path/to/OsmAndMapCreator
```
The full-country OSM can be large. Allocate substantial disk space for OSM XML, temporary indexing databases, and OBF output. The script retains intermediate files for auditing.
Opt-in parallelism: add `--jobs 4`. The output directory must be fresh: the converter refuses to overwrite an existing OSM or staging files.
## What is generated
Prepare and validate the OSM without invoking the Java OBF builder:
```text
BGMountains-build/
BGMountains.osm # converted source, Garmin type and name kept
source-pois.csv # source point audit manifest
poi-types.csv # observed type counts and example coordinates
validation.json # source-to-OSM counts/mismatches
mismatches.csv
converter.log
validator.log
upstream/poi_types.xml # official OsmAnd definitions (if downloaded)
upstream/rendering_types.xml
osmand-config/
poi_types.bgmountains.xml # FULL base OsmAnd POI rules + 1 custom type per observed Garmin code
rendering_types.bgmountains.xml # FULL base render encoding + map encoding for each Garmin code
BGMountains-custom.render.xml # OPTIONAL visual style for custom icons in OsmAnd
type-index-summary.json
mapcreator.log
*.obf # full map (created by OsmAndMapCreator)
obf-poi-index.txt # dump from inspector.sh, if available
build-manifest.json
```sh
python3 scripts/build_bgmountains.py --img ./gmapsupp.img --out ./BGMountains-preview \
--prepare-only
```
For the v0.6 reviewed **national catalog**: 263 decoded types, 410,563 POIs, 87 types with **at least 100** occurrences, 86 with Garmin TYP bitmap extraction. Actual new runs produce their own observed counts. The default threshold is 100; use `--threshold 250` to change it.
**IMPORTANT: You must REBUILD the OSM and OBF.** This update cannot modify an already compiled `.obf` or give v0.7 output its new tags in place.
### Three different meanings of 'icon'
## What the configuration files mean
1. **Native OsmAnd map icon.** Verified type receives proper OSM tags (`natural=spring`, `tourism=hotel`, etc.) from `config/poi-overrides-all-types.csv`. Existing OsmAnd styles choose their built-in symbols.
2. **Custom Garmin type in OBF.** Every observed POI receives `garmin:type` and `bgm_poi=type_0xHEX`; the generated POI and map type XML preserve those types as a dedicated category. Map-index objects at high-frequency types are stored from zoom **15** by default, rare types from zoom **17** (adjustable by `--frequent-zoom`, `--rare-zoom`). Native OsmAnd styles may not draw custom types without a matching style.
3. **Actual Garmin TYP bitmap.** Preserved separately as PNG in `data/poi-icons/`; an OBF **does not embed third-party bitmap resources into the app icon set**. To display these exact symbols requires changing OsmAnd application resources, not just editing XML. A different installed OsmAnd render style may refer to already available built-in symbols.
| File | Purpose |
|---|---|
| `config/poi-overrides-all-types.csv` | **Active** 1,871-row config, including 263 retained types with revised tags. `mode=replace` replaces old hardcoded semantics, retaining provenance/name. All non-present legacy type rows remain for historical comparison. |
| `config/poi-overrides-all-types-v07-user-upload.csv` | Exact backup of the CSV you supplied. |
| `config/poi-labels-bg.csv` | New 263-row Bulgarian fallback name prefixes, editable. |
| `config/poi-icon-audit.csv` | All 263 types, count, original tags vs new tags, classification confidence and sample coordinates. |
| `config/osmand-icons.csv` | Optional **custom render style** icon names/visibility; never controls the default `tourism=information` fallback injected through the active overrides. |
| `data/BGMountains-frequent-poi-types-100plus-v08.csv` | All 87 frequent types, with reviewed tags, classification mode and coordinates. |
| `tools/audit_icon_coverage.py` | Independent POI OSM field audit; one row per present type. |
## Assign additional built-in OsmAnd icons yourself
**How to edit a type:** find `code=0x6500` in active overrides. Edit the **tags** column, leaving a `key=value;key=value` list. Change `mode=replace` only if you understand how the C++ built-in classifications merge. Set a Bulgarian text prefix in `config/poi-labels-bg.csv`. Run again from scratch to rebuild OSM then OBF.
Edit `config/osmand-icons.csv`:
### Why 0x6500 is not `amenity=drinking_water`
```csv
code,show_in_custom_style,minzoom,osmand_builtin_icon,...
0x650b,yes,16,information,...
The original legend labels this a **каптаж** (water intake/catchment), not necessarily a drinking-water source. v0.8 records `man_made=water_intake` but adds an explicit *presentation-only* `tourism=information;information=board;bgm:synthetic_icon=yes` for common OsmAnd rendering. The "information board" is NOT a physical claim. The default Bulgarian name prefix is `Вода - Каптаж`. If you want `Вода - Резервоар`, just edit the label CSV; do not misrepresent a small intake as an artificial lake (`natural=water;water=reservoir`).
`0x6602` already has `tourism=viewpoint` and does not use a synthetic board marker. Its failure to appear in a particular existing OBF could be an old build, incomplete POI indexing, or display suppression—only inspecting that actual OBF can distinguish these.
### Stable anonymous names
For an unnamed POI, `name = Bulgarian_label + " - " + FNV1a_base36_11(mapset|type|quantized_lat|quantized_lon)`. The output is deterministic for an identical IMG and coordinates across threads and builds. A very small collision probability remains; the coordinates/mapset/type remain available. These generated names are for an offline map, **not actual surveyed names**.
### Test against your three sample coordinates
| Type | Coordinates | Expected tags | v0.8 generated name |
|---|---|---|---|
| `0x6602` | `42.0346141,24.6528697` | `tourism=viewpoint` | `Панорама - Гледка - 25IVP49H823` |
| `0x6500` | `42.0347643,24.6522689` | `man_made=water_intake;tourism=information;information=board` | `Вода - Каптаж - 1THF1SDNTKJ` |
| `0x6500` | `42.0340991,24.6520114` | Same | `Вода - Каптаж - ED03D455VTZ` |
These examples were directly extracted from the 422MiB IMG into a Plovdiv test OSM and compared to a source manifest. All 57 POIs in that test passed exact semantic-tag checks, with **zero missing and extra**. Serial vs `--jobs 4` POI identities/tags were equal.
## Review before copying `.obf` into OsmAnd
Pipeline artifacts in `--out`:
- `BGMountains.osm` — exact source XML for the OBF builder
- `source-pois.csv`, `poi-types.csv` — extraction manifest and type counts
- `validation.json`, `mismatches.csv` — original-source to OSM audit
- `poi-coverage.csv`, `poi-coverage.json` — export display mapping coverage
- `osmand-config/poi_types.bgmountains.xml`, `rendering_types.bgmountains.xml` — registrations added to **complete upstream XML**
- `osmand-config/BGMountains-custom.render.xml` — optional style; exact Garmin TYP art is NOT embedded into stock OsmAnd
- `*.obf` plus `mapcreator.log`, `obf-poi-index.txt` (only after a successful Java compilation)
The `tourism=information` fallback should give OsmAnd a native searchable POI category in an appropriate compatible build. **Neither an OSM tag nor an indexed OBF proves a symbol will be drawn at every zoom**: the map style, profile, POI filters, icon availability, density/collision and selected offline map still affect visibility. Test at zoom 17–20 with POI layers enabled. Avoid having an old overlapping BGMountains OBF selected at the same time.
To inspect your built file with OsmAndMapCreator:
```sh
/path/to/OsmAndMapCreator/inspector.sh -vpoi -vpoiobjects \
-bbox=24.64,42.05,24.665,42.02 ./BGMountains-out/*.obf \
> ./BGMountains-out/plovdiv-obf-inspector.txt
```
For common unknowns (>=100 occurrences with no verified standard OSM semantic category) the sample config enables a generic **built-in** `information` icon in an optional style. For verified classes, `show_in_custom_style=no` lets the stock OsmAnd map style draw a more meaningful native icon. The file includes one entry for **all 263 source types**, occurrence counts and sample coordinates. Change the `osmand_builtin_icon` to a valid icon ID supported by your installed OsmAnd version and set `show_in_custom_style=yes` to override the default.
Inspect output for the three named POIs. **There is no tested generic parser of OsmAndMapCreator's inspector text in this package**, and we have not verified a full-country OBF built from v0.8 in this runtime. If OSM has the POI but the inspector doesn't, focus on indexing or generated XML; if inspector has it but the screen doesn't, focus on style/POI display/filter/zoom.
After building, install `BGMountains-custom.render.xml` as a custom OsmAnd rendering style in the application's `rendering` directory, then select the corresponding map style. It depends on OsmAnd's `default` style. It is optional and **not automatically installed** by placing the OBF on the device. Its exact icon rendering has not been device-tested in this release; built-in IDs and application versions may differ. Avoid over-rendering at low zoom because 410k points can be slow.
### Local-only licensing warning
## Test a region first
```bash
python3 scripts/build_bgmountains.py --img gmapsupp.img \
--out ./rila-build --bbox 23.22,42.08,23.47,42.29 \
--creator ~/OsmAndMapCreator
```
Alternatively use `--prepare-only` to stop before Java: this produces the OSM plus generated XML and validation reports.
## Rebuild from existing OSM
Only use an OSM generated by **v0.7 or later** containing `bgm_poi` tags:
```bash
python3 scripts/build_bgmountains.py \
--osm ./BGMountains-build/BGMountains.osm \
--catalog ./BGMountains-build/poi-types.csv \
--creator ~/OsmAndMapCreator \
--out ./obf-rebuild
```
Original v0.6 OSM exports **lack** the `bgm_poi` classifier. The script rejects those for this path rather than quietly producing an incomplete custom POI index. To reuse old data, rerun the IMG extraction with v0.7.
## How POIs are edited
- `config/poi-overrides-all-types.csv`: reviewed original IMG type → semantic OSM tags (five-column CSV). You can override/add rules, preserving provenance. This includes the earlier 67 legend/icon matches.
- `config/poi-name-fixes.csv`: coordinate-specific correction of verified names, never guessed universally.
- `config/poi-additions.csv`: manually add personally verified POIs.
- `config/osmand-icons.csv`: optional on-screen **stock OsmAnd** symbol selections for custom raw types.
- `data/BGMountains-unique-poi-types.csv`: counts, names, five real sample coordinates per Garmin hex type.
- `data/poi-icons`: 96 extracted TYP bitmaps for reference, not automatically installed into OsmAnd.
### Validation
The C++/Python roundtrip test checks extracted source POI manifest against OSM coordinates, names and tags. `ctest --test-dir build --output-on-failure` runs 5 bundled tests, including checks of full upstream XML augmentation. On the tested source bbox (23.22,42.08,23.47,42.29) a v0.7 extraction produced 3,986 POIs across 60 types and zero source-versus-OSM mismatches. **These tests do not assert OBF persistence** until MapCreator runs on your machine and `obf-poi-index.txt` is reviewed.
OsmAnd has separate indexed data for map rendering and POI search, and an additional app-side style/icons component. A successful OBF build does not prove every icon will be visible. To debug OBF use `~/OsmAndMapCreator/inspector.sh -vpoi /path/to/map.obf` and compare the categories to `osmand-config/type-index-summary.json`.
## Upstream and redistribution
The repository's C++ converter has its own `LICENSE`; BGMountains-derived SVG/PNG icons, data, examples and Garmin IMG/OBF remain subject to map licensors' conditions. Do not redistribute the full data release or generated OBF as unrestricted ODbL/CC0 without additional rights clearance. The script downloads official OsmAnd XML into a private build directory rather than repackaging those resource files in this source ZIP.
Official references:
- https://osmand.net/docs/technical/map-creation/create-offline-maps-yourself/
- https://github.com/osmandapp/OsmAnd-resources/tree/master/poi
- https://github.com/osmandapp/OsmAnd-resources/tree/master/obf_creation
- https://osmand.net/docs/technical/osmand-file-formats/osmand-rendering-style/
Do not upload the generated synthetic display tags to OpenStreetMap or assert they accurately describe physical information boards. The original IMG, derived icons and extracts can have third-party copyright restrictions. The source-only code path should be distributed separately from map-derived sample assets until rights are confirmed.