v05

CMakeLists.txt

@@ -1,13 +1,14 @@
 cmake_minimum_required(VERSION 3.16)
-project(garmin2osmand VERSION 0.2.0 LANGUAGES CXX)
+project(garmin2osmand VERSION 0.5.0 LANGUAGES CXX)
 set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
 find_package(PNG REQUIRED)
 find_package(SQLite3 REQUIRED)
+find_package(Threads REQUIRED)
 add_executable(garmin2osmand src/main.cpp)
 add_executable(garmin2osm src/vector.cpp)
-target_link_libraries(garmin2osm PRIVATE PNG::PNG SQLite::SQLite3)
-target_link_libraries(garmin2osmand PRIVATE PNG::PNG SQLite::SQLite3)
+target_link_libraries(garmin2osm PRIVATE PNG::PNG SQLite::SQLite3 Threads::Threads)
+target_link_libraries(garmin2osmand PRIVATE PNG::PNG SQLite::SQLite3 Threads::Threads)
 if(MSVC)
  target_compile_options(garmin2osmand PRIVATE /W4 /EHsc)
 else()

README.md

@@ -0,0 +1,155 @@
+# BGMountains Garmin IMG → OSM → OBF: C++ converter + POI audit toolkit (v0.5)
+
+The source code, PNG icons, full-country inventory, editable 1,871-row POI configuration, source manifest, and comparison tools are included in this archive.
+
+**Important:** The converter source is GPL-3.0-or-later. The original BGMountains map data and any extracted/converted geographic data have **separate** licenses. There is no blanket permission to merge this data into openstreetmap.org. No IMG or OBF binaries are bundled.
+
+## Build
+
+```sh
+sudo apt-get install -y cmake g++ make libpng-dev libsqlite3-dev python3
+cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
+cmake --build build -j "$(nproc)"
+ctest --test-dir build --output-on-failure
+```
+
+## Bundled resources
+
+| File | Purpose |
+|---|---|
+| `data/BGMountains-unique-poi-types.csv` | **All 1,871 observed POI hex types**, counts, named counts, current default classification, icon availability and five source coordinates/names/mapset IDs per hex type |
+| `config/poi-overrides-all-types.csv` | **Editable configurator**: one row per hex, with count, default and an example embedded in the notes. Rules initially have empty tags, therefore do not change output |
+| `config/poi-overrides.csv` | Minimal template with explanation and an intentionally commented example |
+| `data/BGMountains-source-pois.csv.gz` | Complete extracted inventory (416,165 source records) for audit/provenance comparisons; *this is map-derived data* |
+| `data/poi-icons/*.png` | 96 successfully decoded Garmin TYP icons indexed by hex code; facilitates visual comparison with GPXSee |
+| `tools/validate_pois.py` | Disk-backed, streaming comparison of source manifest against OSM POIs; optional semantic override assertions; OsmAnd inspector wrapper |
+| `tests/demo.img` | Synthetic Garmin fixture suitable for reproducing baseline smoke tests |
+
+## Produce a countrywide OSM export + inventory
+
+Default execution is **serial** (`--jobs 1`, even if not supplied). Set `--jobs 4` or another integer 1–64 to opt into parallel mapset processing. Parallel processing uses independent SQLite stages; all OSM nodes are emitted before ways and node refs are rebased correctly.
+
+```sh
+./build/garmin2osm \
+  --input '/path/to/gmapsupp.img' \
+  --output Bulgaria.osm \
+  --poi-overrides config/poi-overrides-all-types.csv \
+  --poi-types-out unique-poi-types.csv \
+  --poi-manifest source-pois.csv \
+  --poi-examples 5 \
+  --jobs 4
+```
+
+For a regional check before rebuilding the whole country:
+
+```sh
+./build/garmin2osm --input '/path/to/gmapsupp.img' \
+  --bbox 23.28,42.11,23.47,42.27 --output Rila.osm \
+  --poi-overrides config/poi-overrides-all-types.csv \
+  --poi-types-out Rila-types.csv --poi-manifest Rila-source.csv
+```
+
+Generate **only** a catalog and manifest (no OSM output or staging DB):
+
+```sh
+./build/garmin2osm --input '/path/to/gmapsupp.img' \
+  --poi-types-out catalog.csv --poi-manifest source.csv --icons-dir my-icons
+```
+
+Other original flags remain: `--inspect`, `--contours`, `--all-levels`, `--keep-temp`, `--temp`, `--limit-maps`. Existing OSM, staging, and manifest output paths are never intentionally overwritten. Source inventory is generated even for ignored or `drop`-configured POIs, so intentional exclusions can be audited explicitly.
+
+## Override known defaults and add new POI types
+
+Edit `config/poi-overrides-all-types.csv` in LibreOffice/Excel or a text editor. Columns:
+
+```
+code,name_contains,mode,tags,note
+0x6414,,replace,man_made=water_well,ONLY IF VERIFIED
+0x6503,,append,information=guidepost;hiking=yes,Example: adds another attribute
+```
+
+Choose a valid mode: `append`, `replace`, `keep_raw`, or `drop`. `name_contains` is optional and case-sensitive; if populated, the rule applies only to names containing that text. Rule order matters within the same code. Actual `garmin:type`, `garmin:feature`, `garmin:mapset` and `name` cannot be overridden. Use `key=` to delete an existing default key. Unknown hex types will remain raw-only until mapped. **Never infer the meaning of a code solely from its hexadecimal number.**
+
+An immediately usable example that modifies all type `0x6414` points follows:
+
+```
+code,name_contains,mode,tags,note
+0x6414,,replace,man_made=water_well,TEST ONLY; not evidence of true semantics
+```
+
+NOTE: `0x6414` presently has a built-in `amenity=drinking_water` default. Its old duplicate fallback to `water_well` was unreachable and has been removed. The icon and field data need to establish the right category. **The sample replacement is not a verified mapping.**
+
+Coordinates in `data/BGMountains-unique-poi-types.csv` refer to the decoded *original IMG* features. For each type, open the coordinates in GPXSee's Garmin map and inspect both the label and icon. `data/poi-icons/0x6414.png` may provide additional context. The fact that a rendered glyph looks like water is **not** enough to distinguish `natural=spring`, `man_made=water_well` and `amenity=drinking_water`.
+
+## Automated validation: IMG → OSM
+
+The `--poi-manifest` output is a separately emitted inventory of every parsed IMG point. To compare against the generated OSM:
+
+```sh
+python3 tools/validate_pois.py compare \
+  --source Rila-source.csv --osm Rila.osm \
+  --rules config/poi-overrides-all-types.csv \
+  --report Rila-report.json \
+  --mismatches Rila-missing-extra.csv
+```
+
+Both `.osm` and `.osm.gz` are supported. The validator checks type, exact 7-digit quantized coordinates, name and multiplicity; it separately checks the assertions provided in the override CSV. Its failure exit code is nonzero; `--mismatches` includes direct OSM web links for investigation. Uses temporary SQLite to avoid holding the entire POI set in RAM. A `drop` override is treated as intentional absence, but a matching input record still appears in the source manifest.
+
+**Proof limit:** This is an internal IMG-to-OSM conversion-consistency check, not independent verification of Garmin decoder fidelity or the semantic meaning of a hex code. It cannot by itself prove that your `.obf` contains all the classified features.
+
+## Automated validation: OSM → OBF
+
+Build with the official OsmAndMapCreator distribution (download separately):
+
+```sh
+./scripts/build_obf.sh /path/OsmAndMapCreator "$PWD/Rila.osm"
+# Custom OsmAnd POI / rendering definitions may be supplied with their FULL respective config XMLs:
+./scripts/build_obf.sh /path/OsmAndMapCreator "$PWD/Rila.osm" \
+  --poi-types /path/to/full-poi_types.xml \
+  --rendering-types /path/to/full-rendering_types.xml
+```
+
+Inspect **your actual, externally compiled OBF**:
+
+```sh
+python3 tools/validate_pois.py obf-inspect \
+  --inspector /path/to/OsmAndMapCreator/inspector.sh \
+  --obf /path/to/Rila.obf \
+  --bbox 23.28,42.11,23.47,42.27 \
+  --report obf-poi-inspection.txt
+```
+
+The wrapper invokes OsmAnd BinaryInspector `-vpoi -vpoiobjects` with an optional bounding box and saves the output. It does **not** yet reconcile OBF internal IDs with the Garmin origin; OsmAnd may exclude unsupported tags from its map and POI indexes. For OBF persistence checks, use a POI (known category, name and coordinate) and inspect it via the rendered map / a bounded BinaryInspector dump; unknown types often require extending the **full** OsmAnd `poi_types.xml`. For line colors or custom map tags, modify **full** `rendering_types.xml`, then rebuild the OBF. A CSV rule only affects the OSM export; it does not change OsmAnd's indexes without a rebuild.
+
+## What is tested
+
+- Existing bundled CTest suite: 3/3 tests passed.
+- Supplied **441,942,016-byte** Bulgaria IMG: decoded **165 map regions**, **416,165 POI records**, **1,871 unique raw point types**, **96 icons**; full inventory generation completed.
+- Rila test bbox `23.28,42.11,23.47,42.27`: source and exported OSM each contained **2,424 POIs**, with **zero missing/extra** after seven-decimal coordinate correction.
+- Serial and `--jobs 4` Rila jobs: exactly the same multiset of **16,786 ways and 2,424 POIs** (including way geometry and tags).
+- A deliberately overridden test type was exported and assertions passed.
+- **Not tested:** user's full final `.obf` (not provided here); comprehensive semantic correspondence of 1,871 Garmin types; original Garmin route-network validity/turn-restrictions; all possible IMG/GMP variants.
+
+## Concurrency caution
+
+The Garmin container file uses a shared `std::ifstream`, protected by a mutex to ensure safe reads. Parser threads work on different `Map` objects with separate SQLite stores. For maximum throughput, `--jobs 4` may help on multi-core CPUs, but the disk read lock, SQLite writes and OSM XML output remain throughput bottlenecks. The default `--jobs 1` is deterministic and safest. Memory and temporary disk use increase with concurrency.
+
+## Add independent POI records (optional)
+
+Use `config/poi-additions.csv` with four CSV fields: `lat,lon,name,tags`.
+The program includes them with `garmin:feature=manual_poi` and `source=user_manual`.
+They are **not** presented as Garmin-extracted features; the source-POI validator
+ignores these intentional extra nodes. Add the flag `--poi-additions config/poi-additions.csv`
+to a command with `--output` (empty file is a no-op).
+
+## Reproduce the bundled regional audit without the source IMG
+
+```sh
+python3 tools/validate_pois.py compare \
+  --source samples/Rila-source-pois.csv \
+  --osm samples/Rila-vector.osm.gz \
+  --report results.json --mismatches differences.csv
+```
+
+For the complete source inventory, use the gzip-compressed CSV directly:
+`python3 tools/validate_pois.py compare --source data/BGMountains-source-pois.csv.gz --osm /your/full-map.osm`.

compile-osm-to-obf.sh

@@ -1,5 +0,0 @@
-export JAVA_OPTS="-Xms256M -Xmx19G"
-
-/home/opthq/vector-v4/garmin2osmand-v3/scripts/build_obf.sh \
-  /home/opthq/vector-v4/osm-map-creator \
-  /home/opthq/vector-v4/garmin2osmand-v3/Bulgaria-Garmin.osm

config/poi-additions.csv

@@ -0,0 +1,3 @@
+# Manually add independent POIs, distinct from the IMG source: lat,lon,name,tags
+# 42.2000000,23.3000000,My survey feature,natural=spring;survey:date=2026-10-14
+lat,lon,name,tags

config/poi-overrides.csv

@@ -0,0 +1,6 @@
+# A copy-ready configurator: code,name_contains,mode,tags,note
+# modes: replace (remove semantic defaults), append (override keys), keep_raw (retain only provenance), drop
+# Empty name_contains applies to every POI with that raw hex code.
+# You can REMOVE an existing tag with key= (empty value).
+code,name_contains,mode,tags,note
+# 0x6414,,replace,man_made=water_well,EXAMPLE ONLY - not confirmed; uncomment after verification