v096
This commit is contained in:
nq
@@ -1,103 +1,169 @@
|
||||
# BGMountains Garmin IMG → OSM → OsmAnd OBF v0.8
|
||||
# BGMountains Garmin IMG → native OsmAnd OBF — v0.95
|
||||
|
||||
**This release fixes the v0.7 icon-visibility blind spot and anonymous POIs.**
|
||||
**This is the integrated source release.** It replaces the v0.92/v0.93 Python
|
||||
pipeline and v0.94 standalone CSVs. Do not combine scripts from older ZIPs:
|
||||
those versions accept different flags and do not apply the 79 name-conditional
|
||||
rules. This archive has **one POI config and one line config**.
|
||||
|
||||
- Every one of the **263 validated retained Garmin POI types** has at least one conventional OSM semantic/display tag in `config/poi-overrides-all-types.csv`, in addition to raw `bgm_poi`/`garmin:*` provenance tags.
|
||||
- **87 codes** occur at least 100 times in the source catalog (409,154 source POIs). **46** use conventional feature categories recognized by stock OsmAnd in principle; **41** use an explicitly marked, synthetic `tourism=information;information=board` **local-OsmAnd display fallback**. Some additional lower-frequency codes also need fallback. This DOES NOT assert that the physical object is an information board.
|
||||
- Missing original IMG names become stable Bulgarian descriptive names, e.g. `Вода - Каптаж - 1THF1SDNTKJ`, with `bgm:generated_name=yes`. Original nonempty names are preserved.
|
||||
- `--jobs N` enables parallel extraction, **default one thread**.
|
||||
- New `tools/audit_icon_coverage.py` verifies source OSM classification coverage, names and produces per-type coordinate report. It **does not** prove OBF index presence or visual display; use an OBF inspector and OsmAnd for that.
|
||||
- Map-derived icon art and datasets remain governed by BGMountains' applicable copyright conditions. These synthetic POI tags are **not appropriate for uploading unchanged to the real OpenStreetMap database**.
|
||||
## Quick start (Ubuntu/Debian)
|
||||
|
||||
## Quick build (Ubuntu/Debian)
|
||||
|
||||
```sh
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y build-essential cmake libpng-dev libsqlite3-dev python3 openjdk-21-jre-headless unzip
|
||||
sudo apt install -y cmake g++ make libpng-dev libsqlite3-dev python3 openjdk-21-jre-headless unzip
|
||||
unzip BGMountains-garmin2osm-v0.95-integrated.zip
|
||||
cd BGMountains-garmin2osm-v0.95
|
||||
```
|
||||
|
||||
Use **exactly** this from the release root (where `scripts/` and `config/` are):
|
||||
|
||||
```bash
|
||||
nohup python3 scripts/build_bgmountains.py \
|
||||
--img "$HOME/gmapsupp.img" \
|
||||
--poi-config config/BGMountains-native-poi-mapping-v094.csv \
|
||||
--line-config config/line-mapping.csv \
|
||||
--out bg-mountains-v095-build \
|
||||
--download-creator \
|
||||
--jobs 8 \
|
||||
--java-opts '-Xms512M -Xmx25G' > bg-v095.log 2>&1 &
|
||||
```
|
||||
|
||||
The `--poi-config` and `--line-config` arguments are **implemented and tested**.
|
||||
Other supported flags: `--creator /path/to/OsmAndMapCreator`, `--bbox W,S,E,N`,
|
||||
`--prepare-only`, `--keep-raw`, `--template`, `--attach-routes`,
|
||||
`--disk-headroom-gb N`, `--ignore-disk-check`, `--jobs N` (default 1).
|
||||
|
||||
**Check your actual disk space first.** Full-country MapCreator compilation has
|
||||
previously required over 25 GiB of temporary disk space in addition to the
|
||||
source OSM (6+ GiB) on your machine. The script uses a conservative estimate
|
||||
`max(2 GiB, 5 × native-OSM-size + 1 GiB)`, unless `--disk-headroom-gb`
|
||||
requires more. These figures are not upper bounds. Choose an output filesystem
|
||||
with generous free space. `-Xmx25G` controls heap RAM, **not disk usage**.
|
||||
|
||||
## Safe regional test, no Java or extra download
|
||||
|
||||
```bash
|
||||
python3 scripts/build_bgmountains.py \
|
||||
--img /absolute/path/to/gmapsupp.img \
|
||||
--out ./BGMountains-out \
|
||||
--download-creator
|
||||
```
|
||||
|
||||
`--download-creator` is for the official OsmAndMapCreator, and the pipeline downloads the original official `poi_types.xml` and `rendering_types.xml` the first time as needed. Network access is required for these unless you already supply the files. **This runtime did not have MapCreator, so full OBF compilation was not executed here.**
|
||||
|
||||
Already installed OsmAndMapCreator:
|
||||
|
||||
```sh
|
||||
python3 scripts/build_bgmountains.py --img ./gmapsupp.img --out ./BGMountains-out \
|
||||
--creator /path/to/OsmAndMapCreator
|
||||
```
|
||||
|
||||
Opt-in parallelism: add `--jobs 4`. The output directory must be fresh: the converter refuses to overwrite an existing OSM or staging files.
|
||||
|
||||
Prepare and validate the OSM without invoking the Java OBF builder:
|
||||
|
||||
```sh
|
||||
python3 scripts/build_bgmountains.py --img ./gmapsupp.img --out ./BGMountains-preview \
|
||||
--img "$HOME/gmapsupp.img" \
|
||||
--poi-config config/BGMountains-native-poi-mapping-v094.csv \
|
||||
--line-config config/line-mapping.csv \
|
||||
--bbox '25.05,42.43,25.16,42.54' \
|
||||
--jobs 8 \
|
||||
--out ./regional-test \
|
||||
--prepare-only
|
||||
```
|
||||
|
||||
**IMPORTANT: You must REBUILD the OSM and OBF.** This update cannot modify an already compiled `.obf` or give v0.7 output its new tags in place.
|
||||
Inspect `regional-test/BGMountains-native.osm`, `semantic-audit.json`,
|
||||
`route-report.json`, `extract.log`, `translate.log`, `audit.log`. The tool
|
||||
must exit nonzero on a failed conversion subprocess. You can instead pass an
|
||||
existing MapCreator with `--creator ...` (omit `--prepare-only`) for a small
|
||||
real OBF build.
|
||||
|
||||
## What the configuration files mean
|
||||
## Two editable configuration files
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `config/poi-overrides-all-types.csv` | **Active** 1,871-row config, including 263 retained types with revised tags. `mode=replace` replaces old hardcoded semantics, retaining provenance/name. All non-present legacy type rows remain for historical comparison. |
|
||||
| `config/poi-overrides-all-types-v07-user-upload.csv` | Exact backup of the CSV you supplied. |
|
||||
| `config/poi-labels-bg.csv` | New 263-row Bulgarian fallback name prefixes, editable. |
|
||||
| `config/poi-icon-audit.csv` | All 263 types, count, original tags vs new tags, classification confidence and sample coordinates. |
|
||||
| `config/osmand-icons.csv` | Optional **custom render style** icon names/visibility; never controls the default `tourism=information` fallback injected through the active overrides. |
|
||||
| `data/BGMountains-frequent-poi-types-100plus-v08.csv` | All 87 frequent types, with reviewed tags, classification mode and coordinates. |
|
||||
| `tools/audit_icon_coverage.py` | Independent POI OSM field audit; one row per present type. |
|
||||
### 1. `config/BGMountains-native-poi-mapping-v094.csv`
|
||||
|
||||
**How to edit a type:** find `code=0x6500` in active overrides. Edit the **tags** column, leaving a `key=value;key=value` list. Change `mode=replace` only if you understand how the C++ built-in classifications merge. Set a Bulgarian text prefix in `config/poi-labels-bg.csv`. Run again from scratch to rebuild OSM then OBF.
|
||||
1,871 base hex-code entries plus 79 **name-specific** exceptions (1,950 data
|
||||
rows). Columns include: `code`, `source_count`, `legend_bg`, `legend_en`,
|
||||
`meaning_confidence`, `osm_tags`, `label_bg`, `priority`, `zoom_hint`, `action`,
|
||||
`note`, examples, and optional `name_contains`. For conditional matching,
|
||||
set `name_contains` in an additional row with the same `code`. The matcher uses
|
||||
case-sensitive substring matching on the **original Garmin name**, with rules
|
||||
applied in file order after the base rule. `replace` changes the converter's
|
||||
semantic defaults; `append` retains previous tags; `raw` leaves unspecified
|
||||
features unclassified until private-map fallback; `drop` discards the feature.
|
||||
|
||||
### Why 0x6500 is not `amenity=drinking_water`
|
||||
The Python pipeline automatically creates `cpp-poi-override.csv` and
|
||||
`cpp-name-labels.csv` **inside the build output directory**. These are generated
|
||||
intermediates, **not third and fourth user configuration files**. There is no
|
||||
need to manually supply them or an icon pack to OsmAnd.
|
||||
|
||||
The original legend labels this a **каптаж** (water intake/catchment), not necessarily a drinking-water source. v0.8 records `man_made=water_intake` but adds an explicit *presentation-only* `tourism=information;information=board;bgm:synthetic_icon=yes` for common OsmAnd rendering. The "information board" is NOT a physical claim. The default Bulgarian name prefix is `Вода - Каптаж`. If you want `Вода - Резервоар`, just edit the label CSV; do not misrepresent a small intake as an artificial lake (`natural=water;water=reservoir`).
|
||||
Important native examples:
|
||||
|
||||
`0x6602` already has `tourism=viewpoint` and does not use a synthetic board marker. Its failure to appear in a particular existing OBF could be an old build, incomplete POI indexing, or display suppression—only inspecting that actual OBF can distinguish these.
|
||||
- `0x6414`: `amenity=drinking_water;man_made=water_tap` (legacy v18 dual
|
||||
classification); named dry sources such as `(Сух.)` remove drinking-water
|
||||
classification and get `drinking_water=no`.
|
||||
- `0x6500`: `man_made=water_intake;amenity=water_point;drinking_water=unknown`
|
||||
for **private-map index compatibility**, not evidence that its water is
|
||||
potable, available or accessible. Named shaft (`Шах...`) exceptions remove
|
||||
the compatibility alias.
|
||||
- `0x6511`: `natural=spring` (potability not asserted).
|
||||
- `0x6402`: `building=yes` by default, explicitly named `Заслон` or `Хижа`
|
||||
may become shelter/hut (availability/access not verified).
|
||||
- `0x661d`: agricultural structures remain agricultural unless specifically
|
||||
identified otherwise.
|
||||
- `0x6600`: keep numeric spot height `ele=*`, **no fake settlement**.
|
||||
- Unknown POIs: `place=locality`, `Неуточнено - <stable hash>` if unnamed,
|
||||
and `bgm:synthetic_fallback=place_locality`; this private-map fallback **does
|
||||
not guarantee Nearby POI indexing**.
|
||||
|
||||
### Stable anonymous names
|
||||
`priority` and `zoom_hint` are documented classification hints only. They do
|
||||
**not force stock OsmAnd to show an icon at zoom 12**. Likewise `garmin:type`
|
||||
remains a provenance tag, not a display rule. Only stock OsmAnd classifications
|
||||
and rendering apply to the resulting OBF.
|
||||
|
||||
For an unnamed POI, `name = Bulgarian_label + " - " + FNV1a_base36_11(mapset|type|quantized_lat|quantized_lon)`. The output is deterministic for an identical IMG and coordinates across threads and builds. A very small collision probability remains; the coordinates/mapset/type remain available. These generated names are for an offline map, **not actual surveyed names**.
|
||||
### 2. `config/line-mapping.csv`
|
||||
|
||||
### Test against your three sample coordinates
|
||||
50 Garmin line types (20 currently have `route_mode=relation`, 30 preserve
|
||||
source geometry). Columns include Garmin `code`, legend, color,
|
||||
`osm_way_tags`, `route_mode`, `osmc_symbol`, `network`, confidence and priority.
|
||||
Selected source overlay segments are grouped into native
|
||||
`type=route;route=hiking` relations with `osmc:symbol`. Garmin source color is
|
||||
preserved as relation `colour=*`. The script does not create duplicate routable
|
||||
paths from decoration lines. Opt-in `--attach-routes` matches overlays to real
|
||||
existing path geometries, but can use significant memory; it is off by default.
|
||||
**Existing physical path routing remains in the data.** Exact phone-side color,
|
||||
icon and at-zoom visibility still depend on stock OsmAnd.
|
||||
|
||||
| Type | Coordinates | Expected tags | v0.8 generated name |
|
||||
|---|---|---|---|
|
||||
| `0x6602` | `42.0346141,24.6528697` | `tourism=viewpoint` | `Панорама - Гледка - 25IVP49H823` |
|
||||
| `0x6500` | `42.0347643,24.6522689` | `man_made=water_intake;tourism=information;information=board` | `Вода - Каптаж - 1THF1SDNTKJ` |
|
||||
| `0x6500` | `42.0340991,24.6520114` | Same | `Вода - Каптаж - ED03D455VTZ` |
|
||||
## New or unknown IMG maps: generate configuration templates
|
||||
|
||||
These examples were directly extracted from the 422MiB IMG into a Plovdiv test OSM and compared to a source manifest. All 57 POIs in that test passed exact semantic-tag checks, with **zero missing and extra**. Serial vs `--jobs 4` POI identities/tags were equal.
|
||||
|
||||
## Review before copying `.obf` into OsmAnd
|
||||
|
||||
Pipeline artifacts in `--out`:
|
||||
- `BGMountains.osm` — exact source XML for the OBF builder
|
||||
- `source-pois.csv`, `poi-types.csv` — extraction manifest and type counts
|
||||
- `validation.json`, `mismatches.csv` — original-source to OSM audit
|
||||
- `poi-coverage.csv`, `poi-coverage.json` — export display mapping coverage
|
||||
- `osmand-config/poi_types.bgmountains.xml`, `rendering_types.bgmountains.xml` — registrations added to **complete upstream XML**
|
||||
- `osmand-config/BGMountains-custom.render.xml` — optional style; exact Garmin TYP art is NOT embedded into stock OsmAnd
|
||||
- `*.obf` plus `mapcreator.log`, `obf-poi-index.txt` (only after a successful Java compilation)
|
||||
|
||||
The `tourism=information` fallback should give OsmAnd a native searchable POI category in an appropriate compatible build. **Neither an OSM tag nor an indexed OBF proves a symbol will be drawn at every zoom**: the map style, profile, POI filters, icon availability, density/collision and selected offline map still affect visibility. Test at zoom 17–20 with POI layers enabled. Avoid having an old overlapping BGMountains OBF selected at the same time.
|
||||
|
||||
To inspect your built file with OsmAndMapCreator:
|
||||
|
||||
```sh
|
||||
/path/to/OsmAndMapCreator/inspector.sh -vpoi -vpoiobjects \
|
||||
-bbox=24.64,42.05,24.665,42.02 ./BGMountains-out/*.obf \
|
||||
> ./BGMountains-out/plovdiv-obf-inspector.txt
|
||||
```bash
|
||||
python3 scripts/build_bgmountains.py --img /path/to/unknown.img \
|
||||
--out ./template-newmap --bbox '23.0,42.0,24.0,43.0' --template
|
||||
```
|
||||
|
||||
Inspect output for the three named POIs. **There is no tested generic parser of OsmAndMapCreator's inspector text in this package**, and we have not verified a full-country OBF built from v0.8 in this runtime. If OSM has the POI but the inspector doesn't, focus on indexing or generated XML; if inspector has it but the screen doesn't, focus on style/POI display/filter/zoom.
|
||||
Produces `poi-mapping.TEMPLATE.csv` and `line-mapping.TEMPLATE.csv` with
|
||||
observed types and sample coordinates. Fill them manually and run with
|
||||
`--poi-config ./template-newmap/poi-mapping.TEMPLATE.csv` and
|
||||
`--line-config ./template-newmap/line-mapping.TEMPLATE.csv`. **Do not assume
|
||||
Garmin codes have the same meaning in unrelated maps.** If you need a complete
|
||||
catalog for a new national map, generate its templates without `--bbox`, but
|
||||
this can require significant time and intermediate disk space.
|
||||
|
||||
### Local-only licensing warning
|
||||
## Validation and provenance
|
||||
|
||||
Do not upload the generated synthetic display tags to OpenStreetMap or assert they accurately describe physical information boards. The original IMG, derived icons and extracts can have third-party copyright restrictions. The source-only code path should be distributed separately from map-derived sample assets until rights are confirmed.
|
||||
```bash
|
||||
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
|
||||
cmake --build build -j 4
|
||||
ctest --test-dir build --output-on-failure
|
||||
```
|
||||
|
||||
There is also `tools/audit_osmand_poi_mapping.py`, which takes the official
|
||||
version-matched OsmAnd `poi_types.xml` and produces a registry compatibility
|
||||
report, **not a claim of final icon visibility**:
|
||||
|
||||
```bash
|
||||
python3 tools/audit_osmand_poi_mapping.py \
|
||||
--mapping config/BGMountains-native-poi-mapping-v094.csv \
|
||||
--poi-types /path/to/poi_types.xml \
|
||||
--report native-poi-audit.csv --json native-poi-audit.json
|
||||
```
|
||||
|
||||
Source: https://github.com/osmandapp/OsmAnd-resources/blob/master/poi/poi_types.xml
|
||||
|
||||
## Tested, and NOT tested
|
||||
|
||||
- C++ Release build, 6 CTest test groups, conditional POI CSV generation,
|
||||
CLI options, and simulated MapCreator orchestration passed.
|
||||
- Original 422 MiB IMG: regional `--jobs 8` and serial preparation succeeded
|
||||
with identical 427 POI coordinate/name/tag records and 6 route relations.
|
||||
- Separate region: `(Сух.)` water-tap conditional correctly produced
|
||||
`man_made=water_tap;drinking_water=no`, and no `amenity=drinking_water`.
|
||||
- **A real OsmAndMapCreator binary was not available here**; its host was
|
||||
unreachable. The OBF orchestration unit test uses a **mock**, so native
|
||||
nationwide OBF compilation and phone rendering are not verified. Do not
|
||||
mistake that mock for real OBF validation.
|
||||
|
||||
License: the source bundle contains historically BGMountains-derived
|
||||
classification information. Treat that data as private-map material under
|
||||
applicable source licenses; the package is **not** authorization to republish
|
||||
BGMountains geography or to import its features into the public OSM database.
|
||||
|
||||
Reference in New Issue
Block a user