v.093
This commit is contained in:
nq
@@ -0,0 +1 @@
|
||||
See README_V093.md for current diagnosis and changes. Historical v0.92 notes were superseded.
|
||||
+130
@@ -0,0 +1,130 @@
|
||||
# BGMountains Garmin IMG → native OSM → OsmAnd OBF — v0.92
|
||||
|
||||
**v0.92** fixes the v0.8–v0.9 synthetic marker/marina fallback approach. It aims to create a **native OsmAnd map** without Garmin TYP icon PNGs or a custom `.render.xml` on the phone.
|
||||
|
||||
The **only two user-editable translation files** are:
|
||||
|
||||
1. `config/poi-mapping.csv` (Garmin point type → native OSM tag(s), Bulgarian naming prefix, legend, priority, historic icon pointer, example coordinates)
|
||||
2. `config/line-mapping.csv` (Garmin extended line type → OSM way tags, color-marked hiking relation policy and `osmc:symbol`)
|
||||
|
||||
Everything under the generated build/output directory is an **intermediate or report**, not a third configuration. You don't have to supply `poi_types.xml`, `rendering_types.xml`, an icons CSV or a custom rendering style.
|
||||
|
||||
## Requirements
|
||||
|
||||
Ubuntu/Debian: `sudo apt install build-essential cmake libpng-dev libsqlite3-dev python3 openjdk-21-jre-headless unzip`
|
||||
|
||||
To **compile an OBF**, provide an official [OsmAndMapCreator](https://osmand.net/docs/technical/map-creation/create-offline-maps-yourself/) installation containing `utilities.sh`, or use the automatic download option (downloads from OsmAnd). **Python, CMake and `make` by themselves do NOT build OBF**.
|
||||
|
||||
## Full map: one command
|
||||
|
||||
```sh
|
||||
python3 scripts/build_bgmountains.py \
|
||||
--img '/path/to/gmapsupp.img' \
|
||||
--out './output' \
|
||||
--creator '/path/to/OsmAndMapCreator'
|
||||
```
|
||||
|
||||
If you don't have MapCreator:
|
||||
|
||||
```sh
|
||||
python3 scripts/build_bgmountains.py \
|
||||
--img '/path/to/gmapsupp.img' --out './output' --download-creator
|
||||
```
|
||||
|
||||
It compiles C++, extracts the Garmin IMG to OSM, uses the two mapping files, assigns stable Bulgarian hashes to unnamed POIs, produces route relations, conservatively attaches matched overlays to actual `highway=*` ways, performs a semantic QA audit, and runs OsmAndMapCreator. The **only file to copy onto your phone is the resulting `.obf`**. Enable hiking routes and relevant POI filters in OsmAnd as necessary. No third-party icons or renderer file is included or required.
|
||||
|
||||
For a fast test that stops before the Java OBF stage:
|
||||
|
||||
```sh
|
||||
python3 scripts/build_bgmountains.py \
|
||||
--img '/path/to/gmapsupp.img' \
|
||||
--out './vihren-test' --bbox '23.36,41.70,23.42,41.76' --prepare-only
|
||||
```
|
||||
|
||||
Parallelism remains **disabled by default** (`--jobs 1`). For opt-in concurrent IMG extraction append `--jobs 4`. The Python mapping, relation, QA and Java steps are not parallelized by this switch.
|
||||
|
||||
Use `--no-route-attachment` to keep source overlay geometries as hiking-relation members instead of attaching matched real path/road ways. `--keep-raw` retains the intermediate IMG-native XML for debugging.
|
||||
|
||||
## Editing the 2 CSV files
|
||||
|
||||
### `config/poi-mapping.csv`
|
||||
|
||||
| Column | What it means |
|
||||
|---|---|
|
||||
| `code` | Garmin POI type, in hex; never changed in provenance |
|
||||
| `source_count` | Count from source inventory; informational only |
|
||||
| `legend_bg`, `legend_en` | Original BGMountains legend meaning (where verified) |
|
||||
| `meaning_confidence` | `legend` / `source_inferred`; review unknowns |
|
||||
| `osm_tags` | `k=v;k=v` for **truthful standard** OSM semantics; no fake category for icons |
|
||||
| `label_bg` | Prefix for otherwise nameless objects, with deterministic alphanumeric suffix |
|
||||
| `priority` | `critical`, `normal` or `terrain`; currently emits `bgm:priority` metadata |
|
||||
| `zoom_hint` | Desirable zoom target for future style/QA work, **not an OsmAnd override** |
|
||||
| `action` | `replace` removes earlier semantics, `append` keeps them, `raw` keeps provenance only, `drop` discards the POI |
|
||||
| `legacy_icon` | `TYP:<hex>` reference for older Garmin icon, for auditing only |
|
||||
| `example_lat`, `example_lon`, `example_name` | Sample place from original source inventory to inspect |
|
||||
|
||||
If a source code is known, edit `osm_tags` and set `action=replace`. Example: `0x6414` → `amenity=drinking_water`. If it isn't known, leave it as `raw` until you inspect the old legend and sample coordinates. Nothing will invent a marina, information board or drinking tap for an unknown symbol. You may append multiple compatible tags separated by semicolons.
|
||||
|
||||
**Water/shelter priority rules already populated**: `0x6414` drinking tap, `0x6511` spring, `0x6500` water intake, `0x6417` well, `0x6609` hut, `0x660a` basic refuge, `0x661e` shelter and `0x6402` building. For requested naming, unnamed `0x6402` uses the prefix **„Заслон - Сграда“**; its factual OSM classification stays `building=yes` (a generic building must not falsely be declared an actual lifesaving shelter). Verify shelter suitability on the ground before relying on it.
|
||||
|
||||
`0x6600` numeric elevation labels preserve `ele=*` instead of being misclassified as peaks, buildings or missing POI categories. `0x650b` winter waymarkers and `0x660b` trail markers remain Garmin-sourced nodes rather than incorrect water/marina/guidepost POIs.
|
||||
|
||||
### `config/line-mapping.csv`
|
||||
|
||||
| Column | Meaning |
|
||||
|---|---|
|
||||
| `code`, `source_count` | Garmin extended polyline code/count |
|
||||
| `legend_bg`, `legacy_color` | Known description and source TYP base color |
|
||||
| `osm_way_tags` | Additional truthful `key=value;key=value` tags on source ways |
|
||||
| `route_mode` | `relation` builds OSM hiking routes; `preserve` keeps raw way without assuming it is a trail |
|
||||
| `osmc_symbol` | Standard OSMC syntax, e.g. `red:white:red_bar` |
|
||||
| `network` | `lwn` (local walking), change if there is evidence for a larger network |
|
||||
| `route_confidence` | `color_only` means inferred from color but pattern not independently established |
|
||||
| `priority` | Descriptive category only, not stock OsmAnd zoom control |
|
||||
|
||||
Colored candidate hiking overlays become `type=route` + `route=hiking` + `osmc:symbol=*` relations. All retain the original `garmin:type` and source line metadata. Matching route overlay lines to existing physical `highway=*` geometry avoids generating fake navigable duplicate paths. A physical road segment is used only if at least 90% of an overlay's segments match and the physical way extends at most 10% beyond it; unmatched overlays remain relation members, flagged in `route-attachment.json`.
|
||||
|
||||
**A source color is not conclusive proof of a painted bar pattern**. All automatic bar-style assignments from color-only evidence explicitly say `route_confidence=color_only`; check the original map and choose the right `osmc_symbol` for each code before treating that detail as verified. See [OSM trail markings](https://wiki.openstreetmap.org/wiki/Trail_markings) and [OSMC symbol syntax](https://wiki.openstreetmap.org/wiki/Key:osmc:symbol).
|
||||
|
||||
## New IMG / future map: generate blank templates
|
||||
|
||||
```sh
|
||||
python3 scripts/build_bgmountains.py \
|
||||
--img '/path/to/another-garmin.img' \
|
||||
--out './new-map-template' --template
|
||||
```
|
||||
|
||||
The result contains `poi-mapping.TEMPLATE.csv` with **all discovered types, counts and sample POI coordinates**, and `line-mapping.TEMPLATE.csv` with all discovered extended line types/counts. Unknowns start as `action=raw`, `route_mode=preserve` and **without semantic tags**. You must fill them based on that map's own legend and examples; BGMountains mappings must not be assumed to apply to another Garmin map. To build it, point `--poi-config` / `--line-config` to the new two reviewed files.
|
||||
|
||||
## Outputs / validation
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `BGMountains-native.osm` | Final translated, standard-tagged OSM intermediate |
|
||||
| `source-pois.csv` | Per-point source type, mapset, coordinates and name |
|
||||
| `discovered-pois.csv` | Observed type inventory with up to 5 sample coordinates |
|
||||
| `route-report.json` | Coloured overlays/route relation counts |
|
||||
| `route-attachment.json` | Reused physical path vs unmatched overlay counts |
|
||||
| `semantic-audit.json` | Detailed native category statistics, unknown types and example positions |
|
||||
| `extract.log`, `translate.log`, `audit.log` | Pipeline logs |
|
||||
| `mapcreator.log` | Java OBF build log when compilation is attempted |
|
||||
| `obf-inspection.txt` | BinaryInspector POI summary (if inspector is installed) |
|
||||
| `*.obf` | Final file for OsmAnd (only if MapCreator succeeds) |
|
||||
|
||||
Tests:
|
||||
|
||||
```sh
|
||||
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
|
||||
cmake --build build -j 2
|
||||
ctest --test-dir build --output-on-failure
|
||||
```
|
||||
|
||||
## What native stock OsmAnd cannot guarantee
|
||||
|
||||
- OsmAnd's **own rendering style controls icon selection, label collision and minimum zoom**; `zoom_hint=12` does not change it. To force all water/hut icons to zoom 11 or 12, a custom OsmAnd style/app configuration is unavoidable. v0.92 deliberately does **not** install one.
|
||||
- Not every valid OSM tag has a stock icon, especially water intakes, bridge points, elevation points and trail blazes. We prefer correct/metadata-only tags over false marinas/boards.
|
||||
- A valid hiking relation is not a guarantee that stock OsmAnd renders every segment. Inspect `route-attachment.json` and verify on-device.
|
||||
- The OSM file parser and XML/route structural tests do **not** establish that the POIs survived final OBF indexing. Use `mapcreator.log`, `obf-inspection.txt` and actual OsmAnd app checks. We cannot claim a complete nationwide OBF was compiled/tested in this environment unless MapCreator is available.
|
||||
- Map-derived sample coordinates and legend crosswalk are subject to BGMountains data licensing; do not relicense or import them to public OSM automatically.
|
||||
|
||||
**v0.92 does not use the Garmin icon PNGs or `BGMountains-outdoor.render.xml`**. You may remove older BGMountains OBFS from OsmAnd to avoid duplicates before installing the new one.
|
||||
@@ -0,0 +1,76 @@
|
||||
# BGMountains v0.93 — bounded-memory translation and private-map POI fallback
|
||||
|
||||
This is a **source patch release**, building on v0.92. All user translation choices live in exactly two files:
|
||||
|
||||
1. `config/poi-mapping.csv`: Garmin hex → OSM semantic tags; Bulgarian legend name and old TYP code; `action`, priority, source count and examples.
|
||||
2. `config/line-mapping.csv`: Garmin line hex → OSM way tags, `osmc:symbol`, route relation choice, marking color and confidence.
|
||||
|
||||
`cpp-poi-override.csv` and `cpp-name-labels.csv` are generated **internal intermediates**, not control files; do not edit them.
|
||||
|
||||
## Direct diagnosis of the supplied nohup
|
||||
|
||||
The same appended logfile shows **different historical attempts**: an early `java.lang.OutOfMemoryError: Java heap space` in OsmAndMapCreator, followed in the last attempt by `java.io.IOException: No space left on device` while `IndexCreator.extractOsmToNodesDB` built the Java/SQLite intermediate database. Earlier `poi-coverage.log` saying `PASS` refers only to OSM semantic checks. It is not a successful OBF creation signal.
|
||||
|
||||
## Changes
|
||||
|
||||
- Streaming OSM parser clears the XML **root**, preventing an accumulating empty Element for every object.
|
||||
- Expensive optional `attach_native_routes.py` step is opt-in (`--attach-routes`); default keeps existing routable highway network unchanged, and generates hiking-route relations from Garmin colored overlays. Without attachment, those relations describe overlay geometry; they are not proof of route-based navigation along the existing path ways.
|
||||
- Unclassified POIs now receive `place=locality`, `bgm:synthetic_fallback=place_locality`, `bgm:classification=unverified`, and a generated Bulgarian name `Неуточнено - <stable hash>` **when the source name was empty or previously synthetic**. Genuine source names remain untouched. Raw `garmin:type` and mapset information are preserved.
|
||||
- **Exception:** `0x6600` numeric spot heights and other elevation annotations (`ele=*`) are not fabricated as localities. They remain elevations rather than false place icons. Known native tags are never overwritten by the fallback.
|
||||
- Native categories for water, drinking water, shelters/huts, bridges and more remain sourced from existing two mapping files. No extracted Garmin artwork or phone-side custom stylesheet is required.
|
||||
- Semantic audit differentiates actual unclassified records, synthetic locality fallbacks and non-POI elevation annotations, and includes example coordinates. `PASS` still does not prove an Android icon renders.
|
||||
- Conservative free-disk preflight before invoking MapCreator, based on `max(2 GiB, 5*OSM size + 1 GiB)` free in the output filesystem. The estimate is based on the reported >25 GiB scratch need for ~6 GiB OSM; it is *not a guarantee*.
|
||||
- JVM heap via `JAVA_TOOL_OPTIONS` as well as `JAVA_OPTS`, for MapCreator builds that ignore the latter; use e.g. `--java-opts '-Xms512M -Xmx6G'` (subject to host RAM). **A large heap cannot solve out-of-disk.**
|
||||
- OBF success only if a newly generated nonempty `.obf` exists, not a stale file from an earlier run.
|
||||
|
||||
## Quick regional test — no Java, no large output
|
||||
|
||||
```sh
|
||||
python3 scripts/build_bgmountains.py \
|
||||
--img '/path/to/gmapsupp.img' \
|
||||
--bbox '23.30,41.67,23.50,41.84' \
|
||||
--out './regional-test' \
|
||||
--prepare-only
|
||||
|
||||
cat regional-test/semantic-audit.json
|
||||
cat regional-test/route-report.json
|
||||
```
|
||||
|
||||
`--jobs 1` is the default. `--jobs 4` is optional; on the source IMG, serial and parallel runs had equal POI counts and candidate colored overlay counts but a slight discrepancy in connected route groups (78 vs 82). **Do not assume identical routing topology with parallel extraction yet.**
|
||||
|
||||
To test a truly unfamiliar map and generate configuration templates:
|
||||
|
||||
```sh
|
||||
python3 scripts/build_bgmountains.py --img another.img --bbox 'W,S,E,N' \
|
||||
--out ./new-map --template
|
||||
# Review: new-map/poi-mapping.TEMPLATE.csv and line-mapping.TEMPLATE.csv
|
||||
```
|
||||
|
||||
The templates are intentionally blank for unknown source meanings; when used by the translation pipeline, unknown POIs will receive the private `place=locality` fallback. A new map should not blindly reuse BGMountains meanings.
|
||||
|
||||
## Full OBF build — only on a machine with enough free scratch/RAM
|
||||
|
||||
```sh
|
||||
python3 scripts/build_bgmountains.py \
|
||||
--img '/path/to/gmapsupp.img' \
|
||||
--out '/large-volume/bgmountains-v093' \
|
||||
--creator '/path/to/OsmAndMapCreator' \
|
||||
--java-opts '-Xms512M -Xmx8G'
|
||||
```
|
||||
|
||||
Or first use `--prepare-only` and later `--osm '/path/to/IMG-raw.osm'` to skip the C++ extraction **if** that raw original file was saved (`--keep-raw`). `--osm` expects the *raw* IMG-derived OSM, not the already translated native OSM.
|
||||
|
||||
The preflight checks output filesystem free space; for shared temp partitions, also check `df -h /tmp "$PWD"` and `df -i`. An OBF may still fail if actual scratch is elsewhere. Only override the preflight (`--ignore-disk-check`) when you have independently confirmed scratch capacity.
|
||||
|
||||
## Limitations / honesty
|
||||
|
||||
- `place=locality` is a **synthetic label hack chosen for this PRIVATE map**, not a correct definition of all unknown source objects and not suitable for direct uploading into OSM.
|
||||
- Stock OsmAnd decides icon set, zoom threshold, collision suppression and Nearby POI search category. A native OSM tag increases compatibility but cannot guarantee every icon is shown or listed in Nearby without a mobile test.
|
||||
- Route colors represented through native OSM hiking relations using `osmc:symbol` for identified Garmin colored overlay types. Color/pattern interpretations with limited source evidence remain explicitly provisional.
|
||||
- No full-country OBF creation has been performed as part of this patch validation, per the user's storage constraints. The final national MapCreator step remains to be verified on the user's workstation.
|
||||
|
||||
## Regression verification
|
||||
|
||||
- Unit tests: `python3 -m unittest discover -s tests -p 'test_v093.py' -v`
|
||||
- Previous v0.92 tests: `python3 -m unittest discover -s tests -p 'test_v092.py' -v`
|
||||
- Regional real IMG: 1,915 POIs, 296 elevation annotations, 150 synthetic localities, zero unclassified after fallback, 273 route-overlay candidates. Single-worker result has 78 connected route groups. The v0.93 code has not been validated end-to-end against stock OsmAnd on Android.
|
||||
@@ -0,0 +1,51 @@
|
||||
code,source_count,legend_bg,legacy_color,osm_way_tags,route_mode,osmc_symbol,network,route_confidence,priority,note
|
||||
0x10e1f,237835,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f12,217991,Неустановена линейна декорация,#e6e6e6,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f16,36027,Неустановена линейна декорация,#e6e6e6,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e0b,15148,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f17,10965,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#ff3041,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e13,10607,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e12,8662,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f14,8475,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f11,6436,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f0a,6231,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#ff0000,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e0a,4347,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#0030ff,,relation,blue:white:blue_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f00,4197,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f0f,4098,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#399500,,relation,green:white:green_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e11,3926,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e05,3011,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#ffff00,,relation,yellow:white:yellow_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e0c,2375,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e1c,2045,Неустановена линейна декорация,#ffffff,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f13,1575,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e19,1469,Неустановена линейна декорация,#525252,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e09,1074,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#0030ff,,relation,blue:white:blue_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e04,1048,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#ffff00,,relation,yellow:white:yellow_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e16,636,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10504,591,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#bd3041,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f0e,488,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#399500,,relation,green:white:green_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10401,394,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#bd3041,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e1b,368,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#ff0000,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10602,354,Неустановена линейна декорация,#777777,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e1d,327,Неустановена линейна декорация,#ffffff,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e18,292,Неустановена линейна декорация,#414141,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e0d,267,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10108,232,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#bd3041,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e1a,205,Неустановена линейна декорация,#ffffff,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e03,199,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#ffff00,,relation,yellow:white:yellow_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e15,184,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f05,162,Неустановена линейна декорация,#7b3000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e1e,162,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e08,119,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#0030ff,,relation,blue:white:blue_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e17,110,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#ff0000,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e14,109,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f15,98,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f04,93,Неустановена линейна декорация,#7b3000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10606,78,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#bd3041,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e02,60,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#ffff00,,relation,yellow:white:yellow_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e0e,56,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10402,47,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#bd3041,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f03,9,Неустановена линейна декорация,#7b3000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10f01,4,Неустановена линейна декорация,#7b3000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10e0f,4,Неустановена линейна декорация,#000000,,preserve,,lwn,unknown,terrain,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10503,3,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#bd3041,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
0x10403,3,"Туристическа линейна маркировка (цветът е потвърден, шарката не)",#bd3041,,relation,red:white:red_bar,lwn,color_only,critical,Provisional symbol until verified against legend; overlay is NOT a routable path
|
||||
|
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,75 @@
|
||||
# Original legend crosswalk, all 67 historic icon matches
|
||||
|
||||
Reference table generated from the earlier reviewed Garmin TYP/legend matching data. **Documentation only**; the two CSV files under `config/` are the only user-editable translation controls. The original icon PNGs are not needed, used, or packaged. Where no native semantic tag is defensible it is shown as “unclassified”.
|
||||
|
||||
| Source Garmin hex | Legend BG | Legend EN | Old TYP icon | v0.92 OSM tags | Priority | Sample lat, lon |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 0x200 | Столица | Capital City | TYP:0x200 | place=city;capital=yes | normal | 41.0612941, 28.8116670 |
|
||||
| 0x800 | Град >75 000 | Large City | TYP:0x800 | place=city | normal | 43.9025259, 22.2788572 |
|
||||
| 0x900 | Град <75 000 | Town | TYP:0x900 | place=town | normal | 42.0570374, 24.5459247 |
|
||||
| 0x2c05 | Училище | School | TYP:0x2c05 | amenity=school | normal | 41.7214394, 24.6845198 |
|
||||
| 0x2c0b | Църква | Church | TYP:0x2c0b | amenity=place_of_worship | normal | 43.0217957, 24.8099613 |
|
||||
| 0x2d04 | Казино | Casino | TYP:0x2d04 | unclassified | normal | — |
|
||||
| 0x2d08 | Пързалка | Ice Rink | TYP:0x2d08 | unclassified | normal | — |
|
||||
| 0x2e05 | Аптека | Pharmacy | TYP:0x2e05 | amenity=pharmacy | normal | 42.7442837, 25.3431416 |
|
||||
| 0x2e07 | Магазин за дрехи | Clothes shop | TYP:0x2e07 | shop=clothes | normal | 43.0698609, 25.6080151 |
|
||||
| 0x2f0a | Синагога | Synagogue | TYP:0x2f0a | amenity=place_of_worship;religion=jewish | normal | 41.5711713, 23.7286234 |
|
||||
| 0x2f0c | Джамия | Mosque | TYP:0x2f0c | amenity=place_of_worship;religion=muslim | normal | 44.0549612, 26.7991304 |
|
||||
| 0x2f0e | Автомивка | Car Wash | TYP:0x2f0e | amenity=car_wash | normal | 42.2939301, 22.7035904 |
|
||||
| 0x3004 | Съд | Courthouse | TYP:0x3004 | unclassified | normal | — |
|
||||
| 0x3008 | Пожарна | Fire Station | TYP:0x3008 | amenity=fire_station | normal | 42.5666571, 27.6193285 |
|
||||
| 0x4700 | Пристанище | Port | TYP:0x4700 | unclassified | normal | 41.6431189, 25.2845192 |
|
||||
| 0x6400 | Кариера | Quarry | TYP:0x6400 | landuse=quarry | normal | 40.8776808, 23.9436078 |
|
||||
| 0x6401 | Мост | Bridge | TYP:0x6401 | man_made=bridge | critical | 42.6645684, 22.9691291 |
|
||||
| 0x6402 | Сграда | Building | TYP:0x6402 | building=yes | critical | 42.5507569, 23.2217932 |
|
||||
| 0x6403 | Гробище, оброк | Graveyard/Tombstone | TYP:0x6403 | landuse=cemetery | normal | 43.0418801, 23.5914874 |
|
||||
| 0x6404 | Църква | Church | TYP:0x6404 | amenity=place_of_worship | normal | 42.1896243, 27.0861053 |
|
||||
| 0x6405 | Ловна хижа | Hunting Lodge | TYP:0x6405 | building=hut | normal | 42.7514720, 27.2207522 |
|
||||
| 0x6406 | Бариера | Barrier/Gate | TYP:0x6406 | barrier=gate | normal | 41.5502286, 24.7455668 |
|
||||
| 0x6407 | Пикник | Picnic Site | TYP:0x6407 | tourism=picnic_site | normal | 41.9156528, 25.3233576 |
|
||||
| 0x6408 | ПСС | Mountain Rescue | TYP:0x6408 | emergency=mountain_rescue | critical | 42.6026416, 23.2414269 |
|
||||
| 0x6409 | Паметник | Monument | TYP:0x6409 | historic=monument | normal | 43.1282043, 23.6027098 |
|
||||
| 0x640a | Опасност | Danger | TYP:0x640a | hazard=yes | normal | 43.1507993, 25.3284001 |
|
||||
| 0x640b | Параклис | Chapel | TYP:0x640b | amenity=place_of_worship | normal | 41.9598985, 24.3688774 |
|
||||
| 0x640c | Мина | Mine Shaft | TYP:0x640c | man_made=mine_shaft | normal | 43.0497336, 22.9328442 |
|
||||
| 0x640d | Лифт | Lift | TYP:0x640d | aerialway=station | normal | 42.6056242, 23.2995987 |
|
||||
| 0x640e | Спирка | Bus Stop/Station | TYP:0x640e | highway=bus_stop | normal | 43.3513427, 25.1430917 |
|
||||
| 0x640f | Гробище, оброк | Graveyard/Tombstone | TYP:0x640f | landuse=cemetery | normal | 42.0345068, 22.4919534 |
|
||||
| 0x6411 | Кула | Tower | TYP:0x6411 | man_made=tower | normal | 41.7411375, 24.8182654 |
|
||||
| 0x6412 | Начало пътека | Trailhead | TYP:0x6412 | unclassified | normal | — |
|
||||
| 0x6413 | Тунел | Tunnel | TYP:0x6413 | unclassified | normal | 42.6872063, 23.3418703 |
|
||||
| 0x6414 | Чешма | Drinking Water | TYP:0x6414 | amenity=drinking_water | critical | 43.1117034, 23.7205982 |
|
||||
| 0x6415 | Място за бивакуване | Camp Site | TYP:0x6415 | tourism=camp_site | critical | 41.7506647, 25.1023221 |
|
||||
| 0x6416 | Руини | Ruins | TYP:0x6416 | historic=ruins | normal | 41.5940452, 24.3094182 |
|
||||
| 0x6417 | Кладенец | Well | TYP:0x6417 | man_made=water_well | critical | 42.2628808, 24.2223430 |
|
||||
| 0x641a | Църква | Church | TYP:0x641a | amenity=place_of_worship | normal | 42.9599547, 23.0566549 |
|
||||
| 0x641b | Църква | Church | TYP:0x641b | unclassified | normal | — |
|
||||
| 0x641c | Спирка | Bus Stop/Station | TYP:0x641c | highway=bus_stop | normal | 42.8542113, 25.3382707 |
|
||||
| 0x6500 | Каптаж | Reservoir | TYP:0x6500 | man_made=water_intake | critical | 42.0310307, 24.7559094 |
|
||||
| 0x6501 | Ресторант | Restaurant | TYP:0x6501 | amenity=restaurant | normal | 43.3456135, 28.0715060 |
|
||||
| 0x6502 | Брод | River Passage | TYP:0x6502 | ford=yes | normal | 42.7929711, 25.2992177 |
|
||||
| 0x6503 | Кръстопът | Crossroad | TYP:0x6503 | unclassified | normal | 43.4149003, 28.4942007 |
|
||||
| 0x6505 | Магазин | Shop | TYP:0x6505 | unclassified | normal | — |
|
||||
| 0x6506 | Музей | Museum | TYP:0x6506 | tourism=museum | normal | 42.1463013, 24.7487855 |
|
||||
| 0x650b | Зимна маркировка | Winter Marking | TYP:0x650b | unclassified | normal | 42.0000887, 23.3820820 |
|
||||
| 0x650c | Задънена улица | Dead End | TYP:0x650c | unclassified | normal | — |
|
||||
| 0x6511 | Извор | Spring | TYP:0x6511 | natural=spring | critical | 42.8732872, 23.3371925 |
|
||||
| 0x6600 | Кота | Triangulation | TYP:0x6600 | unclassified | terrain | 41.2312603, 22.6785922 |
|
||||
| 0x6601 | Природен феномен | Natural Phenomenon | TYP:0x6601 | unclassified | terrain | 42.9589677, 25.2119064 |
|
||||
| 0x6602 | Гледка | Viewpoint | TYP:0x6602 | tourism=viewpoint | normal | 41.7438412, 23.4641790 |
|
||||
| 0x6603 | Кота | Triangulation | TYP:0x6603 | man_made=survey_point | normal | 42.5520873, 23.8540435 |
|
||||
| 0x6608 | Хотел | Hotel | TYP:0x6608 | tourism=hotel | critical | 42.8794885, 27.8957462 |
|
||||
| 0x6609 | Хижа | Hut | TYP:0x6609 | tourism=alpine_hut | critical | 43.2779145, 27.1159101 |
|
||||
| 0x660a | Заслон | Refuge | TYP:0x660a | amenity=shelter;shelter_type=basic_hut | critical | 42.8157377, 23.8160634 |
|
||||
| 0x660b | Маркировка | Mark | TYP:0x660b | unclassified | normal | 43.9378452, 22.4072385 |
|
||||
| 0x6610 | Поляна | Meadow | TYP:0x6610 | unclassified | terrain | 42.8212094, 26.1542845 |
|
||||
| 0x6613 | Водопад | Waterfall | TYP:0x6613 | waterway=waterfall | normal | 42.7435327, 24.6398234 |
|
||||
| 0x6614 | Скала | Rock | TYP:0x6614 | natural=rock | terrain | 42.7437901, 25.2693915 |
|
||||
| 0x6615 | Пещера | Cave | TYP:0x6615 | natural=cave_entrance | normal | 41.8003178, 23.3853436 |
|
||||
| 0x6617 | Връх | Peak | TYP:0x6617 | natural=peak | normal | 41.7893529, 22.9647946 |
|
||||
| 0x6618 | Дърво | Tree | TYP:0x6618 | natural=tree | normal | 42.7752686, 23.9033318 |
|
||||
| 0x6619 | Праисторически обект | Prehistoric site | TYP:0x6619 | historic=archaeological_site | normal | 42.5703692, 25.3767228 |
|
||||
| 0x661a | Античен обект | Antiquity site | TYP:0x661a | historic=archaeological_site | normal | 41.7153454, 26.0307527 |
|
||||
| 0x661b | Средновековен обект | Medieval site | TYP:0x661b | historic=archaeological_site | normal | 42.0922279, 24.2505813 |
|
||||
|
||||
**Caveats:** “unclassified” means the converter retains provenance but does not forge an amenity to force an icon. `zoom_hint` is nonfunctional on stock OsmAnd unless a style is separately changed. Neither legend-derived geometry nor icons may be redistributed under arbitrary licenses.
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"status": "PASS_preparation_only",
|
||||
"subset": "Vihren region; no national OSM/OBF build performed",
|
||||
"bbox": "23.30,41.67,23.50,41.84",
|
||||
"workers": 4,
|
||||
"total_pois": 1915,
|
||||
"unknown_fallback": 150,
|
||||
"spot_heights_preserved": 296,
|
||||
"unclassified": 0,
|
||||
"hiking_route_relations": 82,
|
||||
"candidate_overlay_segments": 273,
|
||||
"caveat": "Serial route grouping differs (78 vs 82). OBF rendering not validated."
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
{
|
||||
"release": "v0.92",
|
||||
"test_environment": "Linux runtime, C++17, Python 3",
|
||||
"unit_tests": "3/3 CTest cases passed; v0.92 suite 5 individual tests",
|
||||
"source_img_size_bytes": 441942016,
|
||||
"vihren": {
|
||||
"bbox": "23.36,41.70,23.42,41.76",
|
||||
"pois": 221,
|
||||
"ways": 5597,
|
||||
"colored_source_segments": 53,
|
||||
"hiking_route_relations": 17,
|
||||
"physical_highway_segments_reused": 48,
|
||||
"unmatched_overlay_segments": 5,
|
||||
"serial_vs_parallel": "identical counts, mapping, and relation statistics"
|
||||
},
|
||||
"plovdiv": {
|
||||
"bbox": "24.63,42.02,24.67,42.05",
|
||||
"example_pois": [
|
||||
{
|
||||
"type": "0x6602",
|
||||
"lat": 42.0346141,
|
||||
"lon": 24.6528697,
|
||||
"osm": "tourism=viewpoint"
|
||||
},
|
||||
{
|
||||
"type": "0x6500",
|
||||
"lat": 42.0347643,
|
||||
"lon": 24.6522689,
|
||||
"osm": "man_made=water_intake"
|
||||
},
|
||||
{
|
||||
"type": "0x6500",
|
||||
"lat": 42.0340991,
|
||||
"lon": 24.6520114,
|
||||
"osm": "man_made=water_intake"
|
||||
}
|
||||
]
|
||||
},
|
||||
"elevation": {
|
||||
"bbox": "22.29,43.88,22.34,43.93",
|
||||
"spot_height_points": 23,
|
||||
"with_ele_tag": 23
|
||||
},
|
||||
"future_template": {
|
||||
"tested": true,
|
||||
"discovered_poi_types_in_bbox": 21,
|
||||
"discovered_line_types_in_bbox": 16
|
||||
},
|
||||
"obf_build": {
|
||||
"status": "NOT_TESTED",
|
||||
"reason": "Official OsmAndMapCreator is not installed and runtime has no network access"
|
||||
},
|
||||
"stock_icon_visibility": {
|
||||
"status": "NOT_VERIFIED",
|
||||
"reason": "Stock OsmAnd icon and zoom behaviour depends on app version, built-in map style and settings"
|
||||
},
|
||||
"notes": [
|
||||
"zero independent Garmin icon PNGs packaged",
|
||||
"two editable CSV mapping files",
|
||||
"C++ jobs default 1",
|
||||
"Native OSM route relations, no custom phone renderer"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Prefer existing physical highway ways as native hiking-route relation members.
|
||||
Never invent highway=path. Only attach where exact coordinate-node edge coverage >=90%
|
||||
and physical-way overrun <=10%; otherwise preserve original overlay member.
|
||||
"""
|
||||
import argparse,collections,json,pathlib,xml.etree.ElementTree as ET
|
||||
from native_translate import mapping,tagdict,top_level_elements
|
||||
|
||||
def edges(refs):
|
||||
for a,b in zip(refs,refs[1:]):
|
||||
if a!=b:yield (a,b) if a<b else (b,a)
|
||||
|
||||
def match(src,dst,line_cfg,report):
|
||||
candidates={c for c,r in mapping(line_cfg).items() if r.get('route_mode')=='relation'}
|
||||
overlay_refs={};lookup=collections.defaultdict(set)
|
||||
with open(src,'rb') as f:
|
||||
for e in top_level_elements(f):
|
||||
if e.tag=='way':
|
||||
t=tagdict(e)
|
||||
if t.get('garmin:type') in candidates and t.get('garmin:feature')=='extended_polyline':
|
||||
oid=int(e.get('id'))
|
||||
ns=[int(n.get('ref')) for n in e.findall('nd')]
|
||||
if len(ns)>=2:
|
||||
overlay_refs[oid]=ns
|
||||
for edge in edges(ns):lookup[edge].add(oid)
|
||||
e.clear()
|
||||
elif e.tag in ('node','relation'):e.clear()
|
||||
matches=collections.defaultdict(collections.Counter);physical_len={};written=0
|
||||
with open(src,'rb') as f,open(dst,'w',encoding='utf-8') as out:
|
||||
out.write('<?xml version="1.0" encoding="UTF-8"?>\n<osm version="0.6" generator="garmin2osm-v0.92" upload="never">\n')
|
||||
replaced={}; relations=0
|
||||
for e in top_level_elements(f):
|
||||
if e.tag=='way':
|
||||
t=tagdict(e)
|
||||
if 'highway' in t:
|
||||
pid=int(e.get('id'));ns=[int(n.get('ref')) for n in e.findall('nd')]
|
||||
physical_len[pid]=max(0,len(ns)-1)
|
||||
for edge in edges(ns):
|
||||
for oid in lookup.get(edge,()):matches[oid][pid]+=1
|
||||
elif e.tag=='relation':
|
||||
if not replaced:
|
||||
for oid,refs in overlay_refs.items():
|
||||
nseg=max(1,len(refs)-1)
|
||||
for pid,matched in matches.get(oid,{}).most_common():
|
||||
# Not allowed to extend a colored marking beyond its actual physical segment.
|
||||
if matched/nseg>=0.9 and physical_len.get(pid,10**9)<=nseg*1.1:
|
||||
replaced[oid]=pid;break
|
||||
tt=tagdict(e)
|
||||
if tt.get('route')=='hiking' and tt.get('bgm:source_feature')=='extended_polyline':
|
||||
seen=set();relations+=1
|
||||
for m in list(e.findall('member')):
|
||||
val=int(m.get('ref'))
|
||||
ref=str(replaced.get(val,val))
|
||||
if ref in seen:e.remove(m)
|
||||
else:m.set('ref',ref);seen.add(ref)
|
||||
out.write(ET.tostring(e,encoding='unicode')+'\n');written+=1;e.clear()
|
||||
out.write('</osm>\n')
|
||||
summary={'candidate_overlay_ways':len(overlay_refs),'attached_to_existing_highway':len(replaced),
|
||||
'unmatched_overlay_members':len(overlay_refs)-len(replaced),'hiking_relations':relations,
|
||||
'policy':'Attach only with >=90% shared physical segments and <=10% physical overrun. No invented routable ways.'}
|
||||
pathlib.Path(report).write_text(json.dumps(summary,ensure_ascii=False,indent=2)+'\n',encoding='utf-8')
|
||||
return summary
|
||||
|
||||
def main():
|
||||
p=argparse.ArgumentParser(description=__doc__)
|
||||
p.add_argument('--osm',required=True);p.add_argument('--out',required=True);p.add_argument('--lines',required=True);p.add_argument('--report',required=True)
|
||||
a=p.parse_args();print(json.dumps(match(a.osm,a.out,a.lines,a.report),ensure_ascii=False,indent=2))
|
||||
if __name__=='__main__':main()
|
||||
Executable
+278
@@ -0,0 +1,278 @@
|
||||
#!/usr/bin/env python3
|
||||
"""v0.93: streaming IMG/OSM native-OsmAnd translation and honest QA.
|
||||
Only user-editable controls: config/poi-mapping.csv and config/line-mapping.csv.
|
||||
"""
|
||||
import argparse,base64,collections,csv,hashlib,json,pathlib,re,sqlite3,sys,xml.etree.ElementTree as ET
|
||||
from xml.sax.saxutils import escape,quoteattr
|
||||
|
||||
def top_level_elements(source):
|
||||
"""Bounded-memory XML pull iterator. ElementTree .clear() alone leaves one
|
||||
empty Element per processed object in the root; clearing the root prevents
|
||||
nationwide Garmin files from exhausting RAM during multi-pass conversion.
|
||||
Callers must fully process each element before requesting the next.
|
||||
"""
|
||||
context=ET.iterparse(source,events=('start','end'))
|
||||
event, root=next(context)
|
||||
if event != 'start' or root.tag != 'osm':
|
||||
raise ValueError('Expected an OSM XML root element')
|
||||
depth=1
|
||||
for event,elem in context:
|
||||
if event=='start':depth+=1;continue
|
||||
if depth==2 and elem.tag in ('node','way','relation'):
|
||||
yield elem
|
||||
root.clear()
|
||||
depth-=1
|
||||
|
||||
def tagdict(elem):return {x.get('k'):x.get('v','') for x in elem.findall('tag')}
|
||||
def set_tags(elem,d):
|
||||
for child in list(elem):
|
||||
if child.tag=='tag':elem.remove(child)
|
||||
for k,v in d.items():
|
||||
if v is not None and v!='':ET.SubElement(elem,'tag',k=k,v=str(v))
|
||||
def read_csv(p):
|
||||
with open(p,encoding='utf-8-sig',newline='') as f:
|
||||
lines=(line for line in f if not line.startswith('#'))
|
||||
yield from csv.DictReader(lines)
|
||||
def code(value):return hex(int(value,0)).lower()
|
||||
def mapping(path):
|
||||
ret={}
|
||||
for row in read_csv(path):
|
||||
try:c=code(row['code'])
|
||||
except Exception as ex:raise ValueError('Invalid code in '+str(path)+': '+str(row)) from ex
|
||||
if c in ret:raise ValueError(f'Duplicate code {c} in {path}')
|
||||
ret[c]=row
|
||||
return ret
|
||||
def tags_from_string(v):
|
||||
d={}
|
||||
for x in v.split(';'):
|
||||
if not x:continue
|
||||
if '=' not in x:raise ValueError(f'Invalid OSM tag {x!r}')
|
||||
k,val=x.split('=',1)
|
||||
if not k or not val:raise ValueError(f'Invalid OSM tag {x!r}')
|
||||
if k in ('name','garmin:type','garmin:feature','garmin:mapset','bgm:generated_name'):
|
||||
raise ValueError('Reserved mapping key: '+k)
|
||||
d[k]=val
|
||||
return d
|
||||
|
||||
# User-requested PRIVATE-map presentation fallback. A locality is a safe stock-map
|
||||
# display tag, but this is synthetically applied to unknown source types and is
|
||||
# therefore NOT suitable for public OSM imports. Elevation annotations are excluded.
|
||||
SEMANTIC_KEYS = frozenset(('amenity','tourism','emergency','natural','man_made','place',
|
||||
'historic','shop','leisure','building','waterway','railway','power','aeroway',
|
||||
'highway','hazard','ford','water','water_source','landuse','barrier',
|
||||
'aerialway','craft','military','geological','public_transport','sport'))
|
||||
GENERATED_SUFFIX = re.compile(r' - ([A-Z0-9]{7,16})$')
|
||||
|
||||
def stable_unknown_hash(elem,tags):
|
||||
name=tags.get('name','')
|
||||
m=GENERATED_SUFFIX.search(name) if tags.get('bgm:generated_name') == 'yes' else None
|
||||
if m:return m.group(1) # preserve previously generated stable identity
|
||||
seed='|'.join((tags.get('garmin:mapset',''),tags.get('garmin:type',''),
|
||||
elem.get('lat',''),elem.get('lon','')))
|
||||
return base64.b32encode(hashlib.blake2s(seed.encode('utf-8'),digest_size=7).digest()).decode().rstrip('=')[:11]
|
||||
|
||||
def add_native_unknown_fallback(elem,tags,rule):
|
||||
"""Only unclassified POIs. Named POIs retain their genuine source names."""
|
||||
if any(k in tags for k in SEMANTIC_KEYS):return False
|
||||
if tags.get('garmin:type') == '0x6600' or 'ele' in tags:
|
||||
return False # spot heights and elevation annotations are not unknown places
|
||||
if rule and rule.get('action','').strip().lower()=='drop':return False
|
||||
tags['place']='locality'
|
||||
tags['bgm:synthetic_fallback']='place_locality'
|
||||
tags['bgm:classification']='unverified'
|
||||
name=tags.get('name','').strip()
|
||||
if not name or tags.get('bgm:generated_name')=='yes' or name.startswith('Garmin обект '):
|
||||
tags['name']='Неуточнено - '+stable_unknown_hash(elem,tags)
|
||||
tags['bgm:generated_name']='yes'
|
||||
return True
|
||||
|
||||
def make_cpp_rules(pois,out,labels):
|
||||
"""C++ v0.9 uses these internal stage files; they are NOT user control files."""
|
||||
with open(out,'w',encoding='utf-8',newline='') as f,open(labels,'w',encoding='utf-8',newline='') as g:
|
||||
w=csv.writer(f);w.writerow(['code','name_contains','mode','tags','note'])
|
||||
l=csv.writer(g);l.writerow(['code','label_bg'])
|
||||
for c,row in sorted(pois.items(),key=lambda t:int(t[0],0)):
|
||||
action=row.get('action','replace').strip().lower()
|
||||
if action not in ('replace','append','raw','drop'):raise ValueError('Invalid action for '+c)
|
||||
vals=tags_from_string(row.get('osm_tags',''))
|
||||
if action=='raw':action='replace';vals={}
|
||||
if action=='drop':vals={}
|
||||
w.writerow([c,'',action,';'.join(k+'='+v for k,v in vals.items()),'v0.92 compiled from poi-mapping.csv'])
|
||||
if row.get('label_bg'):l.writerow([c,row['label_bg']])
|
||||
|
||||
def inspect_counts(osm,pois,lines,out,strict=False):
|
||||
seen=collections.Counter();unresolved=collections.Counter();annotations=collections.Counter();fallback=collections.Counter();bad=collections.Counter();routes=collections.Counter();examples={};category=collections.Counter();n=0
|
||||
with open(osm,'rb') as src:
|
||||
for el in top_level_elements(src):
|
||||
if el.tag=='node':
|
||||
t=tagdict(el)
|
||||
if t.get('garmin:feature')=='poi':
|
||||
n+=1;c=t.get('garmin:type','');seen[c]+=1
|
||||
if c not in pois:bad[c]+=1
|
||||
if t.get('bgm:synthetic_fallback')=='place_locality':fallback[c]+=1
|
||||
if c=='0x6600' and 'ele' in t:
|
||||
annotations['spot_height']+=1
|
||||
elif not any(k in t for k in ('amenity','tourism','emergency','natural','man_made','place','historic','shop','leisure','building','waterway','railway','power','aeroway','highway','ele','hazard','ford','water','water_source','landuse','barrier','aerialway','craft','geological')):
|
||||
unresolved[c]+=1
|
||||
if len(examples.get(c,[]))<5:examples.setdefault(c,[]).append([el.get('lat'),el.get('lon'),t.get('name','')])
|
||||
for k in ('amenity','tourism','emergency','natural','man_made','building','historic','shop','leisure'):
|
||||
if k in t:category[f'{k}={t[k]}']+=1
|
||||
elif el.tag=='relation':
|
||||
t=tagdict(el)
|
||||
if t.get('route')=='hiking':routes[t.get('garmin:type','unknown')]+=1
|
||||
el.clear()
|
||||
summary={'status':'FAIL' if strict and (bad or unresolved) else ('WARN' if bad or unresolved else 'PASS'),
|
||||
'total_pois':n,'mapped_types':len(pois),'observed_types':len(seen),
|
||||
'source_codes_not_in_config':dict(bad),'unclassified_counts':dict(unresolved),
|
||||
'unclassified_total':sum(unresolved.values()),'synthetic_locality_fallback_total':sum(fallback.values()),'synthetic_locality_counts':dict(fallback),'annotations_not_missing_pois':dict(annotations),'route_relation_counts':dict(routes),
|
||||
'native_semantic_categories':dict(category.most_common(100)),
|
||||
'sample_coordinates':examples,
|
||||
'warning':'OSM audit only; stock icons/search not guaranteed. Synthetic localities are private-map fallback, not valid public OSM classification. Elevation-only objects deliberately have no artificial POI icon.'}
|
||||
pathlib.Path(out).write_text(json.dumps(summary,ensure_ascii=False,indent=2)+'\n',encoding='utf-8')
|
||||
return summary
|
||||
|
||||
def transform(source,dest,pois,lines,report,strict=False):
|
||||
"""Streaming write; route metadata held in SQLite so nationwide builds do not need RAM for all nodes/ways.
|
||||
Native route relations are built only from selected Garmin colored overlay ways,
|
||||
NOT from invented highway=path edges.
|
||||
"""
|
||||
dbpath=pathlib.Path(dest).with_suffix('.edges.sqlite')
|
||||
if dbpath.exists():raise FileExistsError(dbpath)
|
||||
db=sqlite3.connect(dbpath)
|
||||
db.execute('PRAGMA synchronous=OFF');db.execute('PRAGMA journal_mode=MEMORY')
|
||||
db.execute('CREATE TABLE edges (code TEXT, way INTEGER, a INTEGER, b INTEGER)')
|
||||
poi_count=0; ways=0; candidate_edges=0; emitted=0; synthetic=0; kinds=collections.Counter();seen=collections.Counter();gaps=collections.Counter()
|
||||
with open(source,'rb') as inp,open(dest,'w',encoding='utf-8') as out:
|
||||
out.write('<?xml version="1.0" encoding="UTF-8"?>\n<osm version="0.6" generator="garmin2osm-v0.93" upload="never">\n')
|
||||
for el in top_level_elements(inp):
|
||||
if el.tag=='node':
|
||||
tags=tagdict(el)
|
||||
if tags.get('garmin:feature')=='poi':
|
||||
poi_count+=1;c=tags.get('garmin:type','')
|
||||
if c in pois:
|
||||
row=pois[c];tags['bgm:legend_bg']=row.get('legend_bg','') or row.get('label_bg','')
|
||||
tags['bgm:priority']=row.get('priority','normal')
|
||||
# only provenance; no false POI semantic category inserted here
|
||||
tags.pop('bgm_poi',None);tags.pop('bgm:synthetic_icon',None)
|
||||
# Unclassified Garmin POIs intentionally remain raw: stock OsmAnd
|
||||
# cannot visually draw a unique icon for each without installed resources.
|
||||
else:gaps[c]+=1;tags.pop('bgm_poi',None)
|
||||
if add_native_unknown_fallback(el,tags,pois.get(c)):
|
||||
synthetic+=1
|
||||
set_tags(el,tags);seen[c]+=1
|
||||
out.write(ET.tostring(el,encoding='unicode')+'\n');emitted+=1
|
||||
elif el.tag=='way':
|
||||
ways+=1;tags=tagdict(el)
|
||||
if tags.get('garmin:feature')=='extended_polyline' or 'bgm_route' in tags:
|
||||
c=tags.get('garmin:type','')
|
||||
r=lines.get(c)
|
||||
tags.pop('bgm_route',None)
|
||||
# Raw geometry and colour may still be needed for diagnosis;
|
||||
# no duplicate road geometry or misleading water/terrain categories.
|
||||
if r:
|
||||
tags['bgm:line_legend']=r.get('legend_bg','')
|
||||
tags['bgm:translation_status']=r.get('route_confidence','unknown')
|
||||
tags.update(tags_from_string(r.get('osm_way_tags','')))
|
||||
if r.get('route_mode','').lower()=='relation':
|
||||
nref=[int(x.attrib['ref']) for x in el.findall('nd')]
|
||||
if len(nref)>=2:
|
||||
db.execute('INSERT INTO edges VALUES (?,?,?,?)',(c,int(el.get('id')),nref[0],nref[-1]));candidate_edges+=1
|
||||
else:gaps[c]+=1
|
||||
set_tags(el,tags)
|
||||
out.write(ET.tostring(el,encoding='unicode')+'\n');emitted+=1
|
||||
elif el.tag=='relation':
|
||||
out.write(ET.tostring(el,encoding='unicode')+'\n');emitted+=1
|
||||
el.clear()
|
||||
db.commit()
|
||||
# Use topological unions on endpoint node refs to avoid one nationwide route per color.
|
||||
grouped=collections.defaultdict(list)
|
||||
for c,w,a,b in db.execute('SELECT code,way,a,b FROM edges ORDER BY code,way'):
|
||||
grouped[c].append((w,a,b))
|
||||
relations=0;components=0;largest=0
|
||||
for c,edges in sorted(grouped.items()):
|
||||
parent=list(range(len(edges)));rank=[0]*len(edges)
|
||||
def root(i):
|
||||
while parent[i]!=i:
|
||||
parent[i]=parent[parent[i]];i=parent[i]
|
||||
return i
|
||||
def union(a,b):
|
||||
ra,rb=root(a),root(b)
|
||||
if ra==rb:return
|
||||
if rank[ra]<rank[rb]:ra,rb=rb,ra
|
||||
parent[rb]=ra
|
||||
if rank[ra]==rank[rb]:rank[ra]+=1
|
||||
endpoint={}
|
||||
for i,(w,a,b) in enumerate(edges):
|
||||
for n in (a,b):
|
||||
if n in endpoint:union(i,endpoint[n])
|
||||
else:endpoint[n]=i
|
||||
comps=collections.defaultdict(list)
|
||||
for i,(w,a,b) in enumerate(edges):comps[root(i)].append(w)
|
||||
for members in comps.values():
|
||||
components+=1;largest=max(largest,len(members));relations+=1
|
||||
# Deterministic, negative IDs because prior C++ IDs are positive
|
||||
rel=ET.Element('relation',id=str(-relations),version='1')
|
||||
for wid in sorted(members):ET.SubElement(rel,'member',type='way',ref=str(wid),role='')
|
||||
row=lines[c]
|
||||
tags={'type':'route','route':'hiking','network':row.get('network') or 'lwn',
|
||||
'garmin:type':c,'bgm:source_feature':'extended_polyline',
|
||||
'bgm:confidence':row.get('route_confidence','color_only')}
|
||||
if row.get('osmc_symbol'):tags['osmc:symbol']=row['osmc_symbol']
|
||||
if row.get('legacy_color'):tags['colour']=row['legacy_color']
|
||||
# Use a non-confusing internal identifier, NOT a fabricated official route name.
|
||||
tags['ref']='BG'+c.replace('0x','').upper()
|
||||
set_tags(rel,tags)
|
||||
out.write(ET.tostring(rel,encoding='unicode')+'\n')
|
||||
out.write('</osm>\n')
|
||||
db.close();dbpath.unlink(missing_ok=True)
|
||||
result={'pois':poi_count,'synthetic_locality_fallbacks':synthetic,'ways':ways,'route_candidate_ways':candidate_edges,
|
||||
'route_relations':relations,'route_components':components,'largest_component_ways':largest,
|
||||
'source_types_missing_mapping':dict(gaps),'notes':['Synthetic place=locality on unknown features is PRIVATE-map-only' ,'Native OSM route relations; no custom phone-side stylesheet','Display zoom remains controlled by stock OsmAnd, not config zoom_hint','Route symbol derived from Garmin color; color_only mappings require review']}
|
||||
pathlib.Path(report).write_text(json.dumps(result,ensure_ascii=False,indent=2)+'\n',encoding='utf-8')
|
||||
return result
|
||||
|
||||
def template(catalog,poi_file,line_file,osm=None):
|
||||
"""Future map: source-driven, abstains from guessing semantic identities."""
|
||||
if pathlib.Path(poi_file).exists() or pathlib.Path(line_file).exists():raise FileExistsError('Refusing to overwrite mapping templates')
|
||||
pr=[]
|
||||
for row in read_csv(catalog):
|
||||
pr.append({'code':row['code'],'source_count':row['count'],'legend_bg':'','legend_en':'',
|
||||
'meaning_confidence':'unknown','osm_tags':'','label_bg':f'Garmin обект {row["code"]}',
|
||||
'priority':'normal','zoom_hint':'17','action':'raw',
|
||||
'note':f'SAMPLE {row.get("example1_lat","")}, {row.get("example1_lon","")}, {row.get("example1_name","")}'})
|
||||
lc=collections.Counter()
|
||||
if osm:
|
||||
with open(osm,'rb') as inp:
|
||||
for el in top_level_elements(inp):
|
||||
if el.tag=='way':
|
||||
t=tagdict(el)
|
||||
if t.get('garmin:feature')=='extended_polyline':lc[t.get('garmin:type','')]+=1
|
||||
el.clear()
|
||||
lr=[{'code':k,'source_count':v,'legend_bg':'','legacy_color':'', 'osm_way_tags':'',
|
||||
'route_mode':'preserve','osmc_symbol':'','network':'lwn','route_confidence':'unknown',
|
||||
'priority':'normal','note':'Unmapped type: check Garmin legend and geometry before enabling'} for k,v in sorted(lc.items())]
|
||||
with open(poi_file,'w',encoding='utf-8',newline='') as f:
|
||||
w=csv.DictWriter(f,fieldnames=list(pr[0]) if pr else ['code','source_count','legend_bg','legend_en','meaning_confidence','osm_tags','label_bg','priority','zoom_hint','action','note']);w.writeheader();w.writerows(pr)
|
||||
with open(line_file,'w',encoding='utf-8',newline='') as f:
|
||||
w=csv.DictWriter(f,fieldnames=list(lr[0]) if lr else ['code','source_count','legend_bg','legacy_color','osm_way_tags','route_mode','osmc_symbol','network','route_confidence','priority','note']);w.writeheader();w.writerows(lr)
|
||||
return {'poi_types':len(pr),'line_types':len(lr)}
|
||||
|
||||
def main():
|
||||
p=argparse.ArgumentParser(description=__doc__);sub=p.add_subparsers(dest='mode',required=True)
|
||||
for cmd in ['prepare','transform','audit','template']:
|
||||
s=sub.add_parser(cmd);s.add_argument('--poi',required=True);s.add_argument('--lines',required=True)
|
||||
if cmd=='prepare':s.add_argument('--out',required=True)
|
||||
elif cmd=='transform':s.add_argument('--osm',required=True);s.add_argument('--out',required=True);s.add_argument('--report',required=True)
|
||||
elif cmd=='audit':s.add_argument('--osm',required=True);s.add_argument('--out',required=True);s.add_argument('--strict',action='store_true')
|
||||
else:s.add_argument('--catalog',required=True);s.add_argument('--osm')
|
||||
a=p.parse_args()
|
||||
if a.mode=='template':res=template(a.catalog,a.poi,a.lines,a.osm)
|
||||
else:
|
||||
ps=mapping(a.poi);ls=mapping(a.lines)
|
||||
if a.mode=='prepare':
|
||||
out=pathlib.Path(a.out);out.mkdir(parents=True,exist_ok=True)
|
||||
make_cpp_rules(ps,out/'cpp-poi-override.csv',out/'cpp-name-labels.csv');res={'poi_types':len(ps),'line_types':len(ls)}
|
||||
elif a.mode=='transform':res=transform(a.osm,a.out,ps,ls,a.report)
|
||||
else:res=inspect_counts(a.osm,ps,ls,a.out,a.strict)
|
||||
print(json.dumps(res,ensure_ascii=False,indent=2));return 0
|
||||
if __name__=='__main__':sys.exit(main())
|
||||
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env python3
|
||||
"""v0.92 offline tests: mapping discipline, native routes, names, future-map templates."""
|
||||
import csv,importlib.util,json,pathlib,subprocess,sys,tempfile,unittest,xml.etree.ElementTree as ET
|
||||
ROOT=pathlib.Path(__file__).resolve().parents[1]
|
||||
spec=importlib.util.spec_from_file_location('translator',ROOT/'scripts/native_translate.py')
|
||||
t=importlib.util.module_from_spec(spec);spec.loader.exec_module(t)
|
||||
POI=ROOT/'config/poi-mapping.csv';LINES=ROOT/'config/line-mapping.csv'
|
||||
class Native092(unittest.TestCase):
|
||||
def test_native_source_semantics(self):
|
||||
rows=t.mapping(POI)
|
||||
self.assertGreaterEqual(len(rows),263)
|
||||
for c,tags in [('0x6414','amenity=drinking_water'),('0x6511','natural=spring'),('0x6500','man_made=water_intake'),('0x660a','amenity=shelter'),('0x6609','tourism=alpine_hut'),('0x6401','man_made=bridge')]:
|
||||
self.assertIn(tags,rows[c]['osm_tags'])
|
||||
self.assertEqual(rows['0x6402']['label_bg'],'Заслон - Сграда')
|
||||
self.assertIn(rows['0x6600']['action'],('append','raw'))
|
||||
self.assertFalse(any('leisure=marina' in r['osm_tags'] for r in rows.values()))
|
||||
self.assertFalse(any('tourism=information;information=board' in r['osm_tags'] for r in rows.values()))
|
||||
self.assertTrue(all('garmin' not in r['osm_tags'] for r in rows.values()))
|
||||
self.assertTrue(all(int(rows[k]['zoom_hint'])<=12 for k in ('0x6414','0x6511','0x6500','0x660a')))
|
||||
def test_config_only_two_and_types(self):
|
||||
self.assertEqual(set(p.name for p in (ROOT/'config').glob('*.csv')),{'poi-mapping.csv','line-mapping.csv'})
|
||||
self.assertTrue(any(x.get('route_mode')=='relation' for x in t.mapping(LINES).values()))
|
||||
for line in t.mapping(LINES).values():
|
||||
if line['route_mode']=='relation':self.assertTrue(line['osmc_symbol'])
|
||||
def test_native_relation_members_and_poi_provenance(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
d=pathlib.Path(directory);src=d/'sample.osm';dst=d/'native.osm';report=d/'report.json'
|
||||
line=next(r for r in t.mapping(LINES).values() if r['route_mode']=='relation')
|
||||
typ=line['code']
|
||||
xml=['<?xml version="1.0" encoding="UTF-8"?><osm version="0.6">']
|
||||
for i in range(1,5):xml.append(f'<node id="{i}" lat="41.{i}" lon="23.{i}"/>')
|
||||
xml+=['<node id="100" lat="41.7" lon="23.4"><tag k="garmin:type" v="0x6414"/><tag k="garmin:feature" v="poi"/><tag k="name" v="Вода - Чешма - HASH"/><tag k="amenity" v="drinking_water"/><tag k="bgm_poi" v="type_0x6414"/></node>']
|
||||
for i,refs in [(10,[1,2]),(11,[2,3])]:
|
||||
xml.append(f'<way id="{i}">'+''.join(f'<nd ref="{n}"/>' for n in refs)+f'<tag k="garmin:feature" v="extended_polyline"/><tag k="garmin:type" v="{typ}"/><tag k="bgm_route" v="type_{typ}"/></way>')
|
||||
xml.append('</osm>');src.write_text('\n'.join(xml))
|
||||
result=t.transform(src,dst,t.mapping(POI),t.mapping(LINES),report)
|
||||
self.assertEqual(result['route_candidate_ways'],2);self.assertEqual(result['route_relations'],1)
|
||||
root=ET.parse(dst).getroot();rel=root.find('relation');tags=t.tagdict(rel)
|
||||
self.assertEqual(tags['route'],'hiking');self.assertTrue(tags['osmc:symbol'].endswith('_bar'))
|
||||
self.assertEqual(sorted(m.attrib['ref'] for m in rel.findall('member')),['10','11'])
|
||||
node=next(n for n in root.findall('node') if n.get('id')=='100');p=t.tagdict(node)
|
||||
self.assertEqual(p['amenity'],'drinking_water');self.assertEqual(p['name'],'Вода - Чешма - HASH')
|
||||
self.assertEqual(p['garmin:type'],'0x6414');self.assertNotIn('bgm_poi',p)
|
||||
for way in root.findall('way'):self.assertNotIn('highway',t.tagdict(way))
|
||||
def test_route_attachment_to_real_way_only(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
d=pathlib.Path(directory);src=d/'src.osm';dst=d/'out.osm';rep=d/'report.json'
|
||||
typ=next(row['code'] for row in t.mapping(LINES).values() if row['route_mode']=='relation')
|
||||
parts=['<osm version="0.6">','<node id="1" lat="41" lon="23"/>','<node id="2" lat="41.1" lon="23.1"/>',
|
||||
'<node id="3" lat="41.2" lon="23.2"/>','<way id="22"><nd ref="1"/><nd ref="2"/><nd ref="3"/><tag k="highway" v="path"/></way>',
|
||||
'<way id="33"><nd ref="1"/><nd ref="2"/><nd ref="3"/><tag k="garmin:type" v="'+typ+'"/><tag k="garmin:feature" v="extended_polyline"/></way>',
|
||||
'<relation id="-1"><member type="way" ref="33" role=""/><tag k="route" v="hiking"/><tag k="bgm:source_feature" v="extended_polyline"/></relation></osm>']
|
||||
src.write_text('\n'.join(parts))
|
||||
subprocess.run([sys.executable,str(ROOT/'scripts/attach_native_routes.py'),'--osm',src,'--out',dst,'--lines',LINES,'--report',rep],check=True,capture_output=True)
|
||||
report=json.loads(rep.read_text());self.assertEqual(report['attached_to_existing_highway'],1)
|
||||
rel=ET.parse(dst).getroot().find('relation');self.assertEqual(rel.find('member').get('ref'),'22')
|
||||
def test_template_for_new_map_without_false_defaults(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
d=pathlib.Path(directory)
|
||||
p=d/'inventory.csv';p.write_text('code,count,example1_lat,example1_lon,example1_name\n0x999,2,42.123,24.321,Unknown place\n')
|
||||
xml=d/'one.osm';xml.write_text('<osm><node id="1" lat="42" lon="24"/><node id="2" lat="43" lon="25"/><way id="3"><nd ref="1"/><nd ref="2"/><tag k="garmin:feature" v="extended_polyline"/><tag k="garmin:type" v="0x11009"/></way></osm>')
|
||||
a=d/'newpoi.csv';b=d/'newlines.csv';stats=t.template(p,a,b,xml)
|
||||
self.assertEqual(stats,{'poi_types':1,'line_types':1})
|
||||
q=list(t.read_csv(a))[0];self.assertEqual(q['action'],'raw');self.assertEqual(q['osm_tags'],'');self.assertIn('42.123',q['note'])
|
||||
r=list(t.read_csv(b))[0];self.assertEqual(r['route_mode'],'preserve');self.assertEqual(r['osmc_symbol'],'')
|
||||
if __name__=='__main__':unittest.main()
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Regression coverage for large XML handling and no false-locality fallbacks."""
|
||||
import csv,importlib.util,os,pathlib,sys,tempfile,unittest,xml.etree.ElementTree as ET
|
||||
ROOT=pathlib.Path(__file__).resolve().parents[1]
|
||||
spec=importlib.util.spec_from_file_location('native_translate', ROOT/'scripts/native_translate.py')
|
||||
t=importlib.util.module_from_spec(spec);spec.loader.exec_module(t)
|
||||
class Reliability093(unittest.TestCase):
|
||||
def test_unknowns_receive_private_locality_and_stable_name(self):
|
||||
from tempfile import TemporaryDirectory
|
||||
with TemporaryDirectory() as d:
|
||||
out=pathlib.Path(d)
|
||||
src=out/'input.osm';dst=out/'output.osm';report=out/'report.json'
|
||||
src.write_text('''<osm version="0.6"><node id="5" lat="42.1345678" lon="24.2345678"><tag k="garmin:feature" v="poi"/><tag k="garmin:type" v="0x9999"/><tag k="garmin:mapset" v="1234"/><tag k="bgm:generated_name" v="yes"/><tag k="name" v="Garmin обект 0x9999 - Q3ABCDEFG9"/></node><node id="6" lat="42.2" lon="24.2"><tag k="garmin:feature" v="poi"/><tag k="garmin:type" v="0x6600"/><tag k="ele" v="176"/><tag k="name" v="176"/></node></osm>''')
|
||||
t.transform(src,dst,t.mapping(ROOT/'config/poi-mapping.csv'),t.mapping(ROOT/'config/line-mapping.csv'),report)
|
||||
nodes=ET.parse(dst).getroot().findall('node');a=t.tagdict(nodes[0]);b=t.tagdict(nodes[1])
|
||||
self.assertEqual(a['place'],'locality');self.assertEqual(a['name'],'Неуточнено - Q3ABCDEFG9')
|
||||
self.assertEqual(a['garmin:type'],'0x9999');self.assertEqual(a['bgm:classification'],'unverified')
|
||||
self.assertNotIn('place',b);self.assertEqual(b['ele'],'176')
|
||||
data=t.inspect_counts(dst,t.mapping(ROOT/'config/poi-mapping.csv'),t.mapping(ROOT/'config/line-mapping.csv'),out/'audit.json')
|
||||
self.assertEqual(data['synthetic_locality_fallback_total'],1)
|
||||
|
||||
def test_known_bgm_types_are_not_invented_localities(self):
|
||||
cfg=t.mapping(ROOT/'config/poi-mapping.csv')
|
||||
self.assertEqual(cfg['0x6600']['osm_tags'],'')
|
||||
self.assertNotIn('place=locality',cfg['0x6600']['osm_tags'])
|
||||
self.assertEqual(cfg['0x650b']['osm_tags'],'tourism=information;information=route_marker;hiking=yes')
|
||||
self.assertEqual(cfg['0x660b']['osm_tags'],'tourism=information;information=trail_blaze;hiking=yes')
|
||||
def test_fallback_does_not_overwrite_known_native_types(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
src=pathlib.Path(d)/'before.osm';out=pathlib.Path(d)/'after.osm';rep=pathlib.Path(d)/'rep.json'
|
||||
src.write_text('''<osm><node id="1" lat="42" lon="24"><tag k="garmin:feature" v="poi"/><tag k="garmin:type" v="0x6400"/><tag k="landuse" v="quarry"/><tag k="name" v="Кариера П"/></node><node id="2" lat="42" lon="24"><tag k="garmin:feature" v="poi"/><tag k="garmin:type" v="0x6406"/><tag k="barrier" v="gate"/></node><node id="3" lat="42" lon="24"><tag k="garmin:feature" v="poi"/><tag k="garmin:type" v="0x640d"/><tag k="aerialway" v="station"/></node></osm>''')
|
||||
t.transform(src,out,t.mapping(ROOT/'config/poi-mapping.csv'),t.mapping(ROOT/'config/line-mapping.csv'),rep)
|
||||
nodes=ET.parse(out).getroot().findall('node')
|
||||
self.assertTrue(all('place' not in t.tagdict(e) for e in nodes))
|
||||
self.assertEqual(t.tagdict(nodes[0])['name'],'Кариера П')
|
||||
def test_root_memory_does_not_accumulate(self):
|
||||
import tracemalloc
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path=pathlib.Path(d)/'many.osm'
|
||||
with path.open('w') as f:
|
||||
f.write('<osm version="0.6">\n')
|
||||
for i in range(130000):f.write(f'<node id="{i+1}" lat="42.1" lon="24.2"><tag k="x" v="y"/></node>\n')
|
||||
f.write('</osm>')
|
||||
tracemalloc.start()
|
||||
c=0
|
||||
with open(path,'rb') as f:
|
||||
for e in t.top_level_elements(f):
|
||||
c+=1
|
||||
self.assertEqual(t.tagdict(e)['x'],'y')
|
||||
_,peak=tracemalloc.get_traced_memory();tracemalloc.stop()
|
||||
self.assertEqual(c,130000)
|
||||
self.assertLess(peak,15_000_000, f'Unbounded XML root growth {peak}')
|
||||
def test_audit_spot_height_and_markers(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path=pathlib.Path(d)/'sample.osm';report=pathlib.Path(d)/'report.json'
|
||||
path.write_text('''<osm version="0.6"><node id="1" lat="43" lon="22"><tag k="garmin:feature" v="poi"/><tag k="garmin:type" v="0x6600"/><tag k="ele" v="132"/></node><node id="2" lat="43" lon="22"><tag k="garmin:feature" v="poi"/><tag k="garmin:type" v="0x650b"/><tag k="tourism" v="information"/><tag k="information" v="route_marker"/></node><node id="3" lat="43" lon="22"><tag k="garmin:feature" v="poi"/><tag k="garmin:type" v="0x6502"/><tag k="ford" v="yes"/></node></osm>''')
|
||||
stats=t.inspect_counts(path,t.mapping(ROOT/'config/poi-mapping.csv'),t.mapping(ROOT/'config/line-mapping.csv'),report)
|
||||
self.assertEqual(stats['unclassified_total'],0)
|
||||
self.assertEqual(stats['annotations_not_missing_pois']['spot_height'],1)
|
||||
if __name__=='__main__':unittest.main()
|
||||
Reference in New Issue
Block a user