8.5 NGA GEOnet Names Server (GNS) Data

This text shows how to download data from NGA GEOnet Names Server (GNS) and import to a MongoDB collection

The GEOnet Names Server (GNS) is the official repository of standard spellings of all foreign geographic names, sanctioned by the United States Board on Geographic Names (US BGN). The database also contains variant spellings (cross-references), which are useful for finding purposes, as well as non-Roman script spellings of many of these names. All the geographic features in the database contain information about location, administrative division, and quality. The database can be used for a variety of purposes, including establishing official spellings of foreign place names, cartography, GIS, GEOINT, and finding places.

8.5.1 – Downloading GNS data

The file containing the data for Brazil, br.zip, was downloaded by clicking in the Download Country Files link to open the Country Files (GNS) page and, then, clicking in the BRAZIL link.

The Country Files (GNS) page contains complete files of Geographic Names for Geopolitical Areas from GNS (ISO/IEC 10646 [Unicode UTF-8] Compliant as of 18 July 2002).

The downloaded file br.zip has a size of 12.2 MB and a total of 117,403 records. Unzipping br.zip into br folder, the following files were extracted:

br_administrative_a.txt
br_hypsographic_t.txt
br_spot_s.txt
br_vegetation_v.txt
br_disclaimer.txt
br_localities_l.txt
br_transportation_r.txt
br_hydrographic_h.txt
br_populatedplaces_p.txt
br.txt (27,402,741 bytes)

8.5.2 – Conversion of GNS data to GeoJSON format

The file br.txt, extracted from br.zip, after being renamed br.csv, was converted to the GeoJSON format using the OGR utility program ogr2ogr from GDAL, which converts simple features data between file formats, as shown bellow:

mv br.txt br.csv
ogr2ogr -f GeoJSON br.geojson br.csv -oo X_POSSIBLE_NAMES=LONG -oo Y_POSSIBLE_NAMES=LAT -oo KEEP_GEOM_COLUMNS=NO

8.5.3 – Creating a collection in MongoDB to import GNS data

To import the data from br.geojson file, the gnsbr collection was created using the commands:

mongo
use reficio;
// Drop the collection, if exists
db.gnsbr.drop();
// Create the collection
db.createCollection("gnsbr");

8.5.4 – Importing data

The file br.geojson produced by ogr2ogr, has a size of 94,093,669 bytes, bigger than the 16MB supported by mongoimport. So, when trying to import the file into MongoDB using the command

mongoimport \
    --stopOnError \
    --db reficio \
    --collection gnsbr \
    --file br.geojson

the following output was produced:

2018-05-20T07:45:05.301-0300    connected to: localhost
2018-05-20T07:45:08.265-0300    [########################] reficio.gnsbr        87.7MB/87.7MB (100.0%)
2018-05-20T07:45:11.265-0300    [########################] reficio.gnsbr        87.7MB/87.7MB (100.0%)
2018-05-20T07:45:13.676-0300    num failures: 1
2018-05-20T07:45:13.676-0300    [########################] reficio.gnsbr        87.7MB/87.7MB (100.0%)
2018-05-20T07:45:13.676-0300    Failed: write tcp 127.0.0.1:35476->127.0.0.1:27017: write: broken pipe
2018-05-20T07:45:13.676-0300    imported 0 documents

To circumvent this error, the Ruby script split_geojson.rb was used to split br.geojson into separate files, one for each feature, in folder br, using the command:

ruby split_geojson.rb br.geojson

To import the 117,403 files, one for each line in the original br.txt file (except the header), created in br folder by split_geojson.rb, the mongoimport.sh script was written as shown bellow

#!/bin/bash
for filename in br/*;
do
mongoimport \
    --stopOnError \
    --db reficio \
    --collection gnsbr \
    --file $filename
done

and executed.

8.5.5 – Example of querying data from gnsbr collection

> use reficio;
switched to db reficio
> db.gnsbr.count();
117403
> db.gnsbr.find(
...     {'features.properties.FULL_NAME_RO' : RegExp("^Aeroporto Santos")}
... ).pretty();
{
        "_id" : ObjectId("5b016fc95bdefb0ac3e42d08"),
        "type" : "FeatureCollection",
        "features" : [
                {
                        "type" : "Feature",
                        "properties" : {
                                "RC" : "1",
                                "UFI" : "10873756",
                                "UNI" : "11856851",
                                "DMS_LAT" : "22:54:22S",
                                "DMS_LONG" : "43:09:56W",
                                "MGRS" : "23KPQ8813865690",
                                "JOG" : "SF23-12",
                                "FC" : "S",
                                "DSG" : "AIRP",
                                "PC" : "",
                                "CC1" : "BR",
                                "ADM1" : "21",
                                "POP" : "",
                                "ELEV" : "",
                                "CC2" : "",
                                "NT" : "D",
                                "LC" : "por",
                                "SHORT_FORM" : "",
                                "GENERIC" : "Aeroporto",
                                "SORT_NAME_RO" : "AEROPORTOSANTOSDUMONT",
                                "FULL_NAME_RO" : "Aeroporto Santos Dumont",
                                "FULL_NAME_ND_RO" : "Aeroporto Santos Dumont",
                                "SORT_NAME_RG" : "SANTOSDUMONT AEROPORTO",
                                "FULL_NAME_RG" : "Santos Dumont, Aeroporto",
                                "FULL_NAME_ND_RG" : "Santos Dumont, Aeroporto",
                                "NOTE" : "",
                                "MODIFY_DATE" : "2010-08-25",
                                "DISPLAY" : "1,2",
                                "NAME_RANK" : "1",
                                "NAME_LINK" : "",
                                "TRANSL_CD" : "",
                                "NM_MODIFY_DATE" : "2012-11-07",
                                "F_EFCTV_DT" : "",
                                "F_TERM_DT" : ""
                        },
                        "geometry" : {
                                "type" : "Point",
                                "coordinates" : [
                                        -43.165664,
                                        -22.90621
                                ]
                        }
                }
        ]
}
>

8.5.6 Seeing Brazil airports on map

To see Brazil airports on the map, was created the gnsbr_airports.py script in Python to generate the GPX (GPS eXchange Format) gnsbr_airports.gpx file. The map is shown below in OpenStreetMap.

8. External data