← Về thư mục
📄 / / profiles / r-and-d / raw_evidence / scientific-agent-skills / skills / geopandas / references / data-io.md

Vector I/O, GeoParquet, Arrow, and PostGIS

GeoPandas 1.x defaults to pyogrio for vector-file I/O. pyogrio and Fiona are bindings to GDAL/OGR; actual formats, options, and behavior depend on the installed native GDAL and drivers.

Local-only intake policy

Do not automatically pass any of these to GeoPandas/GDAL:

GeoPandas officially supports URLs and GDAL supports network/archive virtual filesystems, but that capability crosses network, decompression, parser, and credential trust boundaries. Obtain explicit approval, verify source/hash and size out of band, unpack in a sandbox with resource limits, then process an allowlisted local regular file.

The bundled CLIs reject URL/VSI/archive syntax, symlinks, path traversal, and non-allowlisted suffixes.

Inspect before reading features

For GDAL-backed formats:

from pathlib import Path
import pyogrio

path = Path("approved/input.gpkg")
if not path.is_file() or path.is_symlink():
    raise ValueError("Expected a vetted local regular file")

layers = pyogrio.list_layers(path)
info = pyogrio.read_info(path, layer="approved_layer", force_feature_count=False)
drivers = pyogrio.list_drivers()

Record:

list_drivers() returns capabilities containing r, w, and/or a, but a listed driver is not proof every field/geometry is supported. Treat drivers as an allowlist, not merely a discovered list.

read_file

Stable signature:

gdf = geopandas.read_file(
    filename,
    bbox=None,
    mask=None,
    columns=None,
    rows=None,
    engine="pyogrio",
    use_arrow=True,
)

Rules:

GeoPandas may use HTTP range requests or download an entire URL in memory. This skill therefore does not include remote-read examples.

Filters are not always exact

pyogrio documents:

Do not treat driver FID as a portable stable feature ID.

Writing traditional vector formats

Write a new path and reopen it:

output = Path("derived/result.gpkg")
if output.exists() or output.is_symlink():
    raise FileExistsError("Choose a new output path")

gdf.to_file(
    output,
    layer="result",
    driver="GPKG",
    engine="pyogrio",
    index=False,
    use_arrow=True,
)

roundtrip = geopandas.read_file(output, layer="result", engine="pyogrio")

Never use implicit overwrite/append. Driver behavior varies and multi-file formats complicate atomic writes.

Format tradeoffs

Traditional formats often cannot store lists, structs, arbitrary objects, or multiple geometry columns. Plan conversions and reject silent loss.

GeoParquet and Feather

gdf.to_parquet(
    "derived/result.parquet",
    index=False,
    compression="snappy",
    geometry_encoding="WKB",
    write_covering_bbox=False,
    schema_version="1.0.0",
)

roundtrip = geopandas.read_parquet(
    "derived/result.parquet",
    columns=["feature_id", "geometry"],
)

GeoPandas 1.1.4 write semantics:

Read semantics:

GeoParquet 1.1 metadata requires a geo JSON value, primary_column, and metadata for every geometry column. Geometry columns must be root-level and may be optional; native child coordinates cannot contain nulls. edges defaults to planar. Feature identifiers are outside the core specification, so define and document your own stable-ID metadata/column.

Do not use bbox covering for public sensitive-location data without generalization and approval.

Arrow in memory

GeoPandas 1.0 added to_arrow() and from_arrow() using GeoArrow extension types. These improve interchange but do not make an array self-validating: verify extension metadata, CRS, geometry encoding/type, nulls, and active geometry after round-trip. GeoPandas 1.1 adds to_pandas_kwargs controls for non-geometry Arrow conversion.

PostGIS without credential leakage

Required writing dependencies are SQLAlchemy, GeoAlchemy2, and psycopg/psycopg2. Create connections from named secrets without embedding or logging a connection URL:

import os
from sqlalchemy import URL, create_engine

db_url = URL.create(
    "postgresql+psycopg",
    username=os.environ["GEOPANDAS_POSTGIS_USER"],
    password=os.environ["GEOPANDAS_POSTGIS_PASSWORD"],
    host=os.environ["GEOPANDAS_POSTGIS_HOST"],
    port=int(os.environ["GEOPANDAS_POSTGIS_PORT"]),
    database=os.environ["GEOPANDAS_POSTGIS_DATABASE"],
)
engine = create_engine(db_url)

Read only those named variables (or secret-manager equivalents). Never print db_url, engine.url, exception payloads containing it, or the broader environment.

Use parameterized values and trusted SQL identifiers:

from sqlalchemy import text

query = text(
    "SELECT feature_id, geom FROM approved_schema.features "
    "WHERE category = :category"
)
gdf = geopandas.read_postgis(
    query,
    con=engine,
    geom_col="geom",
    params={"category": approved_category},
    chunksize=10_000,
)

read_postgis infers one CRS from the SRID of the first geometry and assigns it to all rows unless crs= is supplied. Verify all geometries share the expected SRID. With chunksize, it returns an iterator; validate every chunk and enforce a total-row limit.

For writes:

with engine.begin() as connection:
    gdf.to_postgis(
        "derived_features",
        con=connection,
        schema="approved_schema",
        if_exists="fail",
        index=False,
        chunksize=10_000,
    )

Fiona-to-pyogrio migration

GeoPandas 1.0 changed the default engine from Fiona to pyogrio. Differences include:

Set engine= explicitly for reproducibility and test round-trips before migration. Do not assume identical outputs merely because both engines use GDAL.

Export verification and provenance

For every output:

  1. choose a new local path and explicit format/driver/layer;
  2. record source hashes, source layer, stack/native versions, CRS, precision, repair, and transformation choices;
  3. record field names/types/nullability, geometry columns/types, stable ID, index policy, encoding, dimensions, and expected losses;
  4. write, then reopen with an independent code path when feasible;
  5. compare row count, stable-ID set, null/empty/invalid/type counts, CRS, bounds at protected precision, and representative attribute values;
  6. hash the completed artifact and store the audit separately.

Use scripts/vector_inventory.py for redacted intake and scripts/export_plan.py for a non-executing output contract.

Sources (verified 2026-07-23)