Format Selection Decision Framework

How to go from data you must publish to a short list of cloud-optimized format candidates

What this framework is

This framework helps you go from “I have geospatial data to publish or migrate” to a short list of cloud-optimized format candidates, and to the layout choices that make each candidate work.

It is a guide to help make informed decisions, and does not prescribe a single correct format.

What you control comes first: whether the data already exist, whether you may reprocess them, and whether a mandate fixes an archival format. This can settle the answer before format even comes up.
The shape of your data narrows the set of candidate formats.
The intended use and a few practical levers rank the remaining options.
You make the final call.

The framework is data- and cloud-provider-neutral. Within data and cloud-provider contexts, a rule may help make a decision. For example, an object-size policy or a benchmark result may influence the choice. These cases are given as worked examples, not rules.

This is a first, deliberately simple version. It aligns on the questions to ask and the order to ask them in. Follow the links to a format page when you need more detail.

What this framework is not

It is not a compliance checklist. No format is disqualified by a single answer.
It does not prescribe a catalog or a metadata standard. See Metadata and cataloging.
It does not replace a benchmark. When two candidates are close, measure them.

Key terms

The framework uses a few terms in a specific way. Each links to the section that develops it.

Term	What it means here
Access pattern	What subsets readers will request: a time series at one point, a spatial window at one date, a full scan. Intent implies an access pattern, and your layout either serves it or fights it. Defined in the guide’s glossary; the layout consequences are in the grouping lever.
Archival-format mandate	A requirement to maintain a specific archival format for preservation, such as HDF5, netCDF or GRIB, whatever cloud-optimized format you publish alongside it.
Control	Whether the choice is yours to make at all: do the data already exist, and may you reprocess them, technically and legally. This is Stage 1 of the framework, and it can end in “not yet possible”.
Ecosystem maturity	How settled a format is, on an L1 to L5 scale from experimental to de-facto standard (see below). It answers “safe default, or bet?”.
Grouping lever	The knobs each format gives you for grouping bytes into data chunks and objects in cloud storage: COG internal tiling, Zarr v3 chunking and sharding, COPC octree nodes, GeoParquet row groups.
Intent	What the data are for: archive and download, processing, visualization, distribution, or data fusion. Most products serve several intents at once.
Interpretation encoding	Whether the format carries units, CRS, nodata and any scale and offset transform in-format, so every reader recovers the same physical value, or leans on a side catalog.
Object size	The size of the objects your layout actually writes to storage. Object stores reward a larger average object and many apply a minimum size policy.
Refinement	A criterion applied inside a format family to rank its members. There are four: intent, the grouping lever, ecosystem maturity, and interpretation encoding.
Topology	The shape of the data: 2D raster, N-D cubes, point cloud, or vector and tabular. Answering what shape are the data? selects a format family, not a format.

Preconditions

This guide already defines what makes a format cloud-optimized. The Welcome page states it as a four-part cloud-optimized format pattern, and the glossary defines cloud-optimized as the ability to read a meaningful part of a file without downloading all of it.

The framework starts from the definition on the Welcome page. Two requirements are worth explaining in more detail, because they decide whether a candidate belongs in the framework at all:

P1. The format must support partial and parallel reads over HTTP. Random access through HTTP range requests is what makes a format usable from object storage. It depends on the rest of the pattern: metadata in a consistent place that a reader can load in a single read, and then addressable chunks, internal tiling, or both. A reader that cannot locate the bytes it needs from the metadata alone cannot fetch only those bytes.

P2. No opaque bundles. An archive format is not disqualified just for being an archive. PMTiles is an archive and it is cloud-optimized, because its internal directory makes each tile addressable. What disqualifies a candidate is an archive with no addressable internal layout, or a layer of external compression wrapped around data that was already cloud-optimized. A zipped Zarr store, a LAZ file without COPC metadata, and a single Parquet file with no row groups all fail this way: the client must fetch the whole object before it can use any part of it.

The two stages

The framework has two stages:

Stage 1 — control: do you have a choice? Do the data already exist? If they exist, are you able (technically, legally) to reprocess them? This stage can end the process. If you cannot reprocess the data and cannot virtualize them, there is no format decision to make yet.

Stage 2 — topology and refinement: which format? You reach this stage only when Stage 1 leaves you free to choose. Here the shape of the data selects a format family: the small group of cloud-optimized formats built for that shape. The refinements below rank the options within a format family.

Four refinements then rank the candidates: intent, the object-size and grouping lever, ecosystem maturity, and interpretation encoding. The last one deserves a close look, because the formats differ on it more than people expect.

flowchart TD
    A([Data to publish or migrate]) --> B{Existing data?}

    B -- No --> C{Archival-format<br/>mandate?}
    C -- Yes --> C1[Optimize the mandated format<br/>+ publish a cloud-optimized copy]
    C -- No --> T{{What shape are the data?}}

    B -- Yes --> D{Can you<br/>reprocess?}
    D -- Yes --> T
    D -- No --> P{Can you repackage<br/>the delivered files?}
    P -- Yes --> P1[Format free,<br/>content fixed as delivered]
    P1 --> T
    P -- No --> E{Virtualizable?}
    E -- Yes --> E1[VirtualiZarr / Kerchunk / Icechunk<br/>limited, better than nothing]
    E -- No --> E2[Not yet possible<br/>say what is missing]

    T --> T1(2D raster<br/>COG, Zarr, PMTiles)
    T --> T2(N-D cubes<br/>GeoZarr, COG, CO-HDF5)
    T --> T3(Point cloud<br/>COPC, GeoParquet)
    T --> T4(Vector / tabular<br/>GeoParquet, FlatGeobuf, PMTiles)

    T1 --> R([Refine: intent, object-size lever,<br/>maturity, interpretation])
    T2 --> R
    T3 --> R
    T4 --> R

    classDef entry   fill:#e7eefb,stroke:#3f6db3,stroke-width:2px,color:#152744
    classDef ask     fill:#fff6dc,stroke:#ad7c00,stroke-width:2px,color:#4a3600
    classDef hinge   fill:#dfeffa,stroke:#2a7fb8,stroke-width:3px,color:#0e3247
    classDef family  fill:#e4f4ea,stroke:#39875f,stroke-width:2px,color:#12301e
    classDef partial fill:#fdeadb,stroke:#c06520,stroke-width:2px,color:#48260a
    classDef refine  fill:#ece6f7,stroke:#7355a6,stroke-width:2px,color:#291d44

    class A entry
    class B,C,D,E,P ask
    class T hinge
    class T1,T2,T3,T4 family
    class C1,P1,E1,E2 partial
    class R refine

The control fork

No existing data. You are designing a new product, so you can choose freely. Go to the topology question.

If an archival-format mandate applies, it adds a second job, and it does not remove your freedom to choose how readers get access. Whatever the mandated format, make it as range-request-friendly as it can be. Where the mandated format is HDF5 or netCDF, this guide has specific guidance on consolidated metadata and chunk sizing. What is genuinely optional is what you do beyond the archival format.

You can wrap the optimized archive in a virtual Zarr layer: cheaper to maintain, but you inherit the archive’s chunk shape, the same limit as existing data you cannot reprocess below. The stronger, recommended option is to also publish a cloud-optimized copy for readers, since a cloud-optimized copy is an opportunity to provide access beyond the archive’s native chunking.

Existing data you can reprocess. You control the production chain, so you can also choose freely. Go to the topology question.

Existing data you cannot reprocess, but can rewrite. You may still be allowed to repackage the delivered files: a plain TIFF becomes a COG, a LAZ file becomes a COPC, a shapefile becomes GeoParquet. The format is a free choice, but the content is fixed at whatever was delivered. Go to the topology question.

Existing data you cannot reprocess. You can often still publish a virtual view over the bytes you have. The Virtual Zarr approach is to use a library (such as VirtualiZarr or Kerchunk) to build an index that points into the original files. That index is then stored in formats such as Icechunk, kerchunk json, or kerchunk parquet. This gives range-based access without rewriting the data. The result is more limited than a native store, but it is much better than nothing.

Neither. Some sources cannot be repackaged or virtualized today. Examples are HDF4, encrypted files, and a single large blob with no internal structure, whether or not it’s compressed. See Not yet possible.

How to tell which branch you are on

Whether you can reprocess or virtualize usually depends on the data and data project context.

You can reprocess if you have the processor, the inputs, and the budget to re-run the production chain. Then format and layout are both free choices: go to topology.

You can repackage if you cannot re-run the chain but you may rewrite the delivered files. A plain TIFF, JPEG2000, or MrSID image becomes a COG, a LAS or LAZ file becomes a COPC, a shapefile or CSV becomes GeoParquet. The rewrite can be drastic and still count as repackaging, not reprocessing: an HDF5 granule can become a COG, and many single-scene COGs can become one Zarr cube, as long as the underlying values are not recomputed. Repackaging is a byte-level rewrite rather than a science reprocessing, so it is often permitted where reprocessing is not. Format is a free choice; the content is fixed at whatever was delivered.

You can virtualize if the bytes you have can be indexed as they are. Virtual Zarr publishes that index without copying the data. Three tests, in order:

Is the data chunked internally? This is the hard requirement. A virtual store cannot change the source’s chunk structure, so a single contiguous block virtualizes into one enormous chunk and buys you no useful partial access.
Are those chunks a workable size? You mostly inherit the source’s chunk size and shape, so its chunking limits which access patterns stay cheap. There is little room to make a store built for one access pattern also serve a very different one well.
Is there a parser for the format? VirtualiZarr and Kerchunk cover the common archival formats, including netCDF4, HDF5, GRIB2, TIFF and FITS.

Otherwise there is no path yet. The usual blockers are HDF4, encrypted files, a codec no parser can pass through, and a single blob with no internal chunking, whether or not it’s compressed. See Not yet possible.

Passing these tests means you can, not that you should. A producer may be technically and legally free to change format and still be held by what its users’ tools expect. See Existing users and their tools.

Topology: what shape are the data?

The shape of the data selects a format family: the set of format candidates the refinement process narrows down. Intent is what decides between the members.

Data shape	Format family	Where each member wins
2D raster, per scene or per tile	COG, Zarr, PMTiles	COG for distribution and visualization; Zarr if the product is really a cube; PMTiles for pre-rendered tile delivery
N-D cubes, for example `time × y × x × band`	[Geo]Zarr, COG, Cloud-Optimized HDF/NetCDF	GeoZarr for analysis across the cube; COG per time step if slices are read alone; cloud-optimized HDF/netCDF where an archival format is mandated or existing tooling must keep working
Point cloud	COPC, GeoParquet	COPC for spatial subsetting and level of detail; GeoParquet if columnar analytics dominate
Vector or tabular	GeoParquet, FlatGeobuf, PMTiles	GeoParquet for joins and analytics; FlatGeobuf for streaming download; PMTiles for tiled web delivery

The question is about topology, not about the format the data arrives in. A netCDF granule that holds a point cloud belongs on the point-cloud branch. See the format table for how traditional formats map to cloud-optimized ones.

Refinements

These refinements help rank candidate formats within a family.

Intent

What are the data for? Archive and download, processing, visualization, distribution, or data fusion. Three points matter:

Visualization is a first-class intent, not a side effect. It has its own layout needs, mainly overviews or a level of detail.
On-the-fly and batched processing pull the layout in opposite directions. On-the-fly access rewards small chunks, fine overviews, and cache locality: a reader fetches a little and wants it back fast. Batched processing rewards large shards or row groups, so that a bulk scan becomes a few large contiguous reads that many workers can run in parallel, instead of thousands of small scattered ones.
Data fusion raises the stakes on encoding. Combining products from different missions or processors means every input must be read back as the same physical value, and subsets from each must line up. Fusion is a form of processing, but it is worth naming separately: it weights interpretation encoding and consistent grids far higher than single-product analysis does, and it usually rewards a tabular or columnar target where joins are cheap.

Most real datasets serve several intents at once. However, the intents often carry different weight in the decision process. The case studies state an intent mix for each product, which is what decides the ranking inside a candidate set.

The object size and grouping lever

Every candidate format has a knob that sets how bytes are grouped into data chunks and chunks grouped into objects:

Format	Grouping lever
COG	internal tiling and overviews
Zarr v3	chunks and shards (chunk and shard)
COPC	octree nodes
GeoParquet	row groups, plus partitioning

Tune the lever to your object store’s object-size policy (S3, Azure Blob Storage, Google Cloud Storage, or an on-premises equivalent). Object stores degrade when they must index very many tiny objects, so most reward a larger average object.

In general, when it comes to reading from object storage, objects can be too large (readers have to download and decompress more data than they need) or too small (readers make many small requests and pay the network latency for each). See also: the datacube guide on tiny chunks and massive chunks.

One example: the CNES Datalake sorts data into tiers by mean object size, with about 32 MB or more for warm disk and about 100 MB or more for cold tape. This serves the general point: find out whether your storage has an object-size policy, and adjust your groupings accordingly.

Some providers publish a threshold, some only charge you as if your objects were larger, and some have no policy at all.

Size and shape are two different questions. How big the objects are is a question for your object store, answered above. How the bytes are arranged inside them (chunk shape, and the order of the dimensions) is an access-pattern question, and an object store provider has no opinion about the shape. A time × y × x cube chunked long in time serves a time series at one point cheaply and a full-scene map expensively; chunked wide in y and x it does the opposite. This guide works it through, with a diagram, in chunk shape vs chunk size.

Shape rarely decides which format you pick, because every candidate in a family lets you set the shape: a COG chooses its tile dimensions, Zarr its chunk shape and dimension order, GeoParquet its row-group size and row ordering. What shape decides is how well the format you picked performs, not which one you pick. And it is mostly settled back at the control question, because only reprocessing lets you choose it freely.

Ecosystem maturity

Ecosystem maturity may influence the final decision. You may need something production-grade that you can count on, or you may have the flexibility to be an early adopter. Maturity is what separates a safe default from a bet.

To keep that judgement explicit rather than impressionistic, this framework assigns candidate formats on the following five-level scale:

Level	Meaning
L5	De-facto standard: universal tooling, no specification risk
L4	Established: released spec, broad independent tooling, in production use
L3	Stable: released spec, multiple implementations, growing tooling
L2	Emerging: spec stabilizing, reference implementations, early adopters
L1	Experimental: draft or in-flux spec, few implementations

Where the candidates sit today:

Format	Specification	Tooling	Level
COG	De-facto standard, on a GeoTIFF base	GDAL, rio-tiler, TiTiler, every GIS	L5
GeoParquet	v1.x, stable, on an Apache Parquet base	GDAL, GeoPandas, DuckDB, Arrow	L4
Zarr	v2 and v3 stable	GDAL, zarr-python, zarrs	L4
COPC	v1.0, stable, on a LAZ base	PDAL, untwine, QGIS, copc.js	L4
FlatGeobuf	Stable, on a flatbuffers base	GDAL and OGR, QGIS, flatgeobuf.js	L3
GeoZarr	Draft, in an OGC Standards Working Group	Emerging, mostly xarray-based stacks	L2

COG is the reference case and the format the others are measured against: a de-facto standard for years, supported by every relevant library and GIS, with no open specification risk.

For GeoParquet and COPC, the risk is integration effort, not the specification. Each inherits a mature base format and adds a spatial index on top.

New Zarr Standards: Distinguishing Zarr v3 and GeoZarr

This distinction is easy to miss, and it decides how much risk you actually take on.

Zarr v3, the substrate of GeoZarr, is dependable. The core specification was finalized in 2024. Sharding, the feature that matters most for object storage, is stable in zarr-python 3.x and readable by GDAL and xarray. Judged on its own, the Zarr v3 core behaves like an L4.
GeoZarr, the convention layer, is still being ratified. It is what makes a generic Zarr store geospatial: CRS, dimension semantics, and multiscale pyramids. It does for Zarr roughly what the GeoTIFF tags do for a plain TIFF. It is a draft in an OGC Standards Working Group, on the standards track but not yet an adopted OGC standard, and the published draft can lag behind active convention work. Check geozarr.org for the current status before you rely on a specific version.

So the L2 above belongs to the convention layer, not to Zarr. The leading reference implementations of GeoZarr today, such as ESA’s EOPF Sentinel output, are demonstrators rather than production guarantees.

Maturity is a warning, not a veto

An L2 candidate can still be the right answer. Multidimensional cubes have no mature format candidate. A low level does not mean “choose something else”. It means plan for change. Budget for convention churn, pin the versions you depend on, and expect visualization tooling in particular to improve as tooling catches up.

Many factors influence what maturity level you can tolerate. A one-off internal product and a mission archive with a twenty-year retention commitment can reasonably land on different levels from the same set of candidate formats.

Interpretation encoding

Does the format carry the information a reader needs to recover the same physical value, or does it lean on a side catalog? Interpretation encoding includes units, the CRS, the nodata fill value (see also Virtualizarr’s distinction of fill values), and any scale and offset transform.

Interpretation encoding separates the candidates more than people expect:

Zarr v3 makes the information first-class. With the scale_offset codec, a declared fill value, Zarr conventions for projection, CRS and multiscales, the transform is a first-class array property. Every reader recovers the same value.
COG, GeoParquet, and COPC standardize the CRS but leave the value transform to side metadata. Readers honor side metadata unevenly.

If your product stores scaled integers, or if an undeclared fill value would change a computed result, weigh this aspect heavily.

Cross-cutting considerations

These considerations do not impact what format you choose, but they do affect how you implement that format.

Metadata and cataloging

Metadata that a reader can fetch in a single request is part of Precondition P1, not an extra. Every candidate the topology question opens has its own mechanism for it: Zarr consolidated metadata, COG headers, GeoParquet footers, and the COPC header. What this section adds is the level above a single file.

At a scale of tens of thousands to millions of files, collection-level discovery becomes critical. STAC has an essential role there, and it is part of the cloud-native geospatial ecosystem that we recommend. It is not a format and not a decision point in this framework, so this page does not prescribe it. Choose a catalog approach that fits your organization.

Within-format depth

Some choices sit below the format and are foundational:

Base codecs, such as ZSTD and Deflate
Data types, and scale-offset encoding
Sharding in Zarr v3

These are mature and in scope. The per-format pages cover them. More experimental optimizations, such as bit-rounding and advanced compression tricks, are out of scope at this time.

Formats also differ in which data types they support, and in whether types can be mixed within one file, column, or schema. Those specifics live on each format’s own page and are worth checking before committing to a candidate; they can rule one out as surely as topology or intent.

Multiscale and native mixed resolution

Providing downsampled levels of original data in the source resolution is a common pattern for providing good performance, usually for visualization. A common example is COG overviews, which provide tiles at standard resolutions for mapping clients. A multiscale pyramid is a more general concept: a set of levels of detail, each with its own resolution, that can be used for visualization or analysis.

Often these multiscale pyramids are generated at a uniform decimation so a reader can zoom out without reading and resampling full-resolution chunks. COG overviews, COPC’s octree levels of detail, and most GeoZarr pyramids work this way, and it is already covered above under ecosystem maturity and the grouping lever.

How these multiscale pyramids are generated can vary: Sometimes, data is provided at varying resolution where lower resolution data is not derived from the higher resolution data. Sentinel-2 is an example, worked through in case 1 of the case studies: Sentinel-2 collects data in 10m, 20m, and 60m bands from the same sensor. These bands can become a pyramid by aligning the decimation to each step (10m to 20m halves the resolution, 20m to 60m divides it by three), and the pyramid can continue past 60 m into ordinary overviews using the same mechanism. Zarr multiscales convention allows for non-uniform scale factors.

This still interacts with the grouping lever, since each native resolution needs its own tiling or chunk shape, and with interpretation encoding, since each band needs its own geotransform. What changes from the plain-overview case is that these become refinements within one multiscale structure.

Existing users and their tools

Being able to reprocess is not the same as being free to change format. If a long-standing user community reads your product with established tools, changing format breaks them, and that cost is real even when nothing technical or legal stops you.

This resembles an archival-format mandate but behaves differently:

	Archival-format mandate	User and tooling expectations
Comes from	Institutional policy	Your consumer community
Negotiable	Rarely	Yes, with notice
Ends when	Never, you satisfy it indefinitely	You finish a migration you announced

In the short term the answer is usually the same: publish the cloud-optimized format alongside what your users already consume. The endpoint differs. A mandate you keep satisfying; an expectation you migrate out of, with a deprecation window and documentation. Weigh it together with ecosystem maturity, because the newer your target format, the more of your users’ toolchain you are asking them to replace.

Not yet possible

When a source cannot be repackaged or virtualized today, the honest answer is “not yet implemented”, not “impossible”. The blocker is usually a missing parser, an unsupported codec, or source files with no internal chunking at all. Every one of those can change.

The precedent is HDF5 itself: once the standard example of a format that could not be read efficiently from object storage, until a change in write strategy and reader tooling made it viable, see Cloud-Optimized HDF/NetCDF. The reader half is still moving: switching xarray’s default cache for the h5netcdf engine improves dataset opening (xarray #11216, measured by its authors on NISAR data). Formats move, and so do the libraries.

Not yet possible does not mean never possible. Re-evaluate the options to pinpoint what needs to change to make cloud-optimized access possible. This could mean revaluating reprocessing, conversion and virtualization.

Optional: make it quantitative

The two stages above get you to a short list. When two or three candidates remain and the choice is close, a transparent score can make the trade-off explicit and quantitative, reducing bias.

cng-formats-benchmark is an open-source (MIT) harness behind the numbers on the Case Studies page. Per format and per dataset it measures the object-size distribution and tier fitness, write throughput, range-request-aware read latency, and tile-display latency. You describe your own datasets in config files, so no provider or dataset is baked into the code, and you run it on your own infrastructure against your own data. See the documentation and the configuration guide.

Worked examples

Six products run through the framework on the Case Studies page. Five are backed by measurements.

One lesson recurs often enough to state up front: going cloud-optimized is rarely smaller than the source you started from. Overviews, levels of detail, and spatial indexes are structures the source did not carry, so they add volume, and that added volume is the feature you are buying.

Where two candidates do differ in size, look at the overviews before you look at the codec. In the measured cases, a codec change moves the total by 0.3 % to 2 %. Adding an overview pyramid to a store that had none adds 28 % to 34 %. The pyramid also costs a COG only bytes, because its overviews sit inside the one file, and it costs a Zarr store objects, because each level is a separate array.

Acknowledgements

This framework was developed as part of a study for the Centre National d’Études Spatiales (CNES), carried out by Development Seed. The case studies use public Copernicus and NASA missions, and the measurements come from that study’s benchmark work.

The framework builds on earlier work in this guide supported by NASA IMPACT (now NASA ODSI). See Our Supporters.