Allow internal drivers with no dependencies to be disabled at compile time

[tbonfort]

This PR modifies the autotools build system to allow a user to selectively enable or disable some drivers that were previously always enabled and compiled.

Allows for smaller builds e.g when creating docker images

Example configure invocation to build a gdal image use only for accessing COGs (supposing the result is written to a MEM dataset)

./configure \
        --disable-all-optional-drivers \
	--with-threads     \
	--with-png=internal \
	--with-jpeg=internal \
	--with-libtiff=internal \
	--with-libz=internal \
	--with-curl       \
	--without-bsb     \
	--without-cfitsio     \
	--without-cryptopp     \
	--without-ecw     \
	--without-expat     \
	--without-fme     \
	--without-freexl     \
	--without-gif     \
	--without-gnm     \
	--without-grass     \
	--without-grib     \
	--without-hdf4     \
	--without-hdf5     \
	--without-idb     \
	--without-ingres     \
	--without-jasper     \
	--without-jp2mrsid         \
	--without-kakadu     \
	--without-libgrass     \
	--without-libkml     \
	--without-mrf     \
	--without-mrsid     \
	--without-mysql     \
	--without-netcdf     \
	--without-odbc     \
	--without-ogdi     \
	--without-openjpeg     \
	--without-pcidsk     \
	--without-pcraster     \
	--without-pcre     \
	--without-perl     \
	--without-pg        \
	--without-python     \
	--without-qhull     \
	--without-sde     \
	--without-sqlite3     \
	--without-webp     \
	--without-xerces     \
	--without-xml2 \
	--without-lerc \
	--without-geos

OGR drivers that cannot be disabled and will always be built: generic geojson kml mem mitab vrt

GDAL drivers that cannot be disabled and will always be built: derived gtiff hfa mem raw vrt

[rouault]

Thanks for tackling this. A bit hard to review given the amount of lines changed though

• I have perhaps missed it, but I didn't see where the OGR Curl drivers were enabled only if curl is enabled itself.
• More fundamentally, the reality of driver interdependency is much more complicated. For example frmts/iso8211 is used by a number of other drivers. This is just an example. I'm not sure if we want to track driver interdependency, or just accept (and warn !) that this is a advanced functionality and users must be ready to have compilation failures if they disable wrong combinations of drivers.
• More complicated, and perhaps more theoretical drivers: some drivers or algorithms may require, at runtime, another driver (using GDALGetDriverByName("foo")), but this is generally base drivers like MEM, GTiff, etc. Normally the code properly null-checks the result to error out (since the user can exclude at runtime any driver)
• From a user point of view, I'm not sure the naming --disable-format-foo vs --disable-driver-foo is really needed. Yes this is how things have been traditionnaly divided, bt I'd prefer --disable-driver-foo both for frmts/ and ogr/ drivers (ok, some have the same name, but in that case we can disable/enable both)
• Still from user point of view, I believe we need a --disable-all flag to be used in combination with a few --enable-driver-foo invokation. I guess this is how this functionnality will be used by 99% of people.
• We'll probably need to add in CONTRIBUTING.md guidelines of what is needed now when adding a new driver.

[rouault]

is this change needed ?

[tbonfort]

probably not, I'll check

[tbonfort]

Thanks for tackling this. A bit hard to review given the amount of lines changed though

I'm reworking the change to use looping in configure.ac, should cut down the size of the diff

• I have perhaps missed it, but I didn't see where the OGR Curl drivers were enabled only if curl is enabled itself.

yeah, the gdal and ogr build stretgies are somewhat different. The curl stuff in OGR is (as of now) done by the

ifneq ($(DRIVER_AMIGOCLOUD_ENABLED),no)
   SUBDIRS-$(CURL_SETTING) += amigocloud
 endif

lines in ogrsf_frmts/GnuMakefile. I'm planning on moving that to a more explicit check in the configure script.

• More fundamentally, the reality of driver interdependency is much more complicated. For example frmts/iso8211 is used by a number of other drivers. This is just an example. I'm not sure if we want to track driver interdependency, or just accept (and warn !) that this is a advanced functionality and users must be ready to have compilation failures if they disable wrong combinations of drivers.

Correct, there are many more. I'd be OK with marking this feature as advanced usage. We can also add some specific checks at configure time, but that would have to be done case-by-case. The mitab ogr driver and raw gdal ones are specific cases, as they seem to be used in other places than just other drivers, I'll try to dig a bit more

• More complicated, and perhaps more theoretical drivers: some drivers or algorithms may require, at runtime, another driver (using GDALGetDriverByName("foo")), but this is generally base drivers like MEM, GTiff, etc. Normally the code properly null-checks the result to error out (since the user can exclude at runtime any driver)

I left in mem, vrt and raw drivers for that reason. kml and geojson are also somewhat related to this, and they have been left in. If there are others we can also leave them out of this patch.

• From a user point of view, I'm not sure the naming --disable-format-foo vs --disable-driver-foo is really needed. Yes this is how things have been traditionnaly divided, bt I'd prefer --disable-driver-foo both for frmts/ and ogr/ drivers (ok, some have the same name, but in that case we can disable/enable both)

given that the build setup for gdal and ogr are quite different, handling the disabling of both ogr and gdal drivers with the same name is going to prove a bit tricky. I'll see what I can do.

• Still from user point of view, I believe we need a --disable-all flag to be used in combination with a few --enable-driver-foo invokation. I guess this is how this functionnality will be used by 99% of people.

Agreed. Not sure how to handle this in practice, but I'll have a look also

• We'll probably need to add in CONTRIBUTING.md guidelines of what is needed now when adding a new driver.

Hopefully this will just sum up to adding the name of the directory in configure.ac

[tbonfort]

I've heavily refactored my first draft to take into account your remarks:

• use --disable-driver-* for gdal and ogr
• remove bloat
• check for ogr and gdal drivers with same name

Still remaining to do:

• figure out weird indentation in generated configure

Still remaining to do, but I'd prefer to add this in a following pull request

• support for --disable-all
• look into refactoring "raw" and "mitab" drivers so they can be disabled
• move logic of enabling/disabling optional drivers that require a dependency from the Makefiles to the configure script

[tbonfort]

@rouault I've fixed the indentation part, however I fail to understand why the (only python?) tests are failing on some platforms

[rouault]

A few minor questions/notes, but LGTM !
Once you're done, please squash the commits in a single one.

[rouault]

https://trac.osgeo.org/gdal/wiki/BuildingOnUnixWithMinimizedDrivers will need to be updated

[tbonfort]

PR squashed and ready for merge. --disable-all-drivers has been renamed to --disable-all-optional-drivers.