mosaic causes R crash if SpatRasterCollection elements have multiple layers #1524

johnbaums · 2024-05-30T03:55:16Z

R crashes when using mosaic on a SpatRasterCollection with elements having multiple layers. Reprex provided for one of the working cases, below.

library(terra)
set.seed(1)
e <- ext(113, 154, -44, -10)
r <- replicate(2, {
  x <- round(sort(runif(2, e[1], e[2])), 2)
  y <- round(sort(runif(2, e[3], e[4])), 2)
  init(rast(ext(c(x, y)), res = 0.05, crs = "EPSG:4326", nlyrs = 2), runif)
}) |> sprc()
terra::mosaic(r)
## Floating point exception (core dumped)

Interestingly, the mosaic operation completes successfully if we use merge first:

library(terra)
#> terra 1.7.71
set.seed(1)
e <- ext(113, 154, -44, -10)
r <- replicate(2, {
  x <- round(sort(runif(2, e[1], e[2])), 2)
  y <- round(sort(runif(2, e[3], e[4])), 2)
  init(rast(ext(c(x, y)), res = 0.05, crs = "EPSG:4326", nlyrs = 2), runif)
}) |> sprc()
terra::merge(r)
#> class       : SpatRaster 
#> dimensions  : 260, 451, 2  (nrow, ncol, nlyr)
#> resolution  : 0.05, 0.05  (x, y)
#> extent      : 123.89, 146.44, -24.52, -11.52  (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326) 
#> source(s)   : memory
#> names       :        lyr.1,        lyr.2 
#> min values  : 0.0001064336, 8.431729e-06 
#> max values  : 0.9999305937, 9.999035e-01
terra::mosaic(r)
#> class       : SpatRaster 
#> dimensions  : 260, 451, 2  (nrow, ncol, nlyr)
#> resolution  : 0.05, 0.05  (x, y)
#> extent      : 123.89, 146.44, -24.52, -11.52  (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326) 
#> source(s)   : memory
#> names       :        lyr.1,        lyr.2 
#> min values  : 0.0001064336, 8.431729e-06 
#> max values  : 0.9999305937, 9.999035e-01

^{Created on 2024-05-30 with reprex v2.1.0}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.3.2 (2023-10-31)
#>  os       Ubuntu 20.04.6 LTS
#>  system   x86_64, linux-gnu
#>  ui       X11
#>  language (EN)
#>  collate  en_AU.UTF-8
#>  ctype    en_AU.UTF-8
#>  tz       Australia/Melbourne
#>  date     2024-05-30
#>  pandoc   2.5 @ /usr/bin/ (via rmarkdown)
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  ! package     * version date (UTC) lib source
#>  P cli           3.6.2   2023-12-11 [?] RSPM (R 4.3.0)
#>  P codetools     0.2-19  2023-02-01 [?] CRAN (R 4.2.2)
#>  P digest        0.6.34  2024-01-11 [?] RSPM (R 4.3.0)
#>  P evaluate      0.23    2023-11-01 [?] RSPM (R 4.3.0)
#>  P fastmap       1.1.1   2023-02-24 [?] RSPM (R 4.3.0)
#>    fs            1.6.3   2023-07-20 [1] RSPM (R 4.3.2)
#>  P glue          1.7.0   2024-01-09 [?] RSPM (R 4.3.0)
#>  P htmltools     0.5.8.1 2024-04-04 [?] CRAN (R 4.3.2)
#>  P knitr         1.45    2023-10-30 [?] RSPM (R 4.3.0)
#>  P lifecycle     1.0.4   2023-11-07 [?] RSPM (R 4.3.0)
#>  P magrittr      2.0.3   2022-03-30 [?] RSPM (R 4.3.0)
#>  P purrr         1.0.2   2023-08-10 [?] RSPM (R 4.3.0)
#>    R.cache       0.16.0  2022-07-21 [1] RSPM (R 4.3.2)
#>    R.methodsS3   1.8.2   2022-06-13 [1] RSPM (R 4.3.2)
#>    R.oo          1.26.0  2024-01-24 [1] RSPM (R 4.3.2)
#>    R.utils       2.12.3  2023-11-18 [1] RSPM (R 4.3.2)
#>  P Rcpp          1.0.12  2024-01-09 [?] CRAN (R 4.3.2)
#>    renv          1.0.3   2023-09-19 [1] CRAN (R 4.3.2)
#>  P reprex        2.1.0   2024-01-11 [?] CRAN (R 4.3.2)
#>  P rlang         1.1.3   2024-01-10 [?] RSPM (R 4.3.0)
#>  P rmarkdown     2.27    2024-05-17 [?] CRAN (R 4.3.2)
#>  P sessioninfo   1.2.2   2021-12-06 [?] CRAN (R 4.3.2)
#>    styler        1.10.2  2023-08-29 [1] RSPM (R 4.3.2)
#>  P terra       * 1.7-71  2024-01-31 [?] RSPM (R 4.3.0)
#>  P vctrs         0.6.5   2023-12-01 [?] RSPM (R 4.3.0)
#>  P withr         3.0.0   2024-01-16 [?] RSPM (R 4.3.0)
#>  P xfun          0.41    2023-11-01 [?] RSPM (R 4.3.0)
#>  P yaml          2.3.8   2023-12-11 [?] RSPM (R 4.3.0)
#> 
#>  [1] /home/unimelb.edu.au/jbau/23a-v2/renv/library/R-4.3/x86_64-pc-linux-gnu
#>  [2] /home/unimelb.edu.au/jbau/.cache/R/renv/sandbox/R-4.3/x86_64-pc-linux-gnu/9a444a72
#> 
#>  P ── Loaded and on-disk path mismatch.
#> 
#> ──────────────────────────────────────────────────────────────────────────────

Similarly, the mosaic works fine if the elements of the collection have a single layer:

library(terra)
set.seed(1)
e <- ext(113, 154, -44, -10)
r <- replicate(2, {
  x <- round(sort(runif(2, e[1], e[2])), 2)
  y <- round(sort(runif(2, e[3], e[4])), 2)
  init(rast(ext(c(x, y)), res = 0.05, crs = "EPSG:4326", nlyrs = 1), runif)
}) |> sprc()

terra::mosaic(r)
## class       : SpatRaster 
## dimensions  : 403, 444, 1  (nrow, ncol, nlyr)
## resolution  : 0.05, 0.05  (x, y)
## extent      : 123.89, 146.09, -33.26, -13.11  (xmin, xmax, ymin, ymax)
## coord. ref. : lon/lat WGS 84 (EPSG:4326) 
## source(s)   : memory
## name        :     lyr.1 
## min value   : 0.0000000 
## max value   : 0.9994188

The text was updated successfully, but these errors were encountered:

dklinges9 · 2024-12-17T18:30:24Z

Has there been a resolution here? I am facing the same issue of the session crashing (terra version 1.7-78), and it also happens for me when mosaicing two spatRasters:

library(terra) # version 1.7-78
set.seed(1)
e <- ext(113, 154, -44, -10)
r <- replicate(2, {
  x <- round(sort(runif(2, e[1], e[2])), 2)
  y <- round(sort(runif(2, e[3], e[4])), 2)
  init(rast(ext(c(x, y)), res = 0.05, crs = "EPSG:4326", nlyrs = 2), runif)
})

r1 <- r[[1]]
r2 <- r[[2]]

terra::mosaic(r1, r2)

Session Info

sessioninfo::session_info()
─ Session info ────────────────────────────────────────────────────────────────────────────────────────────
setting value
version R version 4.3.2 (2023-10-31)
os macOS Sonoma 14.4.1
system x86_64, darwin20
ui RStudio
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz America/New_York
date 2024-12-17
rstudio 2023.12.0+369 Ocean Storm (desktop)
pandoc 3.1.1 @ /Applications/RStudio.app/Contents/Resources/app/quarto/bin/tools/ (via rmarkdown)

─ Packages ────────────────────────────────────────────────────────────────────────────────────────────────
package * version date (UTC) lib source
abind 1.4-8 2024-09-12 [1] CRAN (R 4.3.3)
askpass 1.2.0 2023-09-03 [1] CRAN (R 4.3.0)
assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.3.0)
cachem 1.1.0 2024-05-16 [1] CRAN (R 4.3.3)
car 3.1-2 2023-03-30 [1] CRAN (R 4.3.0)
carData 3.0-5 2022-01-06 [1] CRAN (R 4.3.0)
cli 3.6.3 2024-06-21 [1] CRAN (R 4.3.3)
codetools 0.2-19 2023-02-01 [1] CRAN (R 4.3.2)
crayon 1.5.2 2022-09-29 [1] CRAN (R 4.3.0)
curl 5.2.2 2024-08-26 [1] CRAN (R 4.3.3)
digest 0.6.33 2023-07-07 [1] CRAN (R 4.3.0)
dplyr 1.1.4 2023-11-17 [1] CRAN (R 4.3.0)
evaluate 0.23 2023-11-01 [1] CRAN (R 4.3.0)
fansi 1.0.6 2023-12-08 [1] CRAN (R 4.3.0)
fastmap 1.2.0 2024-05-15 [1] CRAN (R 4.3.3)
fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.0)
gargle 1.5.2 2023-07-20 [1] CRAN (R 4.3.0)
generics 0.1.3 2022-07-05 [1] CRAN (R 4.3.0)
glue 1.7.0 2024-01-09 [1] CRAN (R 4.3.0)
googleAuthR 2.0.2 2024-05-22 [1] CRAN (R 4.3.3)
googleCloudStorageR 0.7.0 2021-12-16 [1] CRAN (R 4.3.0)
htmltools 0.5.7 2023-11-03 [1] CRAN (R 4.3.0)
httr 1.4.7 2023-08-15 [1] CRAN (R 4.3.0)
jsonlite 1.8.8 2023-12-04 [1] CRAN (R 4.3.0)
knitr 1.45 2023-10-30 [1] CRAN (R 4.3.0)
lattice 0.21-9 2023-10-01 [1] CRAN (R 4.3.2)
lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.3.0)
magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.0)
Matrix 1.6-5 2024-01-11 [1] CRAN (R 4.3.0)
memoise 2.0.1 2021-11-26 [1] CRAN (R 4.3.0)
microclimf 0.2.0 2024-10-24 [1] Github (ilyamaclean/microclimf@ac2c69e)
ncdf4 1.23 2024-08-17 [1] CRAN (R 4.3.3)
openssl 2.2.1 2024-08-16 [1] CRAN (R 4.3.3)
pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.0)
pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.0)
png 0.1-8 2022-11-29 [1] CRAN (R 4.3.0)
processx 3.8.3 2023-12-10 [1] CRAN (R 4.3.0)
ps 1.7.5 2023-04-18 [1] CRAN (R 4.3.0)
R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.0)
Rcpp 1.0.13 2024-07-17 [1] CRAN (R 4.3.3)
reticulate 1.40.0 2024-11-15 [1] CRAN (R 4.3.3)
rgee 1.1.7 2023-09-27 [1] CRAN (R 4.3.0)
rlang 1.1.4 2024-06-04 [1] CRAN (R 4.3.3)
rmarkdown 2.25 2023-09-18 [1] CRAN (R 4.3.0)
rstudioapi 0.16.0 2024-03-24 [1] CRAN (R 4.3.2)
sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0)
sp 2.1-2 2023-11-26 [1] CRAN (R 4.3.0)
terra * 1.7-78 2024-05-22 [1] CRAN (R 4.3.3)
tibble 3.2.1 2023-03-20 [1] CRAN (R 4.3.0)
tidyselect 1.2.1 2024-03-11 [1] CRAN (R 4.3.2)
utf8 1.2.4 2023-10-22 [1] CRAN (R 4.3.0)
vctrs 0.6.5 2023-12-01 [1] CRAN (R 4.3.0)
xfun 0.41 2023-11-01 [1] CRAN (R 4.3.0)
yaml 2.3.10 2024-07-26 [1] CRAN (R 4.3.3)
zip 2.3.0 2023-04-17 [1] CRAN (R 4.3.0)

[1] /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/library

dklinges9 · 2024-12-17T19:02:24Z

An update: such crashing does not occur for me with terra 1.6-53, 1.7-3, and 1.7-29, after some testing. But it does with 1.7-78 and 1.8-5

h-a-graham · 2024-12-17T20:17:45Z

I can reproduce this issue too. Whilst not perfect I found this work around helpful - using your example @dklinges9:

library(terra) # version 1.7-78
#> terra 1.8.0
set.seed(1)
e <- ext(113, 154, -44, -10)
r <- replicate(2, {
  x <- round(sort(runif(2, e[1], e[2])), 2)
  y <- round(sort(runif(2, e[3], e[4])), 2)
  init(rast(ext(c(x, y)), res = 0.05, crs = "EPSG:4326", nlyrs = 2), runif)
})

r1 <- r[[1]]
r2 <- r[[2]]

mosaic_safe <- function(x, y, ...,
                        fun = "mean", filename = "",
                        overwrite = FALSE, wopt = list()) {
  stopifnot(terra::nlyr(x) == terra::nlyr(y))
  terra::rast(lapply(seq_along(1:terra::nlyr(x)), function(i) {
    terra::mosaic(x[[i]], y[[i]], ...,
      fun = fun, filename = filename,
      overwrite = overwrite, wopt = wopt
    )
  }))
}

r3 <- mosaic_safe(r1, r2)
#> ext 0: 123.89 146.44 -24.52 -11.52 
#> ext 1: 129.27 146.42 -18.65 -11.5 
#> ext 0: 123.89 146.44 -24.52 -11.52 
#> ext 1: 129.27 146.42 -18.65 -11.5
plot(r3)

^{Created on 2024-12-17 with reprex v2.1.1}

rhijmans · 2024-12-17T22:14:43Z

@dklinges9 can you please try with the current (CRAN or development) version of terra? For me, your code returns:

>Error: [mosaic] resolution does not match

And I believe that was the underlying problem of all these cases.

dklinges9 · 2024-12-17T23:34:34Z

Indeed with the current CRAN version I receive the same error as Rob with this previous toy example.

Yet I still am facing this issue of crashing upon mosaic with my actual program. I made a smaller example from my real files below, which do indeed have the same resolution.

library(terra) # 1.8-5
# Small download, 2.6 KB
download.file("https://github.com/user-attachments/files/18172434/Archive.zip",
              destfile = paste0(getwd(), "/test.zip"))
unzip("test.zip")
r1 <- rast("r1.tif")
r2 <- rast("r2.tif")
out <- terra::mosaic(r1, r2)

Thanks as always for a speedy reply, Rob. Don't mean to cause trouble. Thanks, Dave

rhijmans · 2024-12-18T02:23:35Z

I believe this has now been fixed. I am sorry it took me so long.

# Version 1.8-15 ## Bug Fixes - `Readrds` Failed For Rasters With Timestep="Seconds" [#1711](Https://github.com/Rspatial/terra/issues/1711) by Pascal Oettli - `divide<SpatVector>` always returned NULL [#1724](rspatial/terra#1724) by Márcia Barbosa - `erase` failed in some cases [#1710](rspatial/terra#1710) by erkent-carb ## enhancements - `bestMatch` now has argument "fun" to allow the use of different distance measures, and a <matrix> method - `wrap` (and `writeRDS`) now captures varnames/longnames [#1719](rspatial/terra#1719) by Andrew Gene Brown - improved raster metadata writing [#1714](rspatial/terra#1714) by Andrew Gene Brown - `vect` and `writeVector` now properly read and write date and datetime data. [#1718](rspatial/terra#1718) by Andrew Gene Brown - improved estimate of available memory on linux systems [#1506](rspatial/terra#1506) by Cedric Rossi # version 1.8-10 Released 2025-01-13 ## bug fixes - `expanse<SpatRaster>(transform=TRUE)` crashed R when the crs was "local". [#1671](rspatial/terra#1671) by Michael Chirico - `patches(values=TRUE)` wrapped around the edges [#1675](rspatial/terra#1675) by Michael Chirico - `spin` now correctly handles spherical coordinates [#1576](rspatial/terra#1576) by jeanlobry - `mosaic` sometimes crashed R [#1524](rspatial/terra#1524) by John Baums, Dave Klinges, and Hugh Graham. - `spatSample` ignored argument "exp" when taking a random sample with na.rm=TRUE on a large raster [#1437](rspatial/terra#1437) by Babak Naimi - `split<SpatVector,SpatVector>` did not work properly [#1619](rspatial/terra#1619) by Michael Sumner - `autocor` improved handling of NA cells for global Moran computation [#1992](rspatial/terra#1592) by Nicholas Berryman - `shade` is more memory-safe. [#1452](rspatial/terra#1452) by Francis van Oordt and Chris English - fixed bug in `rasterize` revealed when using `crop(mask=TRUE)` [#1686](rspatial/terra#1686) by edixon1 - fixed `to_id = NA` bug in `nearest` [#1471](rspatial/terra#1471) by Mats Blomqvist - better handling of date/unit [#1684](rspatial/terra#1684) and [#1688](rspatial/terra#1688) by Andrew Gene Brown - `spatSample(method="regular")` on a raster with one column returned too many samples [#1362](rspatial/terra#1362) by Daniel R Schlaepfer ## enhancements - `plot<SpatVector>` now uses the same default viridis color palette as `plot<SpatRaster>` [#1670](rspatial/terra#1670) by Márcia Barbosa - `relate` now accepts relation="equals" [#1672](rspatial/terra#1672) by Krzysztof Dyba - `init` now accepts additional arguments for function "fun" - better handling of the 32 connections limitation set by the HDF4 library [#1481](rspatial/terra#1481) by Dimitri Falk - When using RStudio a once per session warning is given when using draw, sel or click [#1063](rspatial/terra#1063) by Sergei Kharchenko - `distance<SpatRaster>` from lon and lat lines/polygons computes distance to the edges instead of the nodes [#1462](rspatial/terra#1462) by Derek Friend - `distance<SpatVector,SpatVector>` now works for lon/lat data [#1615](rspatial/terra#1615) by Wencheng Lau-Medrano - using overviews for faster plotting of COGs over http [#1353](rspatial/terra#1353) by Michael Sumner and [#1412](rspatial/terra#1412); and argument `plot(x, overview=)` to change the default behavior. - `extract` with points is now faster for rasters accessed over http [#1504](rspatial/terra#1504) by Krzysztof Dyba - `extract` with many points on very large rasters was slower in compared to doing the same with "raster" (which uses terra for that!) [#1584](rspatial/terra#1584) by Hassan Masoomi - `merge` now has three alternative algorithms [1366](rspatial/terra#1366) by Hassan Masoomi and [#1650](rspatial/terra#1650) by Agustin Lobo ## new - `$<SpatRaster>` can now be used to get a categorical SpatRaster with a different active category - `scale_linear<SpatRaster>` method for linear scaling of cell values between a minimum and maximum value such as 0 and 1 - `distance` and related methods get argument "method" to choose the distance algorithm for lon/lat data [#1677](rspatial/terra#1677) by Márcia Barbosa - `divide<SpatRaster>` and `divide<SpatVector>` methods - `nseg` counts the number of segments in a SpatVector [#1647](rspatial/terra#1674) by Michael Chirico - `extract` argument "search_radius" to extract values from the nearest raster cell that is not `NA` [#873](rspatial/terra#873) by matthewseanmarcus - `combineLevels` to combine the levels of all layers [link](https://stackoverflow.com/questions/79340152/how-to-set-factor-levels-in-a-rast-stack-using-terra-when-different-levels-exi) on SO by Sam

rhijmans closed this as completed in ddd1609 Dec 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mosaic causes R crash if SpatRasterCollection elements have multiple layers #1524

mosaic causes R crash if SpatRasterCollection elements have multiple layers #1524

johnbaums commented May 30, 2024

dklinges9 commented Dec 17, 2024

dklinges9 commented Dec 17, 2024 •

edited

Loading

h-a-graham commented Dec 17, 2024 •

edited

Loading

rhijmans commented Dec 17, 2024

dklinges9 commented Dec 17, 2024

rhijmans commented Dec 18, 2024

mosaic causes R crash if SpatRasterCollection elements have multiple layers #1524

mosaic causes R crash if SpatRasterCollection elements have multiple layers #1524

Comments

johnbaums commented May 30, 2024

dklinges9 commented Dec 17, 2024

dklinges9 commented Dec 17, 2024 • edited Loading

h-a-graham commented Dec 17, 2024 • edited Loading

rhijmans commented Dec 17, 2024

dklinges9 commented Dec 17, 2024

rhijmans commented Dec 18, 2024

dklinges9 commented Dec 17, 2024 •

edited

Loading

h-a-graham commented Dec 17, 2024 •

edited

Loading