tolerance <- 0.05Appendix B — Near Equality Tests
This document provides unit tests for the LogoClim NetLogo model. The tests perform near-equality comparisons to validate tolerance expectations during the processing of WorldClim data in NetLogo.
B.1 Problem
LogoClim integrates WorldClim data into NetLogo models through a multi-step conversion process. WorldClim provides raster data in GeoTIFF format, a high-precision binary format for geospatial data, but NetLogo’s GIS extension requires data in Esri ASCII format, a text-based format that stores values with a limited decimal precision. This conversion can introduce numerical differences through two main sources:
- Decimal precision loss: Esri ASCII files store values as text with a fixed number of decimal places, rounding continuous values from the original GeoTIFF.
- Spatial interpolation: When NetLogo’s patch grid doesn’t perfectly align with the raster resolution, the GIS extension applies interpolation that may alter cell values.
These discrepancies propagate to summary statistics when comparing the original WorldClim GeoTIFF data with values extracted from LogoClim patches. The unit tests in this document verify that such differences remain within acceptable tolerance levels, ensuring the reliability of downstream simulations and analyses.
B.2 Methods
B.2.1 Source of Data
Data used in this report come from the following sources:
-
WorldClim
- Historical Climate Data series, including 12 monthly data points representing long-term average climate conditions for the period 1970-2000 (Fick & Hijmans, 2017). It provides averages on minimum, mean, and maximum temperature, precipitation, solar radiation, wind speed, vapor pressure, elevation, and on bioclimatic variables.
- Historical Monthly Weather Data series, including 12 monthly data points for each year from 1951 to 2024. Based on downscaled data from CRU-TS-4.09 (Harris et al., 2020), developed by the Climatic Research Unit at the University of East Anglia. It provides monthly averages for minimum temperature, maximum temperature, and total precipitation.
- Future Climate Data series, including 12 monthly data points from downscaled climate projections derived from CMIP6 models (Eyring et al., 2016) for four future periods: 2021-2040, 2041-2060, 2061-2080, and 2081-2100. The projections cover four SSPs (126, 245, 370, and 585), with data available for average minimum temperature, average maximum temperature, total precipitation, and bioclimatic variables.
-
GADM: Database of Global Administrative Areas
- Data on administrative boundaries and regions (Hijmans, n.d.), utilized for plotting country boundaries and cropping raster datasets.
B.2.2 Data Munging
The data munging followed the data science workflow outlined by Wickham et al. (2023), as illustrated in Figure B.1. All processes were made using the Quarto publishing system (Allaire et al., n.d.), the NetLogo environment, the R programming language (R Core Team, n.d.), and several R packages.
For data manipulation and workflow, priority was given to packages from the Tidyverse, rOpenSci and rspatial ecosystems, as well as other packages adhering to the tidy tools manifesto (Wickham, 2023).
Source: Reproduced from Wickham et al. (2023).
B.2.3 Data Extraction and Transformation
Data extraction was performed using the worldclim_download() function from the orbis R package (Vartanian, 2026c). This function scrapes climate data from the WorldClim website and downloads the relevant GeoTIFF files for the specified variables and time periods.
Following the extraction, the transformation from GeoTIFF to Esri ASCII format is carried out using the worldclim_to_ascii() function from the orbis R package.
B.2.4 NetLogo Integration
Integration with NetLogo (Wilensky, 1999) is facilitated by the logolink R package (Vartanian, 2026b). This package enables the execution of BehaviorSpace experiments directly from R.
Output is extracted in Table and Lists format, containing values, latitude, and longitude for patches, along with global variables that describe the model’s settings.
No Java dependencies are required. NetLogo bundles its own Java Runtime Environment (JRE), ensuring independent operation regardless of the system’s Java installation.
B.2.5 Continuous Integration
The tests use the latest release of NetLogo and are automated using GitHub Actions provided by the LogoActions project (Vartanian, 2026a). Each commit to the code repository triggers test execution, ensuring that changes to the codebase are validated against defined tolerance levels.
B.2.6 Near Equality Tests
The data validation is performed using error tolerance tests with expectations functions from the testthat R package (Wickham, 2011). These tests compare the number of observations (n), minimum, mean, and maximum value of the WorldClim data loaded in LogoClim against the original WorldClim dataset.
Each test begins by selecting a random country to crop the WorldClim data. For each of the three WorldClim series, a random combination of variable, month, year, and other series-specific parameters is drawn using the worldclim_random() function from the orbis R package. No seed is set for the random number generator, so results vary between runs.
Elevation, bioclimatic variables, and the models FIO-ESM-2-0, GFDL-ESM4, and HadGEM3-GC31-LL are excluded due to their data limitations.
B.2.6.1 Minimum Number of Cells
To ensure valid statistical analysis, a minimum number of cells is required. Resolution for each country was determined based on its area, aiming to achieve approximately 1,000 cells. This calculation considers the available resolutions and their approximate cell areas at the Equator:
- 10 minutes (~340 km² per cell)
- 5 minutes (~85 km² per cell)
- 2.5 minutes (~21 km² per cell)
- 30 seconds (~1 km² per cell)
Micronations, like Dominica, were excluded from the analysis due to their small size and limited data availability.
B.2.6.2 Tolerance Level
The all.equal and testthat expect_equal functions were used to perform near-equality tests, both relying on relative tolerance. Comparisons are conducted between the original GeoTIFF file and patch values extracted directly from the LogoClim model.
Relative tolerance is proportional to the value of the quantity being measured. The principle is that larger values can tolerate larger errors.
Given \(x\) and \(y\), relative tolerance can be expressed as:
\[ |x - y| \leq \text{tolerance} \times \max(|x|, |y|) \]
or
\[ \frac{|x - y|}{\max(|x|, |y|)} \leq \text{tolerance} \]
where:
- \(x\) and \(y\) are the values being compared
- \(\text{tolerance}\) is the relative tolerance level
- \(\max(|x|, |y|)\) is the maximum absolute value of \(x\) and \(y\)
For this analysis, the following tolerance level is used:
This means the absolute difference between the numbers can be up to 5% of the larger number’s magnitude. In other words, \(x\) and \(y\) are considered nearly equal if:
\[ \frac{|x - y|}{\max(|x|, |y|)} \leq 0.05 \]
This tolerance may appear high, but it accounts for the maximum cumulative effects of multiple sources of numerical differences, including edge cases encountered during random testing. In practice, observed differences are expected to be substantially smaller than this threshold, reflecting a high degree of agreement between the datasets.
B.2.7 Code Style
The Tidyverse Tidy Tools Manifesto (Wickham, 2023), code style guide (Wickham, n.d.-a) and design principles (Wickham, n.d.-b) were followed to ensure consistency and enhance readability.
B.2.8 Reproducibility
The pipeline is fully reproducible and can be run again at any time. To ensure consistent results, the renv package (Ushey & Wickham, 2025) was used to manage and restore the R environment. See the README file in the code repository to learn how to run it.
B.3 Set Environment
B.3.1 Load Packages
library(brandr)
library(checkmate)
library(cli)
library(dplyr)
library(fs)
library(geodata)
library(ggplot2)
library(here)
library(ISOcodes)
library(knitr)
library(leaflet)
library(logolink)
library(magrittr)
library(moments)
library(orbis) # github.com/danielvartan/orbis
library(patchwork)
library(purrr)
library(sf)
library(stringr)
library(terra)
library(testthat)
library(tidyr)
library(tidyterra)B.3.2 Load Custom Functions
The source code for the functions below can be found in the R directory of the code repository.
here("R", "worldclim_raster.R") |> source()
here("R", "logoclim_raster.R") |> source()
here("R", "print_setup.R") |> source()
here("R", "compare_plots.R") |> source()
here("R", "plot_difference.R") |> source()
here("R", "compare_statistics.R") |> source()
here("R", "test_near_equality.R") |> source()B.3.3 Set Data Directory
The here R package (Müller, 2025) is used to construct file paths relative to the project root directory, ensuring portability across different systems.
A local temporary directory is used to help with debugging and to avoid access privilege issues that can arise when writing outside the project directory. This is particularly relevant for GitHub Actions when using macOS runners.
if (!dir_exists(data_dir)) {
dir_create(data_dir)
} else {
dir_ls(data_dir) |> file_delete()
}B.3.4 Set Initial Variables
Setting the JAVA_TOOL_OPTIONS environment variable is optional, but recommended to avoid unnecessary messages from the Java Media Framework.
Sys.setenv(JAVA_TOOL_OPTIONS = "-Dcom.sun.media.jai.disableMediaLib=true")Setting ZIP_PROGRESS environment variable is required by the zip R package to display progress bars when unzipping data.
Sys.setenv(ZIP_PROGRESS = "true")model_path <- here("nlogox", "logoclim.nlogox")B.4 Select Random Country
B.4.1 Select Country
The list of countries is based on the ISO 3166-1 alpha-3 standard and draw using the ISOcodes R package (Hornik & Buchta, 2025).
country <-
country_names("alpha 3") |>
str_subset("ATA|ISL|MLI|TZA", negate = TRUE) |>
sample(1)country
#> Comoros
#> "COM"B.4.2 Download Country Shape
The rspatial geodata R package (Hijmans et al., 2024) is used to download country shapes from the GADM database (Hijmans, n.d.).
B.4.3 Calculate Shape Area
This while loop filters out micronations. The country’s area is divided by 21 km², the approximate area of a single cell at 2.5-minute resolution at the Equator. This resolution was chosen because it’s the highest available across all three WorldClim datasets (the Historical Monthly Weather Data series tops out at 2.5 minutes). The division ensures the selected country has enough area to yield at least 1,000 cells for analysis.
while (!(shape_area / 21 >= 1000)) {
country <- country_names("alpha 3") |> sample(1)
country_shape <-
country |>
gadm(
level = 0,
path = path(data_dir),
resolution = 1
)
shape_area <-
country_shape |>
expanse(unit = "km") |>
magrittr::extract(1)
}
#> Cached as: /home/runner/work/logoclim/logoclim/.data-temp/gadm/gadm41_ASM_0_pk.rds
#> Cached as: /home/runner/work/logoclim/logoclim/.data-temp/gadm/gadm41_NOR_0_pk.rdsshape_area
#> [1] 325121.2322country
#> Norway
#> "NOR"B.4.4 Visualize Country Shape
The rotate() function from the terra package (Hijmans, 2026) is used to adjust shapes that cross the International Date Line (IDL) (e.g., Russian territory).
This adjustment applies only for visualizing countries on leaflet and does not affect the near-equality tests. The Esri ASCII transformation function (worldclim_to_ascii()), used to prepare data for NetLogo, applies the necessary rotation to ensure proper alignment of the raster data with the model’s patch grid.
leaflet() |>
addProviderTiles(providers$Esri.WorldStreetMap) |>
fitBounds(
lng1 = country_shape_leaflet |>
st_bbox() |>
magrittr::extract("xmin") |>
unname(),
lat1 = country_shape_leaflet |>
st_bbox() |>
magrittr::extract("ymin") |>
unname(),
lng2 = country_shape_leaflet |>
st_bbox() |>
magrittr::extract("xmax") |>
unname(),
lat2 = country_shape_leaflet |>
st_bbox() |>
magrittr::extract("ymax") |>
unname()
) |>
addPolygons(
data = country_shape_leaflet,
fillColor = "transparent",
color = "blue",
weight = 2,
opacity = 1
)