Calibrated from GEE execution on 2026-03-29 (actual: 9.22%). Will be validated against QGIS and folia outputs. Percentage of AOI classified as suitable for solar siting. Criteria: slope < 5 degrees, south-facing aspect (135-225), non-forested/non-water land cover. Cross-platform variation from threshold sensitivity and aspect binning.
| Workflow | Model | Backend | Status | Answer | Error | Cost | Latency |
|---|---|---|---|---|---|---|---|
| exec | gold | gee | PASS | 9.22034396053534 | 2.4% | $0.000019 | 351ms |
| exec | gold | folia-rust | PASS | 13.690134882926941 | 52.1% | --- | 92ms |
| exec | gold | qgis | PASS | 14.13641930272552 | 57.1% | --- | 500ms |
Known-correct folia spec for this problem. This is the reference implementation used for backend quality testing.
# Platform Comparison: Weighted Overlay (Solar Siting) — Gold Spec
#
# Identify areas suitable for solar installation in central Utah
# using slope, aspect, and land cover criteria.
# Ground truth: ~9% of AOI.
#
# This spec is designed to run through both the Python backend and
# browser-wasm executor (folia bench exec -b browser-wasm).
name: weighted-overlay
version: "1.0"
description: >
Solar siting analysis via weighted overlay. Combine slope (<5 deg),
south-facing aspect (135-225 deg), and non-forest/non-water land cover
to find suitable areas in central Utah.
Ground truth: ~9% of AOI (calibrated from GEE 2026-03-29).
settings:
default_bbox: [-112.5, 39.5, -111.5, 40.5]
default_crs: EPSG:4326
layers:
# ============================================================
# SOURCE LAYERS
# ============================================================
source/dem:
uri: stac://earth-search/cop-dem-glo-30
type: raster
description: >
Copernicus GLO-30 DEM (30m) for central Utah.
params:
bbox: [-112.5, 39.5, -111.5, 40.5]
source/worldcover:
uri: stac://earth-search/esa-worldcover
type: raster
description: >
ESA WorldCover 2021 (10m) for central Utah.
params:
bbox: [-112.5, 39.5, -111.5, 40.5]
# ============================================================
# COMPUTE: TERRAIN CRITERIA
# ============================================================
compute/slope:
type: raster
description: >
Slope in degrees.
compute:
op: terrain_slope
inputs:
dem: { layer: source/dem }
params:
units: degrees
compute/aspect:
type: raster
description: >
Aspect in degrees (0-360).
compute:
op: terrain_aspect
inputs:
dem: { layer: source/dem }
compute/slope-ok:
type: raster
description: >
Binary mask: slope < 5 degrees.
compute:
op: raster_threshold_mask
inputs:
data: { layer: compute/slope }
params:
threshold: 5
operator: lt
compute/aspect-ok:
type: raster
description: >
Binary mask: south-facing aspect (135-225 degrees).
compute:
op: raster_calc
inputs:
aspect: { layer: compute/aspect }
params:
expression: "where((aspect >= 135) & (aspect <= 225), 1, 0)"
# ============================================================
# COMPUTE: LAND COVER CRITERIA
# ============================================================
compute/lc-ok:
type: raster
description: >
Binary mask: exclude forest (10), water (80), wetland (90).
compute:
op: raster_calc
inputs:
lc: { layer: source/worldcover }
params:
expression: "where((lc != 10) & (lc != 80) & (lc != 90), 1, 0)"
# ============================================================
# COMPUTE: COMBINED SUITABILITY
# ============================================================
compute/suitable:
type: raster
description: >
Combined suitability: all three criteria must be met.
compute:
op: raster_calc
inputs:
slope_ok: { layer: compute/slope-ok }
aspect_ok: { layer: compute/aspect-ok }
lc_ok: { layer: compute/lc-ok }
params:
expression: "slope_ok * aspect_ok * lc_ok"
# ============================================================
# RESULT: SUITABILITY PERCENTAGE
# ============================================================
result/suitable-pct:
type: table
description: >
Percentage of the AOI meeting all suitability criteria.
Ground truth: ~9%.
compute:
op: analysis_zonal_stats
params:
stats: [mean]
inputs:
raster: { layer: compute/suitable }
The prompt given to LLMs in single-shot workflow benchmarks.
Problem: Identify areas suitable for solar installation in central Utah
using a weighted overlay of terrain and land cover data.
Criteria:
- Slope < 5 degrees (from SRTM)
- South-facing aspect: 135-225 degrees (from SRTM)
- Land cover: exclude forest (class 10), water (class 80),
wetland (class 90) from ESA WorldCover
Report the percentage of the AOI that meets all criteria.
Study area: -112.5, 39.5, -111.5, 40.5 (central Utah).
Expected answer: approximately 9% of AOI.