{"slug": "multi-temporal-sentinel-2-super-resolution-by-optimization", "title": "Multi-temporal Sentinel-2 super-resolution by optimization", "summary": "A new PyTorch-based optimization method reconstructs high-resolution Sentinel-2 reflectance scenes from approximately 30 low-resolution observations by exploiting sub-pixel jitter between repeat passes, using Gaussian splat parameterization and analytic integration. The approach achieves 2–5 m resolution without training data or neural networks, but is limited by sensor PSF width and jitter magnitude.", "body_md": "**Jump to: Quick start | How it works | Scripts | Repo layout | Citation**\n\nReconstruct a high-resolution Sentinel-2 reflectance scene from ~30 low-resolution observations of the same area by exploiting the sub-pixel jitter between repeat passes. The scene is parameterized as a continuous field of 2D Gaussian splats; each S2 observation is the analytic integral of that field over shifted pixel footprints convolved with the sensor PSF. We jointly optimize the splat weights and the per-observation sub-pixel shifts in PyTorch. Pure optimization, with no training data and no neural network.\n\nThis repo provides:\n\n- A\n**PyTorch SR solver**() that fits a Gaussian splat field to a stack of Sentinel-2 observations using one of four optimization strategies (`sr.py`\n\n`adam`\n\n,`lbfgs`\n\n,`adam_lbfgs`\n\n,`lbfgs_adam`\n\n), with PSF / TV / scale / coarse-to-fine knobs exposed on the command line. - A\n**parameterized STAC downloader**() that pulls a cloud-free Sentinel-2 L2A time series over an arbitrary AOI / TOI from the Microsoft Planetary Computer and saves the cropped scenes as Cloud-Optimized GeoTIFFs.`download_data.py`\n\n**Figure 1.** A scene near Redmond, WA reconstructed at 8× from 8 Sentinel-2 observations. (**Left**) One raw S2 observation at the original 10 m grid, nearest-neighbor displayed at the SR output size. (** Middle**) Bicubic upsample of the temporal mean across the 8 observations — clean but soft. (** Right**) The Gaussian-splat fit at 1.25 m, recovering edge structure that no single observation contains. Reproduce with `python experiments/psf_sweep.py`\n\n; the per-σ outputs are committed under [ experiments/psf_sweep/](/calebrob6/s2-superres/blob/main/experiments/psf_sweep).\n\nThe achievable resolution gain is bounded by the ratio of the per-observation jitter to the sensor PSF width. For Sentinel-2 the jitter is roughly 1.5 m std and the PSF is roughly 3–5 m wide, so the ~30 shifted observations mostly contain redundant spatial information. Multi-temporal optimization produces clean denoised renders at 2–5 m, but it cannot recover genuine 1 m structure from the data alone. A sharper sensor, larger jitter, or an external structural prior (such as an aerial basemap) would be needed to push further.\n\n```\ngit clone https://github.com/calebrob6/s2-superres.git\ncd s2-superres\npip install -r requirements.txt\n\n# Download the default Redmond, WA dataset (~32 cloud-free scenes, 1024x1024 at 10m)\npython download_data.py\n\n# Run super-resolution with the default lbfgs_adam strategy on a 256x256 crop\npython sr.py --crop 256 --save-png\n\n# Look at the result\nls output/sr_*/\n```\n\nThe default `sr.py`\n\ninvocation runs LBFGS coarse-to-fine to recover the splat weights and per-observation shifts, then Adam sharpens the result with a TV anneal followed by an unregularized phase. On a single GPU a 256 × 256 LR crop takes a few minutes; full ~1000 × 1000 scenes take longer.\n\nTo run on a different area, pass an AOI bbox and a date range to `download_data.py`\n\n:\n\n```\npython download_data.py \\\n    --bbox -122.450,37.700,-122.350,37.800 \\\n    --start 2024-04-01 --end 2024-10-31 \\\n    --output-dir data_sf/\n\npython sr.py --data-dir data_sf/ --crop 256 --save-png\n```\n\nEach LR observation is modeled as\n\n```\ny[t,c,i,j] = sum_k w_k[c] * erf_box(mu_k, sigma_eff, pixel(i,j) - delta_t)\n```\n\nwhere `w_k[c]`\n\nis the per-channel weight of splat `k`\n\n, `erf_box(...)`\n\nis the analytic Gaussian integral over a square LR pixel footprint, and `delta_t`\n\nis a learnable per-observation 2D shift. Both the splat basis and the PSF are Gaussian, so their convolution collapses to `sigma_eff = sqrt(sigma_splat^2 + sigma_psf^2)`\n\nevaluated at the splat-pixel offset. There is no HR pixel grid in the forward model and no numerical integration. The gradient flows analytically through `erf`\n\nto both the weights and the shifts.\n\nThe optimizer picks one of four strategies:\n\n| Strategy | Description |\n|---|---|\n`adam` |\nJoint Adam on weights and shifts at one splat scale. Simple baseline. |\n`lbfgs` |\nBlock-coordinate LBFGS, alternating between weights and shifts, run coarse-to-fine across `--c2f-levels` . |\n`adam_lbfgs` |\nShort Adam warmup at the coarsest level (escapes bad basins), then LBFGS C2F. |\n`lbfgs_adam` |\nLBFGS C2F to convergence, then a long Adam phase that anneals TV to zero and then runs unregularized for sharpening. Default; highest measured edge contrast. |\n\nThe shared core is in [ src/](/calebrob6/s2-superres/blob/main/src):\n\n—`src/splat_field.py`\n\n`SplatField`\n\n`nn.Module`\n\nwith the analytic`erf`\n\nforward model.—`src/strategies.py`\n\n`run_strategy(...)`\n\ndispatches to the four strategies above.—`src/sharpness.py`\n\n`edge_contrast`\n\n,`laplacian_var`\n\n,`gradient_energy`\n\nfor evaluating real-data outputs.— phase-correlation shift initialization with parabolic sub-pixel refinement.`src/shift_estimation.py`\n\n— STAC GeoTIFF stack loader with cloud masking and COG writer.`src/data_loader.py`\n\nSearches the Microsoft Planetary Computer STAC catalog and saves cropped Sentinel-2 L2A scenes as COGs.\n\n```\npython download_data.py \\\n    --bbox -122.186,47.629,-122.056,47.719 \\\n    --start 2025-04-01 --end 2025-10-31 \\\n    --max-cloud-cover 1.0 \\\n    --bands B02,B03,B04,B08 \\\n    --min-scenes 32 \\\n    --output-dir data/\n```\n\nPass `--basemap`\n\nto also fetch an ESRI World Imagery aerial mosaic over the same AOI for visual comparison. Run `python download_data.py --help`\n\nfor the full flag list.\n\n```\npython sr.py \\\n    --data-dir data/ \\\n    --output-dir output/my_run/ \\\n    --strategy lbfgs_adam \\\n    --psf 0.5 \\\n    --tv 1e-3 \\\n    --c2f-levels 2,4,8 \\\n    --n-obs 8 \\\n    --crop 256 \\\n    --save-png\n```\n\nUseful flags:\n\n| Flag | Description |\n|---|---|\n`--strategy` |\nOne of `{adam, lbfgs, adam_lbfgs, lbfgs_adam}` . |\n`--psf` |\nSensor PSF Gaussian σ in LR pixels (Sentinel-2 is approximately 0.4–0.6). |\n`--tv` |\nHuber-TV regularization weight on the splat weights. |\n`--c2f-levels` |\nComma-separated splat-scale ladder for the LBFGS strategies. |\n`--n-obs` |\nNumber of lowest-cloud observations to use. |\n`--crop` |\nCenter-crop size in LR pixels (`0` = full scene). |\n`--save-cog` |\nWrite a georeferenced Cloud-Optimized GeoTIFF (only when `--crop 0` ). |\n\nRun `python sr.py --help`\n\nfor everything.\n\nSweeps `sigma_psf ∈ {0.3, 0.4, 0.5, 0.6, 0.7, 0.8}`\n\nover the central 256 × 256 crop with `lbfgs_adam`\n\nand otherwise-default settings. Writes per-σ RGB PNGs, a 2 × 4 comparison grid (input + bicubic + 6 σ values), an `edge_contrast`\n\nvs `sigma_psf`\n\nplot, and a metrics CSV to [ experiments/psf_sweep/](/calebrob6/s2-superres/blob/main/experiments/psf_sweep). The PNGs in this README come from this script.\n\n```\npython experiments/psf_sweep.py\n```\n\nSee [ experiments/psf_sweep/README.md](/calebrob6/s2-superres/blob/main/experiments/psf_sweep/README.md) for the most recent results.\n\n```\nsr.py                       SR entry point\ndownload_data.py            Parameterized STAC downloader\nsrc/                        Reusable library (SplatField, strategies, sharpness, ...)\nexperiments/                Experiment scripts and their committed PNG/CSV outputs\nimages/                     Figures embedded in this README\n```\n\n`data/`\n\nand `output/`\n\nare gitignored. Fill them locally with `download_data.py`\n\nand `sr.py`\n\nruns.\n\nIf you use this repo, please cite it:\n\n```\n@misc{robinson2026s2superres,\n  author       = {Robinson, Caleb},\n  title        = {{s2-superres}: multi-temporal {Sentinel-2} super-resolution by optimization},\n  year         = {2026},\n  howpublished = {\\url{https://github.com/calebrob6/s2-superres}}\n}\n```\n\nMIT. See [ LICENSE](/calebrob6/s2-superres/blob/main/LICENSE).", "url": "https://wpnews.pro/news/multi-temporal-sentinel-2-super-resolution-by-optimization", "canonical_source": "https://github.com/calebrob6/s2-superres", "published_at": "2026-06-24 09:31:40+00:00", "updated_at": "2026-06-24 09:44:44.845173+00:00", "lang": "en", "topics": ["computer-vision", "machine-learning", "ai-research", "ai-tools"], "entities": ["Sentinel-2", "PyTorch", "Microsoft Planetary Computer", "Redmond"], "alternates": {"html": "https://wpnews.pro/news/multi-temporal-sentinel-2-super-resolution-by-optimization", "markdown": "https://wpnews.pro/news/multi-temporal-sentinel-2-super-resolution-by-optimization.md", "text": "https://wpnews.pro/news/multi-temporal-sentinel-2-super-resolution-by-optimization.txt", "jsonld": "https://wpnews.pro/news/multi-temporal-sentinel-2-super-resolution-by-optimization.jsonld"}}