High-Resolution Historical Images: A Massive New Dataset and Super-Resolution Model

## Reviving the Past: The Quest for Crystal-Clear Historical Imagery Historical photographs offer invaluable glimpses into bygone eras, but many suffer from low resolution due to aging film, poor scanning, or limited original quality. Researchers at NYU's Adelphi Lab have tackled this head-on with a massive new dataset and a cutting-edge super-resolution model. This initiative transforms pixelated relics into high-definition treasures, enabling deeper analysis in fields like genealogy, urban planning, and cultural studies. Super-resolution (SR) techniques use AI to upscale images while preserving or enhancing details. Traditional methods like bicubic interpolation often introduce artifacts, but modern deep learning approaches—especially diffusion models—generate photorealistic results. The new work builds on this, focusing specifically on the unique characteristics of historical photos: sepia tones, grain, scratches, and period-specific artifacts. ## A Monumental Dataset: History in High Resolution At the core is the **History in High Resolution (HiHR)** dataset, comprising **2 million images** at **512x512 pixels**. Sourced from public-domain archives like the New York Public Library and Library of Congress, these span from the 1840s to the 1970s, covering portraits, street scenes, architecture, and events. ### Key Dataset Features - **Scale and Diversity**: Over 2M images dwarf previous historical SR datasets (e.g., DIV2K has only 800 training pairs). It includes varied subjects: people (45%), buildings (30%), landscapes (15%), and objects. - **High-Quality Ground Truth**: Images were meticulously scanned at high DPI and cropped to avoid distortions, providing true high-res references for training. - **Metadata Richness**: Each image comes with timestamps, locations, captions, and categories, facilitating targeted research. - **Licensing**: Fully public domain or CC0, ready for unrestricted use. You can access the full dataset via [GitHub](https://github.com/nyu-adelphi/history-in-high-resolution), including download scripts, previews, and evaluation splits (train: 1.8M, val: 100K, test: 100K). **Practical Example**: For urban historians, query images by city (e.g., 50K+ NYC photos) to track architectural evolution. Load a sample: ```bash git clone https://github.com/nyu-adelphi/history-in-high-resolution cd history-in-high-resolution python download.py --split train --num 1000 ``` This dataset addresses a critical gap: most SR benchmarks (Set5, BSD100) use modern photos, failing on historical quirks like emulsion noise or handwritten annotations. ## HiResNet: A Diffusion Powerhouse for Historical SR Trained on HiHR, **HiResNet** is a diffusion-based model excelling at 4x and 8x upscaling from low-res inputs (128x128 or 64x64). Diffusion models iteratively denoise random noise into coherent images, outperforming GANs (e.g., ESRGAN) on fidelity metrics. ### Model Architecture Deep Dive - **Backbone**: U-Net with attention blocks, conditioned on low-res input via FiLM layers. - **Diffusion Process**: 50 denoising steps; uses DDIM sampler for speed. - **Loss Function**: Combines L1 reconstruction, perceptual (VGG), and adversarial losses tailored for historical textures. - **Training Details**: 100K A100-GPU hours; batch size 32; LR 1e-4 with cosine annealing. HiResNet crushes baselines: | Model | PSNR (4x) | SSIM (4x) | LPIPS (4x) | |-------------|-----------|-----------|-------------| | Bicubic | 26.1 | 0.78 | 0.32 | | SwinIR | 28.4 | 0.85 | 0.24 | | HAT | 29.2 | 0.87 | 0.22 | | **HiResNet**| **30.8** | **0.91** | **0.18** | On 8x, gains are even larger. Visually, it recovers faded faces, sharpens signage, and minimizes hallucinations. Implement it easily: ```python from hiresnet import HiResNet from PIL import Image model = HiResNet.from_pretrained('nyu-adelphi/hiresnet-4x') img = Image.open('lowres_historical.jpg') hires = model(img, scale=4) hires.save('superres.jpg') ``` Repo: [HiResNet on GitHub](https://github.com/nyu-adelphi/hiresnet) with inference code, weights (Hugging Face), and training scripts. ## Real-World Applications and Extensions ### 1. **Academic Research** Pair with segmentation models (e.g., SAM) for object detection in crowdsourced history projects. Example: Analyze 19th-century fashion trends by upscaling 10K portraits, then applying CLIP for attribute classification. ### 2. **Cultural Heritage** Museums can upscale archives for VR exhibits. A pilot with the Smithsonian restored 5K Civil War photos, revealing hidden details like uniform insignias. ### 3. **Genealogy and AI Tools** Integrate into apps like Ancestry.com: Users upload blurry heirlooms; HiResNet enhances for facial recognition matches. **Code Snippet for Batch Processing**: ```python import torch from pathlib import Path dataset_path = Path('historical_lowres') model.eval() with torch.no_grad(): for img_path in dataset_path.glob('*.jpg'): img = load_image(img_path) sr_img = model(img) sr_img.save(img_path.parent / f'sr_{img_path.name}') ``` ### 4. **Benchmarking and Fine-Tuning** HiHR serves as a new standard. Fine-tune on domain-specific subsets (e.g., aerial photos for GIS). ## Challenges Overcome and Future Directions Challenges included: - **Domain Shift**: Modern SR models blur historical grain; HiResNet learns to preserve it. - **Data Curation**: Automated filtering removed duplicates/low-quality scans using CLIP similarity. - **Compute Scale**: Distributed training via DeepSpeed. Future: Video SR for historical footage, multilingual captions via mCLIP, and integration with foundation models like Stable Diffusion for inpainting scratches. ## Getting Started: Step-by-Step 1. Clone repos: Dataset [here](https://github.com/nyu-adelphi/history-in-high-resolution), model [here](https://github.com/nyu-adelphi/hiresnet). 2. Install: `pip install hiresnet torch torchvision`. 3. Download subset: Use provided scripts. 4. Run inference: As shown above. 5. Evaluate: PSNR/SSIM scripts included. This resource democratizes high-res history, fueling AI-driven discoveries. Dive in and resurrect the past in stunning detail. --- <div style="text-align: center; margin-top: 2rem;"> <a href="https://www.deeplearning.ai/the-batch/history-in-hi-res/" target="_blank" rel="noopener noreferrer" class="view-full-resource-btn" style="display: inline-block; background-color: #f97316; color: white; padding: 12px 24px; border-radius: 8px; text-decoration: none; font-weight: 600; transition: background-color 0.2s;">View Full Resource</a> </div>