{"id":"arxiv:2402.17966","kind":"preprint","title":"STC-ViT: Spatio Temporal Continuous Vision Transformer for Medium-range Global Weather Forecasting","authors":["Hira Saleem","Flora Salim","Cormac Purcell"],"author_count":3,"year":2024,"published":"2024-02-28","venue":null,"cited_by":0,"doi":null,"arxiv_id":"2402.17966","nct_id":null,"source_url":"https://arxiv.org/abs/2402.17966","archive_url":"https://space-kitty.com/papers/arxiv:2402.17966","updated":"2025-11-28","pmid":null,"pmcid":null,"openalex_id":"W4399061540","pdf_url":"https://arxiv.org/pdf/2402.17966","abstract":"Operational Numerical Weather Prediction (NWP) system relies on computationally expensive physics-based models. Recently, transformer models have shown remarkable potential in weather forecasting achieving state-of-the-art results. However, traditional transformers discretize spatio-temporal dimensions, limiting their ability to model continuous dynamical weather processes. Moreover, their reliance on increased depth to capture complex dependencies results in higher computational cost and parameter redundancy. We address these issues with STC-ViT, a Spatio-Temporal Continuous Vision Transformer for weather forecasting. STC-ViT integrates a Fourier Neural Operator (FNO) for global spatial operators with a transformer parameterised Neural ODE for continuous-time dynamics, yielding a space-time continuous model for weather forecasting. Our proposed method achieves competitive forecasting performance even with a shallow, single-layer transformer encoder mitigating the reliance on deeper networks. STC-ViT generates complete forecast trajectories with an inference speed of only 0.125 seconds and achieves strong medium-range forecasting skill on 1.5-degree WeatherBench 2 as compared to state-of-the-art data-driven and NWP models trained on higher-resolution data, with substantially lower data and compute costs. We also provide detailed empirical analysis on model's performance with respect to denser time grids, higher-accuracy ODE solvers, and deeper transformer stacks.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.LG"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[{"slug":"ai-weather-climate","name":"AI Weather & Climate Modeling","domain_slug":"energy"}],"cited_in_dossiers":[]}