Neural Network Implemented in SQL

A team developing the open‑source array database library Xarray‑SQL has announced a breakthrough that could reshape how geospatial and climate data are processed. While overseeing a Google Summer of Code intern’s work during a babymoon in Corfu, the lead developer added a `to_dataset()` method that enables a seamless round‑trip between multidimensional array data and a relational tabular representation. The feature allows every N‑dimensional array to be mapped to a 2‑D table, treating orthogonal dimensions as primary keys, which in turn makes common geospatial operations—such as regridding and matrix multiplication—expressible as standard SQL joins, group‑by aggregations, and sparse matrix‑vector products.

Building on this insight, the team leveraged Coiled’s geospatial benchmark suite to demonstrate that many high‑volume (≈100 TB) climate queries are fundamentally relational operations. They further extended the framework by implementing automatic differentiation on DataFusion’s visitor pattern, inspired by JAX, and showed that gradient‑based physics calculations could be executed entirely within the database. By reducing Jacobian computations to row‑wise operations, the work suggests that neural networks could be trained using pure SQL, taking advantage of relational databases’ logical‑physical separation to potentially scale training across thousands of GPUs.

The research was presented at DataFusion’s inaugural showcase and is now being explored in collaboration with scientists and engineers at XQL Systems. Interested developers can join the discussion on the project’s Discord channel (https://xql.systems) and view the full benchmark documentation on GitHub (https://github.com/xqlsystems/xarray-sql/blob/main/docs/geos...). The team’s findings point toward a future where relational databases serve not only as data stores but also as distributed compute engines for complex scientific workflows.