Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL

A demo shows a FlashAttention kernel deduced automatically from roughly 25 lines of SQL that expresses only vanilla attention, built on three portable SQL extensions the team is developing: xarray-sql, which represents petabyte-scale array datasets in the cloud as relations; ddx, a database extension adding JAX-style automatic differentiation to standard SQL; and einfold, a modular query planner for fast tensor contractions in vanilla SQL. The project's thesis is that separating the logical layers of a tensor program from the physical layers is both more correct and more scalable, and it asks whether the Bitter Lesson can be formalized with relational algebra. The team is recruiting collaborators at the intersection of database theory and ML systems via Discord at xql.systems.

In this demo, we automatically deduce a FlashAttention kernel with ~25 lines of SQL, which just expresses vanilla attention. This is possible due to three portable SQL extensions that we’re building: - xarray-sql: a library that efficiently represents petabyte scale array datasets in the cloud as tables relations . - ddx: a database extension that adds automatic differentiation in the style of JAX to standard SQL, assuming the above array data model. - einfold: a modular query planner for fast tensor contractions in vanilla SQL, again, assuming the above array data model. My hope is that the relational model and algebra provide fresh perspectives on how we might do accelerator programming. Here, our thesis is that separating the logical layers in a tensor program from the physical layers is both more correct and more scalable. If the Bitter Lesson is a central dogma of machine learning, then why not formalize it with relational algebra? If this interests you and you’d like to tinker with us at the intersection of database theory and ML systems, join us on Discord at https://xql.systems https://xql.systems link at the bottom . Cheers. Comments URL: https://news.ycombinator.com/item?id=50309479 https://news.ycombinator.com/item?id=50309479 Points: 5 Comments: 0