Quill + Spark = Better Together!
Spark offers a curious compromise, either you get a un-typed DataFrame API which does some great optimizations with column selection and filtering but gives you almost no guarantees about compile-time type safety. Alternatively, you can use a decent Scala-esque, typesafe-esque Dataset API that has no data-level optimizations at all. Quill-Spark gives you the best of both worlds, and then some! You can query in 100% typesafe way that’s fully optimized, and you get a richer set of joining semantics! What’s more, it’s a simple API that developers can get started with, in minutes, and it’s IDE friendly (and you don’t need to know Shapeless!). Want to learn more?