Big data analytics frameworks like Apache Spark and Flink enable users to implement queries over large, distributed databases using functional APIs. In recent years, these APIs have grown in popularity because their functional interfaces abstract away much of the minutiae of distributed programming required by traditional query languages like SQL. However, the convenience of these APIs comes at a cost because functional queries are often less efficient than their SQL counterparts. Motivated by this observation, we present a new technique for automatically transpiling functional queries to SQL. While our approach is based on the standard paradigm of counterexample-guided inductive synthesis, it uses a novel column-wise decomposition technique to split the synthesis task into smaller subquery synthesis problems. We have implemented this approach as a new tool called RDD2SQL for translating Spark RDD queries to SQL and empirically evaluate the effectiveness of RDD2SQL on a set of real-world RDD queries. Our results show that (1) most RDD queries can be translated to SQL, (2) our tool is very effective at automating this translation, and (3) performing this translation offers significant performance benefits.
Fri 27 OctDisplayed time zone: Lisbon change
16:00 - 17:30 | |||
16:00 18mTalk | Aliasing Limits on Translating C to Safe Rust OOPSLA Mehmet Emre University of San Francisco, Peter Boyland University of California at Santa Barbara, Aesha Parekh University of California at Santa Barbara, Ryan Schroeder University of California at Santa Barbara, Kyle Dewey California State University, Ben Hardekopf University of California at Santa Barbara DOI Pre-print | ||
16:18 18mTalk | Adventure of a Lifetime: Extract Method Refactoring for Rust OOPSLA Sewen Thy Ahrefs Research, Yale-NUS College, Andreea Costea National University of Singapore, Kiran Gopinathan National University of Singapore, Ilya Sergey National University of Singapore DOI Pre-print | ||
16:36 18mTalk | Inductive Program Synthesis Guided by Observational Program Similarity OOPSLA Jack Feser Hamilton College, Işıl Dillig University of Texas at Austin, Armando Solar-Lezama Massachusetts Institute of Technology DOI | ||
16:54 18mTalk | Automated Translation of Functional Big Data Queries to SQL OOPSLA Guoqiang Zhang North Carolina State University, Benjamin Mariano University of Texas at Austin, Xipeng Shen North Carolina State University, Işıl Dillig University of Texas at Austin DOI | ||
17:12 18mTalk | User-Customizable Transpilation of Scripting Languages OOPSLA Bo Wang National University of Singapore, Aashish Kolluri National University of Singapore, Ivica Nikolić National University of Singapore, Teodora Baluta National University of Singapore, Prateek Saxena National University of Singapore DOI | ||