Master Thesis in Enhancing and Evaluating Natural Language to SQL Capabilities for LLM Applications

The increasing use of Natural Language to SQL (NL2SQL) techniques is transforming the way large language models (LLMs) help bridge the gap between complex industrial data and users, enabling domain experts to interact with data using natural language. However, challenges remain in optimizing and evaluating NL2SQL outputs, particularly for interactive AI applications and specialized domains like semiconductor data visualization. This project aims to investigate and improve NL2SQL methods to support our "Data Viz" tool. During your thesis you will conduct a thorough analysis of the existing domain knowledge, advanced NL2SQL methods, and existing evaluation techniques. This includes reviewing methods such as fine-tuning, retrieval-augmented generation (RAG), and researching how AI agents are applied in NL2SQL tasks and exploring potential approaches. You will explore existing advanced techniques and identify key areas for improvement. You will automate the process of collecting, preparing, and managing training and testing datasets by developing scripts and tools to automate the extraction and preprocessing of training and testing data. You will ensure data quality to represent domain knowledge across various scenarios. Furthermore, you will implement methods to enhance NL2SQL performance and retrieval accuracy. You will design and implement multiple suitable methods to enhance NL2SQL performance and benchmark different methods. Moreover, you will establish robust evaluation methods to measure the improvements in NL2SQL results. You will define metrics for evaluating NL2SQL performance (e.g. accuracy, efficiency) and implement evaluation protocols to systematically assess improvements. Finally, you will rigorously test and validate the effectiveness of the enhanced NL2SQL system comparing results against a baseline to measure improvement and compile comprehensive documentation of the entire process, including methodologies, codes, processes, and results. You will prepare a final report summarizing findings and potential future work.