Master Thesis in Multimodal Foundation Models with Knowledge Graphs for Visual Entity Recognition

During your Master thesis, you will learn state-of-the-art methods for visual entity recognition, combining information from KGs, images and text. You will develop more advanced algorithms based on CLIP-based approaches or autogressive approaches: you will train advanced knowledge graph embedding methods such as TransE, TransH and some other GNN-based methods. Furthermore, you will train the VLMs such as CLIP, Pali. In addition, you will perform ablation studies and run existing baselines. Last but not least, you will develop your own methods for combining KG and foundation models.