Nach positiver Absolvierung der Lehrveranstaltung sind Studierende in der Lage

• relevante Aspekte von Formalismen des maschinellen Lernens zu benennen,
• die aktuelle Forschung zu einem gewählten Thema aus dem Bereich des maschinellen Lernens  zu erläutern, sowie
• relevante Literatur zu einem Thema aus dem Bereich des maschinellen Lernens  zu finden.

At the end of the course, the students will be able to read and critique scientific papers. They will be able to analyse the methods proposed in the field of neuro-symbolic AI, and evaluate their strengths and weaknesses. They will be able to prepare an oral presentation of their findings and discuss the studied methods in detail. Finally, they will be able to assess their peers’ work.

The seminar will cover topics belonging to a relevant subfield of machine learning and artificial intelligence (AI): neuro-symbolic AI.

Neuro-symbolic AI is a growing field in machine learning and AI, whose aim is to augment and combine the strengths of statistical AI with the capabilities of human-like symbolic knowledge and reasoning. Typical problems in neuro-symbolic AI include, e.g., how to integrate background knowledge into machine learning models, how to teach neural models to reason, and how to make machine learning compliant with a set of requirements.

Potential topics for this seminar include:

• Injecting background knowledge into deep learning models
• Deep learning with logical requirements
• Neuro-symbolic AI for large language models
• Reasoning shortcuts

Die Studierenden müssen

• eine Literaturrecherche zu einem gewählten Thema durchführen,
• ein Vortragskonzept zum gewählten Thema erstellen, und
• einen 25 minütigen Vortrag halten.

Students attend the kick-off meeting. In the kick-off meeting, each student is allocated to a topic and a paper. After the kick-off meeting, each student is expected to read the assigned paper and the papers belonging to their topic, and to perform a literature review of the related work. Four weeks after the kick-off meeting, each student is expected to schedule a meeting with their advisor to discuss their findings. In June, there will be a presentation day. During the presentation day, each student is expected to prepare a 25 min presentation where their findings are summarised (which is followed by a 5-10 min discussion). After the presentation, students are given feedback on their presentation and seminar paper. Each student will write a seminar paper (6-7 pages + references) on their topic, where the central ideas and methods presented in the talk are summarised. 

Reading List by Topic

Background knowledge integration

• Fischer et al. DL2: Training and Querying Neural Networks with Logic DL2: Training and Querying Neural Networks with LogicProceedings of Machine Learning Researchhttp://proceedings.mlr.press › fischer19a
• Xu et al. A Semantic Loss Function for Deep Learning with Symbolic Knowledge https://arxiv.org/abs/1711.11157
• Diligenti et al. Semantic-based regularization for learning and inference https://www.sciencedirect.com/science/article/pii/S0004370215001344
• Nandwani et al. A Primal-Dual Formulation for Deep Learning with Constraints https://proceedings.neurips.cc/paper_files/paper/2019/file/cf708fc1decf0337aded484f8f4519ae-Paper.pdf

Deep learning with requirements

• Ahmed et al. Semantic Probabilistic Layers for Neuro-Symbolic Learning https://arxiv.org/pdf/2206.00426.pdf
• Giunchiglia et al. Multi-Label Classification Neural Networks with Hard Logical Constraints https://dl.acm.org/doi/pdf/10.1613/jair.1.12850
• Hoernle et al. MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networks https://arxiv.org/abs/2111.01564

Learning and reasoning

• Wang et al. SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver https://arxiv.org/pdf/1905.12149.pdf
• Manhaeve et al. DeepProbLog: Neural Probabilistic Logic Programming https://arxiv.org/abs/1805.10872
• Yang et al. NeurASP: Embracing Neural Networks into Answer Set Programming https://www.ijcai.org/proceedings/2020/243

Reasoning shortcuts

• Li et al. Learning with Logical Constraints but without Shortcut Satisfaction https://openreview.net/forum?id=M2unceRvqhh
• Marconato et al. Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts https://arxiv.org/pdf/2305.19951.pdf

Constraining LLMs

• Lu et al. NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints https://arxiv.org/abs/2010.12884
• Meng et al. Controllable Text Generation with Neurally-Decomposed Oracle https://proceedings.neurips.cc/paper_files/paper/2022/file/b40d5797756800c97f3d525c2e4c8357-Paper-Conference.pdf
• Zhang et al. Tractable Control for Autoregressive Language Generation https://arxiv.org/pdf/2304.07438.pdf

Prüfungsimmanent