192.024 Seminar in Artificial Intelligence: Neurosymbolic Artificial Intelligence
This course is in all assigned curricula part of the STEOP.
This course is in at least 1 assigned curriculum part of the STEOP.

2024S, SE, 2.0h, 3.0EC

Properties

  • Semester hours: 2.0
  • Credits: 3.0
  • Type: SE Seminar
  • Format: Blended Learning

Learning outcomes

After successful completion of the course, students are able to

  • to name relevant aspects about machine learning,
  • to describe the current research for a chosen topic from the area of machine learning, and
  • to find relevant literature for a topic from research about machine learning.

At the end of the course, the students will be able to read and critique scientific papers. They will be able to analyse the methods proposed in the field of neuro-symbolic AI, and evaluate their strengths and weaknesses. They will be able to prepare an oral presentation of their findings and discuss the studied methods in detail. Finally, they will be able to assess their peers’ work.

Subject of course

The seminar will cover topics belonging to a relevant subfield of machine learning and artificial intelligence (AI): neuro-symbolic AI.

Neuro-symbolic AI is a growing field in machine learning and AI, whose aim is to augment and combine the strengths of statistical AI with the capabilities of human-like symbolic knowledge and reasoning. Typical problems in neuro-symbolic AI include, e.g., how to integrate background knowledge into machine learning models, how to teach neural models to reason, and how to make machine learning compliant with a set of requirements.

Potential topics for this seminar include:

  • Injecting background knowledge into deep learning models
  • Deep learning with logical requirements
  • Neuro-symbolic AI for large language models
  • Reasoning shortcuts

Teaching methods

The students have to

  • perform a literature research for a selected topic,
  • prepare a presentation draft about the chosen topic, and
  • give a presentation of 50min.

Students attend the kick-off meeting. In the kick-off meeting, each student is allocated to a topic and a paper. After the kick-off meeting, each student is expected to read the assigned paper and the papers belonging to their topic, and to perform a literature review of the related work. Four weeks after the kick-off meeting, each student is expected to schedule a meeting with their advisor to discuss their findings. In January, there will be a presentation day. During the presentation day, each student is expected to prepare a 15 min presentation where their findings are summarised. After all students covering a particular topic have presented, there will be a 30 min panel discussion. During the panel discussion, each student will advocate for the ideas presented in their paper, and compare and contrast them to the ideas presented by others. After the presentation, the advisor meets their student to give them feedback on their presentation and discuss the structure of the seminar paper. Each student will write a seminar paper (6-7 pages + references) on their topic, where the central ideas and methods presented in the talk are summarised. Each student will also write two reviews (~0.5 pages) of the manuscripts submitted by other fellow students.

Mode of examination

Immanent

Additional information

Reading List by Topic

Background knowledge integration

  • Fischer et al. DL2: Training and Querying Neural Networks with Logic DL2: Training and Querying Neural Networks with LogicProceedings of Machine Learning Researchhttp://proceedings.mlr.press › fischer19a
  • Xu et al. A Semantic Loss Function for Deep Learning with Symbolic Knowledge https://arxiv.org/abs/1711.11157
  • Diligenti et al. Semantic-based regularization for learning and inference https://www.sciencedirect.com/science/article/pii/S0004370215001344
  • Nandwani et al. A Primal-Dual Formulation for Deep Learning with Constraints https://proceedings.neurips.cc/paper_files/paper/2019/file/cf708fc1decf0337aded484f8f4519ae-Paper.pdf

Deep learning with requirements

  • Ahmed et al. Semantic Probabilistic Layers for Neuro-Symbolic Learning https://arxiv.org/pdf/2206.00426.pdf
  • Giunchiglia et al. Multi-Label Classification Neural Networks with Hard Logical Constraints https://dl.acm.org/doi/pdf/10.1613/jair.1.12850
  • Hoernle et al. MultiplexNet: Towards Fully Satisfied Logical Constraints in Neural Networks https://arxiv.org/abs/2111.01564

Learning and reasoning

  • Wang et al. SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver https://arxiv.org/pdf/1905.12149.pdf
  • Manhaeve et al. DeepProbLog: Neural Probabilistic Logic Programming https://arxiv.org/abs/1805.10872
  • Yang et al. NeurASP: Embracing Neural Networks into Answer Set Programming https://www.ijcai.org/proceedings/2020/243

Reasoning shortcuts

  • Li et al. Learning with Logical Constraints but without Shortcut Satisfaction https://openreview.net/forum?id=M2unceRvqhh
  • Marconato et al. Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts https://arxiv.org/pdf/2305.19951.pdf

Constraining LLMs

  • Lu et al. NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints https://arxiv.org/abs/2010.12884
  • Meng et al. Controllable Text Generation with Neurally-Decomposed Oracle https://proceedings.neurips.cc/paper_files/paper/2022/file/b40d5797756800c97f3d525c2e4c8357-Paper-Conference.pdf
  • Zhang et al. Tractable Control for Autoregressive Language Generation https://arxiv.org/pdf/2304.07438.pdf
Please consider the plagiarism guidelines of TU Wien when writing your seminar paper: Directive concerning the handling of plagiarism (PDF)

Lecturers

Institute

Course dates

DayTimeDateLocationDescription
Wed14:00 - 15:0013.03.2024 https://tuwien.zoom.us/j/8104603001?pwd=WTlBMERDRkJFMVZNU2Z6aHRDUk9Zdz09Kickoff Meeting

Examination modalities

Immanent

Course registration

Begin End Deregistration end
15.02.2024 10:00 11.03.2024 22:00 08.03.2024 22:00

Curricula

Study CodeObligationSemesterPrecon.Info
066 931 Logic and Computation Mandatory elective
066 936 Medical Informatics Mandatory elective

Literature

No lecture notes are available.

Language

English