Seminar Overview
This graduate seminar investigated the mathematical formulation of Deep Neural Networks (DNNs) as Mixed-Integer Linear Programs (MILPs). By modeling piecewise linear activation functions (specifically ReLU units) using exact binary integer constraints, we evaluated methods for rigorous feature visualization, adversarial perturbation analysis, and certified network verification.
Key Research Concepts
- Exact ReLU Linearization: Formulating non-convex piecewise activation functions as linear inequalities coupled with binary indicator variables ($y = \max(0, x)$).
- Branch-and-Bound Solvers: Leveraging modern integer programming solvers (Gurobi) to explore discrete network activation states with global optimality guarantees.
- Certified Adversarial Analysis: Determining exact minimal $\ell_\infty$ perturbation bounds required to flip model classification boundaries.
- Feature Inversion: Generating mathematically constrained input reconstructions matching target hidden-layer activations.
Methodology & References
Based on foundational literature in exact neural network verification, including:
- Fischetti, M., & Jo, J. (2018). Deep neural networks and mixed integer linear optimization. Constraints, 23(3), 296-309.
- Tjeng, V., Xiao, K. Y., & Tedrake, R. (2019). Evaluating robustness of neural networks with mixed integer programming. ICLR.