Generalized Convexity and Smoothness via Conjugate Duality: Optimization Theory for Deep Neural Networks

Deep neural network (DNN) training with stochastic gradient descent (SGD) and its variants achieves strong empirical performance, yet classical optimization theory does not fully explain this success. This limitation arises because conventional analyses rely on assumptions such as differentiability,...