This paper focuses on developing effective computational methods to enable the real-time application of model predictive control (MPC) for nonlinear systems. To achieve this goal, we follow the idea of approximating the MPC control law with a Deep Neural Network (DNN). To train the deep neural network offline, we propose a new “optimize and train” method that combines the steps of data generation and neural network training into a single high-dimensional stochastic optimization problem. This approach directly optimizes the closed loop performance of the DNN controller over a finite horizon for a number of initial states. The large-scale optimization problem can be solved efficiently using parallel computing techniques. The benefits of this approach over the conventional “optimize then train” protocol is illustrated through numerical results.