With the urge to steadily improve performance of deep neural network models, the necessity emerges to scale out model training, in order…