We study the Kibble-Zurek scaling dynamics and universal second order phase transitions in a relativistic scalar field theory in 1+1 dimensions. A current challenge in modern cosmology is whether the vacuum contains topological defects generated during the initial phase transitions in the Early Universe, such as domain walls, cosmic strings and textures. Using tensor networks techniques as a non-perturbative non-equilibrium numerical tool for quantum field theory, we perform an analysis of the formation of topological defects in a non-equilibrium quantum system, as a realistic toy model of the formation of cosmological defects in the large-scale structure of the Early Universe. We employ Matrix product states and the time dependent variational principle to predict the Kibble-Zurek scenario for a system undergoing a symmetry breaking phase transition in the context of a scalar field theory and study the non-equilibrium time evolution of a quantum field theory in 1+1 space-time dimensions.