Surrogate modeling alleviates the computational burden of numerical simulations and enables the rapid exploration of complex systems. Deep learning based models, such as graph neural networks (GNN), have shown success in modeling a variety of dynamic processes. Physical constraints in tandem with deep learning techniques can narrow the solution space of a learned surrogate and increase the likelihood of characteristic outcomes. However, this requires specific domain knowledge and may undercut the capability of sophisticated deep learning architectures to learn physical constraints without direct implementation. We investigate the performance of a GNN with different levels of physics guidance using a simple molecular dynamics system as a case study. Specifically, we explore the model’s capacity to learn skew symmetry of the force tensor, time reversibility, three body interactions, and cut-off radii by modifying the GNN architecture accordingly.