Many real-world systems, such as moving planets, can be considered as multi agent dynamic systems, where objects interact with each other and co-evolve along with the time. Such dynamics is usually difficult to capture, and understanding and predicting the dynamics based on observed trajectories of objects become a critical research problem in many domains. Most existing algorithms assume the observations are regularly sampled and all the objects can be fully observed at each sampling time, which is impractical for many applications. To tackle the above challenge, we present LG-ODE, a latent ordinary differential equation generative model for modeling multi-agent dynamic system with known graph structure, which is the first Graph ODE model in this direction. It can simultaneously learn the embedding of high dimensional trajectories and infer continuous latent system dynamics. Experiments on motion capture, spring system, and charged particle datasets demonstrate the effectiveness of our approach.