High entropy (HE) ceramics are an emerging material class which, like their metal counterparts, are defined by multiple equi-molar species on a crystal lattice. They are unique within the HE alloys by having multiple sublattices. HE carbides, nitrides and oxides, for example, have a rock salt structure with C, N or O on a fcc sublattice, respectively, and a fcc sublattice containing a random cation population. This talk will focus on first principles and molecular modeling studies of these materials. For oxides, the talk will include how charge compensation accommodates aliovalent elements and effects phonon thermal transport, and how random Cu placement creates a unique Jahn-Teller distortion. For HE carbides and diborides, calculations of phase stability, binding energy, bulk modulus, vacancy formation energy, and thermal conductivity will be discussed. This work was supported by MURI grant N00014-15- 1-2863.