Friction stir welding (FSW) has been successfully applied in fabricating many critical structures, e.g. rocket fuel tanks. Generally, CFD simulation is required to better understand the in-process material flow during FSW. In this paper, we discuss the concepts and the approaches that have been employed in the recent development and application of the CFD simulation for FSW. First, special considerations on friction, heat generation and transient tool motion have been adopted to capture the fully-coupled heat-and-mass-flow phenomenon during FSW. Second, temporal evolution of the material state during welding is analyzed by interpolation and integration along the flow paths, which is further used to predict the typical feature and defects in the welds. Third, the CFD-based predictions on the temperature and the material flow are validated by experimental measurements. Finally, the current concepts and approaches in the simulation of FSW could be applied in CFD-based studies on other similar thermal-mechanical processes.