Dipeptidyl peptidase 4 (DPP-4), a substrate-specific serine protease, has been validated as a promising drug target for the treatment of type 2 diabetes. DPP-4 inhibitors significantly lowered blood glucose levels in patients with type 2 diabetes without common body weight gain, hypoglycemia and gastrointestinal disturbance side effects. Therefore, DPP-4 inhibitors attracted more and more attention. In particular, non-peptidomimetic DPP-4 inhibitors have been a focus of research and development and made great progress in recent years, which resulted in the discovery of a wide variety of potent non-peptidomimetic DPP-4 inhibitors. Some of them, such as sitagliptin, alogliptin and linagliptin have already been used as marketed drugs, while others have been into clinical trials. Based on the core structural features of non-peptidomimetic DPP-4 inhibitors, seven types were classified in the article. For each type, we focused on the description of strategies for design and optimization, together with a discussion on concluded structure-activity relationships (SAR). In addition, the contribution of specific substituents to the inhibition of DPP-4 was summarized. Selectivity towards the inhibition of DPP-4 over dipeptidyl peptidase 8 (DPP-8) and dipeptidyl peptidase 9 (DPP-9) was also presented.