Rheumatoid arthritis (RA) is a chronic systemic disorder characterized by persistent synovitis and systemic inflammation. Currently, the widely used drugs for the treatment of RA are disease-modifying antirheumatic drugs, biological agents and glucocorticoids. But their clinical use has been limited because of their adverse effects with a high frequency and high cost of treatment. It is essential to find novel candidate agents. Traditional Chinese medicine (TCM) has been used for RA treatment for a long period of time. In recent years, significant amounts of studies have shown that some TCMs and their active ingredients have obvious therapeutic effects on RA. In this review, the compounds in TCMs that have an effect in clinic or animal experiments of RA are critically reviewed and summarized. Moreover, the relationship between chemical structures of the compound and their activities is analyzed. The relevant researches are described from the aspects of source, methods, result, and related mechanism analysis. The existing studies show that most effective compounds in TCM for RA treatment belong to alkaloids, flavonoids, terpenoids, phenols and quinines. It is hoped that the data summarized in this review will be beneficial to the screening of new nature-derived antirheumatic drugs.