The cannabinoid CB1 and CB2 receptors belong to the Class A, rhodopsin-like family of G protein-coupled receptors. Antagonists for each receptor sub-type, as well as four structural classes of agonists that bind to both receptors, have been identified. An extensive amount of SAR has been developed for agonists and antagonists that bind at CB1, while the SAR of CB2 ligands is only now emerging in the literature. Cannabinoid agonists have been suggested to have potential therapeutic uses as appetite stimulants, analgesics, anti-emetics, anti-diarrheals, antispasmodics, tumor anti-proliferative agents, anti-glaucoma agents and as agents for the treatment of diseases associated with inappropriate retention of aversive memories such as post-traumatic stress disorders and phobias. Cannabinoid CB1 antagonists have been suggested to have potential therapeutic uses as appetite suppressants and as agents that improve memory. This review focuses first on recent CB1 and CB2 SAR and on the pharmacophores that have been developed for ligand recognition at the CB1 receptor. Emerging ideas about how the cannabinoid receptors are activated by agonists or inactivated by inverse agonists are then presented. Challenges for future SAR and pharmacophore development are also identified.