Abnormal regulation of the ubiquitin-proteasome system (UPS) has been known to be involved in the pathogenesis of a variety of human diseases. A number of studies have focused on the identification of small modifiers for the UPS. Even though the proteasome inhibitor Bortezomib (Velcade®) has been approved for the therapy of multiple myeloma and mantle cell lymphoma, there are still no DUB inhibitors endorsed for clinical usage. Since deubiquitinating enzymes (DUBs) are becoming as a new class of modifiers in the UPS, potential drugs that target specific DUBs have been investigated with the development of experimental technologies for screening small inhibitor molecules. However, the molecular mechanisms of these molecules are poorly understood. In order to design and develop specific small inhibitor molecules for specific DUBs, identification of specific substrates and molecular structures for each DUB is required. Here, we review structures, substrates, and small inhibitor molecules of DUBs identified up to date, providing a clear rationale for the development of novel small inhibitor molecules of DUBs for cancer.