Matrix metalloproteinases (MMPs) are zinc-dependent endopeptidases, capable to degrade the extracellular matrix (ECM) in physiologic conditions. Because of their overexpression and pivotal role in many pathological events, they have been proposed as a therapeutic and prognostic target for a number of diseases. Selectivity among MMPs is essential for realizing the clinical potential of inhibitors. The design of MMP inhibitors (MMPIs) has largely focused on development of various compounds containing a zinc binding group (ZBG) in their structure, with hydroxamate being the most potent one. Due to the high degree of homology in the catalytic domain of all the MMPs, the specificity and selectivity of first generation hydroxamate MMPIs were minimal, with several off-target effects and binding to other metzincins. This review highlights the role of phosphonate as ZBG in the design and development of new MMPIs.