The prevalence of Alzheimer’s disease (AD) in the elderly is growing rapidly worldwide, and the deteriorating clinical course of AD places a heavy burden on both the patients and their families. Early detection and intervention of mild cognitive impairment in the early phase of AD is vital for the purpose of improving the cognitive performance of patients and preventing the existing deficits from worsening. However, the main compounds currently used to treat early AD, acetylcholinesterase inhibitors (AChEIs), are unsatisfactory in efficacy and safety. Moreover, evidence indicates that AChEIs are ineffective in treating AD at extremely early stages, which implies that mechanisms other than those targeted by AChEIs underlie the pathogenesis of AD. Dysfunctional glutamatergic neurotransmission, particularly that mediated by the N-methyl-D-aspartate (NMDA) receptor, has been reported to play a role in the pathophysiology of AD. The NMDA receptor (NMDAR) regulates synaptic plasticity, memory, and cognition, and the attenuation of NMDAR-mediated neurotransmission can result in impaired neuroplasticity and cognitive deficits in the aging brain. Furthermore, NMDARs also interact with amyloid beta peptide/amyloid precursor protein and tau protein, whose production represents the main manifestations of AD. In this paper, we review the evidence supporting NMDA dysfunction in both animal models of AD and patients afflicted with AD, and we also review the literature that suggests that NMDA-enhancing agents such as glycine and D-cycloserine can improve cognitive functions. The benefits and limitations of NMDAR antagonists that can diminish the excitatory neurotoxicity triggered by glutamate are also addressed in relation to AD. We propose that enhancing glutamatergic neurotransmission by activating the NMDAR may be effective in treating the cognitive decline that occurs in AD. Prospective studies on AD in which NMDA-enhancing agents are used are required to verify this hypothesis and confirm the long-term efficacy and safety of the treatment agents.