Human endophilin 1 (hEndo1) is a multifunctional protein that was found to bind a wide spectrum of prolinerich endocytic proteins through its Src homology 3 (SH3) domain. In order to elucidate the unknown biological functions of hEndo1, it is essential to find out the cytoplasmic components that hEndo1 recognizes and binds. However, it is too time-consuming and expensive to synthesize all peptide candidates found in the human proteome and to perform hEndo1 SH3–peptide affinity assay to identify the hEndo1-binding partners. In the present work, we describe a structure/ sequence-hybrid approach to perform proteome-wide inference of human hEndo1-binding peptides using the information gained from both the primary sequence of affinity-known peptides and the interaction profile involved in hEndo1 SH3–peptide complex three-dimensional structures. Modeling results show that (i) different residue positions contribute distinctly to peptide affinity and specificity; P-1, P2 and P4 are most important, P1 and P3 are also effective, and P-3, P-2, P0, P5 and P6 are relatively insignificant, (ii) the consensus core PXXP motif is necessary but not sufficient for determining high affinity of peptides, and some other positions must be also essential in the hEndo1 SH3–peptide binding, and (iii) the alternating arrangement of polar and nonpolar amino acids along peptide sequence is critical for the high specificity of peptide recognition by hEndo1 SH3 domain. In addition, we also find that the residue type at a specific position of hEndo1-binding peptides is not stringently invariable; amino acids that possess similar polarity could replace each other without substantial influence on peptide affinity. In this way, hEndo1 presents a broad specificity in the peptide ligands that it binds.