Numerous algorithms have met complexity in recognizing the face, which is invariant to plastic surgery, owing to the texture variations in the skin. Though plastic surgery serves to be a challenging issue in the domain of face recognition, the concerned theme has to be restudied for its hypothetical and experimental perspectives. In this paper, Adaptive Gradient Location and Orientation Histogram (AGLOH)-based feature extraction is proposed to accomplish effective plastic surgery face recognition. The proposed features are extracted from the granular space of the faces. Additionally, the variants of the local binary pattern are also extracted to accompany the AGLOH features. Subsequently, the feature dimensionality is reduced using principal component analysis (PCA) to train the artificial neural network. The paper trains the neural network using particle swarm optimization, despite utilizing the traditional learning algorithms. The experimentation involved 452 plastic surgery faces from blepharoplasty, brow lift, liposhaving, malar augmentation, mentoplasty, otoplasty, rhinoplasty, rhytidectomy and skin peeling. Finally, the proposed AGLOH proves its performance dominance.