The only Rockabilly came between Kill Uncle and Your Arsenal.
No, there were still rockabilly elements on Vauxhall & I. It doesn't mean that it was a fully blown rockabilly album, or that it was as "rockabilly" as Your Arsenal. Your Arsenal wasn't an entirely rockabilly oriented album either. You're taking this too literally.
"The Lazy Sunbathers" was really just a slowed-down Buddy Holly style song. It had the progressions, and the tone of the genre, similar to what Chris Issaks was doing around the same time, and what Richard Hawley, a rockabilly oriented artist, does on his albums. It was his rockabilly crooner album, or Elvis pulling away from his Sun years album.
The slide breakdown on "The More You Ignore Me" was another example. You could even hear the rockabilly crooner angle in Used To Be a Sweet Boy; it's just slowed down. "Hated For Loving" was another classic rockabilly influenced track. The slides, the scales, the bends, and the overall production of the track contain the spirit. Even some of the lead guitar passages on Hold On To Your Friends have a bit of rockabilly DNA in them that comes through nicely.
The point is, these elements, however subtle, affected his overall sound, and gave him his most perfectly fitting musical identity. After that, he never really found a sound that complemented his voice, or his lyrics they way the rockabilly influenced did; or the way the distinct, alternative style of The Smiths, and his early solo work did. He was at his best on the rockabilly style tracks on The Smiths albums as well.
I think that's the problem: After Vauxhall, Morrissey's sound seemed closer to generic, L.A. style pop-punk bands, or British pub rock than it did to the haunting, and carefully manicured sounds of his previous albums.
He stopped sounding like an "alternative" artist, and started sounding more like he was trying to fit in musically. I think that's what many fans of that era lament the most: Morrissey's lost his aura of other-worldliness. People kept thinking Morrissey was going to eventually become a Bowie, Eno, or Cale style artist, and instead, he sound more like Oasis.
It's why I think "World Peace" was actually an interesting musical achievement, even if I didn't entirely enjoy the lyrical content of the album. At the very least, it seemed closer to what he was willing to try at his peak, than anything he did after it.