Brendan Rodgers' men made it back-to-back wins in the league for the first time this season and leapfrogged above the Hammers - but they had to come from behind late in the game to do so.
Glen Johnson put them into the lead after just 10 minutes when he produced a goal-of-the-season contender from the edge of Jussi Jaaskelainen's penalty area.
Mark Noble slotted from the spot after Joe Allen was adjudged to have handled in the penalty area before Steven Gerrard turned a Matt Jarvis cross past his own 'keeper to give West Ham the lead.
However, Joe Cole struck against his old club to equalise and then Collins turned a Jordan Henderson cross into his own net to send the travelling Reds into euphoria.
All the build-up to the clash revolved around the absence of Luis Suarez, who was ruled out after picking up his fifth yellow card against Southampton, but his replacement Shelvey proved to be the hero of the hour.
Upton Park was on its feet on six minutes when Jarvis's cross was scrambled clear by Daniel Agger and Mohamed Diame fired the loose ball against Carlton Cole, who deflected it a yard wide of Pepe Reina's post.
The Hammers posed a real threat down the left flank with Jarvis, who was constantly keen to spin a deadly cross in behind Martin Skrtel and Agger and this would prove to be a real thorn in the Reds' side throughout the match.
Full report: Liverpool FC Official
