Kenny Dalglish's side have already squared up against the Seagulls this season when goals from Craig Bellamy and Dirk Kuyt gave the Reds a hard-fought 2-1 Carling Cup third round victory at the Amex Stadium in September.
And veteran centre-back Carragher wants to shrug off the favourites tag with a place in the quarter-finals at stake knowing Gus Poyet's side is capable of creating an upset at Anfield.
"There are no easy games in the FA Cup - I could tell you that from the teams we've been knocked out by over the years," Carragher told www.liverpoolfc.tv.
"We've had some bad results against lower league opposition, but that's what makes the FA Cup what it is.
"That's why it's so special when you do win it because you realise the pitfalls along the way.
"It would be disappointing if we were to go out having knocked Manchester United out of the last round.
"It's a massive game for us and if we win we'll be into the quarter-finals, and if you do well there then you're at Wembley. It certainly keeps the season going."
Carragher, who has won two FA Cup winners' medals with Liverpool, knows any side that visits Anfield in the competition does so with a 'nothing to lose' mentality - something he is especially wary of against Brighton.
The 34-year-old has been part of Reds sides that have been knocked out of the Cup competition at Anfield, most notably against Crystal Palace, Barnsley and Reading, and is expecting a tough challenge from Poyet's side.
"We've beaten them before and we would expect to win with the game being at Anfield, but it certainly won't be easy," Carragher said.
"The football they played [in the Carling Cup game] was outstanding for a team at that level. You don't normally expect that, but that's great credit to their players and also to the manager Gus Poyet for what a great job he's done.
"It will be a tough game as it was against them in the Carling Cup. They're a very good footballing team and keep possession of the ball well, and that's something we'll have to try and stop at Anfield."
"Lower league teams can play like they've got nothing to lose when they come to Anfield because they haven't - they've got no fear and no worries. If they get beat they're expected to lose.
"For them it's a win/win really, but we're used to that. Most teams we play at Anfield will approach the game like that and there's always pressure on us to produce at home - and rightly so."
Liverpool have impressed in cup competitions this season, reaching the final of the Carling Cup at the expense of Barclays Premier League leaders Manchester City and progressing to the fifth round of the FA Cup after beating rivals Manchester United 2-1.
Carragher believes the Reds' success is down to the respect they have paid to the tournaments.
"I put it down to the manager for putting out strong sides from the very beginning [in the Carling Cup] and giving the competition the respect it deserves," he said.
"We're in the [Carling Cup] final and that's what we deserve for how much we've respected the competition.
"In the FA Cup we're only just starting - we've played Manchester United which was a great game for us and Oldham was difficult for us, 5-1 wasn't a true reflection.
"Hopefully we can win the Carling Cup, but when that's finished and put to bed, we'll still need to be in the FA Cup and keep something else to look forward to at the end of the season."