Saints ended a seven-year exile from the top flight after automatic promotion from the Championship, but have so far endured a torrid start to life back in the big time.
Yesterday's [Saturday] 4-1 defeat at the hands of fellow new boys West Ham means the south coast side have just four points to show from eight Premier League matches, letting in 24 goals in the process.
No side has ever conceded as many goals at this stage of a Premier League season and Adkins knows things need to change quickly.
"We've obviously conceded some poor goals again," the Southampton manager said.
"We'll remain positive in what we do. We'll work hard on the training ground.
"In the first half the players did very well and let's remember, we defend as a team.
"Let's not highlight any individuals. It is my responsibility and if anyone wants to point the finger, point it me.
"At the end of the day, we defend as a team and we attack as a team and that is the important thing."
Southampton's team selections this season have at times raised a few eyebrows and yesterday's decision to start with top scorer Rickie Lambert on the bench confused many.
"I thought Jay Rodriguez did very well in the first half," Adkins said in defence of the team selection.
"We've got Jay Rodriguez, Emmanuel Mayuka, Rickie Lambert - we've got several players at the football club.
"Me as manager, it is my responsibility to pick from the group of players I have to select from.
"Everyone will have a different opinion, everyone will pick a different team.
"Whether I start Rickie Lambert, whether I bring Rickie Lambert on, whatever team is selected, there will always be a decision made about it.
"Ultimately, it is my decision as the manager to pick the team and the buck stops with me."
While the pressure mounts on Adkins and his side, things for yesterday's [Saturday] opponents West Ham look far rosier.
The Hammers dominated proceedings in east London after a dour first half, securing victory thanks to Mark Noble's double and goals from Kevin Nolan and Modibo Maiga.
The victory was the east Londoners' fourth since their return to the Premier League and pleased man-of-the-match Noble.
"We all said at half-time that we have been very good at home but we cannot play like we have every game," he told West Ham TV.
"You have to give credit to Southampton because they closed us down and made it very difficult for us.
"We defended very well and at 0-0 we were still massively in the game and knew we had a chance to go on and win it which thankfully we did.
"To get four against Southampton was fantastic but that is what can happen when you have quality players like Yossi Benayoun, big Andy Carroll and Nobby (Nolan).
"Matt Jarvis also had a very good game and Tonks (James Tomkins) did a fantastic job playing at right-back, despite having not played two months. It was great for the lads".