Ramsey scored his eighth goal in nine matches to add to an earlier strike from teenager Serge Gnabry, and the Gunners held on after Ben Davies pulled a goal back for the hosts.
The result sees Arsenal top the table by two points from rivals Tottenham, with Chelsea a further two points back.
That is prompting hopes that the Gunners' long wait for a trophy could be over as they put themselves firmly in the title race.
"That's our target," Ramsey told Sky Sports 1. "We have a lot of quality. We've had a lot of injuries recently but we've managed to dig in and put a load of wins together and hopefully now we get players back and we're in with a shout."
After a less than inspiring first half Arsenal found their stride after the break.
Gnabry capped a well-worked move in the 58th minute and although Mesut Ozil squandered a chance to double the lead almost immediately, it was not long before Ramsey found the net.
"Swansea is a difficult place to come but we've passed the ball around really well," Ramsey said. "We had to dig in and stay solid and try not to concede an early goal. We've done that, we've started to defend as a unit and the goals have come in the second half.
"We knew it was going to be a tough last 20 minutes but we managed to dig in and get the win.
"In the first half we tried to press them a bit too high up but in the second half we let them have the ball, we waited for them to come into our half and defended quite deep so we could get them on the counter-attack.
"We broke with pace and quality and managed to gain a few times."
Ramsey fired home the second goal in the 62nd minute, and he is delighted with his own goalscoring run.
"I felt like throughout my career I've been getting into these positions but I was perhaps a bit rash in front of goal," he said. "I'm a lot more composed now, I'm feeling confident and things are starting to click into place quite nicely."
It was a welcome anniversary present for manager Arsene Wenger, who was marking 17 years in charge of the Gunners.
"There is nothing better than winning," the Frenchman said with a smile, his day made all the better with Manchester City, Manchester United,
Tottenham and Chelsea all dropping points before Arsenal even kicked off.
"The first half was quite difficult, we were slow in our combinations and Swansea were sharper than us. It was a question of being patient without making mistakes. After we were much stronger and playing at our level.
"We took advantage of the chances we have created. We have substance in the team and we could see that today in the second half. We came out with purpose in our game and that was important."
Arsenal have now won five games on the bounce since losing on the opening day, and, asked if his team could go on and challenge for a league title that the club last won in 2004, Wenger said: "It is early in the season but we had an oppounity today. The sign of quality in a team is that when you have an opportunity you take it. Today we took it.
"Our spirit has been tested after the first home game of the season but the way we have responded is very positive. In this job it is very important that you keep focused and you have a purpose and desire to improve. Let's focus on that."