After a largely forgettable 70 minutes in the season curtain-raiser, Cole Palmer looked to have won it for City when the substitute curled home a stunning strike.
But Leandro Trossard equalised in the 101st minute when his shot deflected into the back of the net.
That took the game to spot-kicks, where substitute Kevin de Bruyne struck the crossbar before Rodri saw his effort saved and then Fabio Vieira converted to secure the silverware for Arsenal.
strengthen in the summer, although Mateo Kovacic - one of their two big summer signings along with Josko Gvardiol - started in this game.
He slotted in well to City's midfield and for large parts of this game they looked to have picked up from where they left off last season, controlling play and dominating possession.
Golden Boot winner Erling Haaland, who scored 52 goals in all competitions last season, had an off day in front of goal but did so when he made his debut in last year's Community Shield loss to Liverpool and both he and City went on to enjoy an exceptional campaign.