The Gunners had taken a 45th minute lead from Martin Odegaard, but the Toffees hit back in the second half with Richarlison equalising in the 79th minute after having two goals disallowed for offside earlier.
And Gray came up with a winner two minutes into stoppage time, leaving Arteta disappointed and Arsenal down in seventh after three losses in four.
“I’m very disappointed,” Arteta said. “I think in the first half we were inconsistent with the ball and we had some moments of control, we didn’t concede anything apart from the goal that was disallowed by the set-piece, but we didn’t have enough penetration or threat on the opponents’ goal.
“One of the few times we did it we scored a goal from Martin and then in the second half we had to grab the game and we tried to do that, but again, in moments we were sloppy.”
At 1-1, Eddie Nketiah should have put Arsenal ahead but inexplicably hit the post from a close-range header.
“We created four big, big chances, with Eddie, two with Martin and then one with Auba at the end, and when you don’t put them in the net away from home its very complicated to win the game, especially if you concede in the way that we did for example with the first goal,” Arteta said.
The result is relief for Everton boss Rafael Benitez amid mounting pressure, following football director Marcel Brands’ decision to step aside.
The win lifts Everton up to 12th spot, ending a run of three straight defeats, having been 16th prior to Monday’s kick-off.
“You could see today the spirit. Even the decisions - good or bad - against us, the reaction of the team was amazing and the fans were right behind them,” Benitez said.
“Everything was perfect in the way it happened - the team showed character in adversity and everybody was really delighted at the end. The fans and the players… when we stick together we are stronger.”