The All Whites, who hadn’t qualified for the World Cup finals since 2010, went down to the Ticos after Joel Campbell’s third minute goal.
New Zealand were denied an equaliser from Newcastle United forward Chris Wood by VAR, while Kosta Barbarouses was sent off in the second half upon review.
Wood’s goal was disallowed after the VAR intervened, claiming that New Zealand’s Matt Garbett had pulled Costa Rica’s Oscar Duarte to the ground in the build-up to his cross for the strike.
“If I’m being honest, I thought some of the officiating was absolutely atrocious,” Hay told reporters.
“The disallowed goal was two players battling for the ball. The foul could have gone to Matt Garbett to start with. Obviously VAR got involved and overturned that.”
On the Barbarouses red card, Hay added: “I haven’t looked closely at the other one. Our analyst, who was watching in the stands, wasn’t convinced, but we could be proven wrong.”
Hay added he felt let down by FIFA, with UAE referee Mohammed Abdulla given the responsibility to handle the one-off game, rather than a more experienced official like Slovenian Slavko Vincic who led Monday’s Australia-Peru playoff.
“They have let us down in putting somebody in charge who has clearly not officiated at this sort of level,” Hay said.
“When we looked at what Australia got against Peru last night with good quality European officiating, I thought FIFA made a mistake in such an important game for us.”