The result left Barcelona ninth in the league, with president Joan Laporta opting to remove Koeman after a prolonged period of pressure.
Barca had endured an ordinary start to the new season, struggling to adapt to life without Lionel Messi, losing their opening two UEFA Champions League games too.
“FC Barcelona has relieved Ronald Koeman of his duties as first team coach,” the club wrote in a statement.
“The president of the club, Joan Laporta, informed him of the decision after the defeat against Rayo Vallecano. Ronald Koeman will say goodbye to the squad on Thursday at the Ciutat Esportiva.
“FC Barcelona wishes to thank him for his service to the Club and wishes him all the best in his professional career.”
The early contenders to replace Koeman include club legend Xavi along with Marcelo Gallardo, Andrea Pirlo, Roberto Martinez and Erik ten Hag.
Koeman had taken over at Camp Nou in August 2020 on a two-year deal, leading the club to the Copa del Rey title last term, along with a third-place finish in the league.
Barcelona struggled in Europe, bowing out of the Champions League in the last 16 to Paris Saint-Germain.
Laporta’s return to the club presidency also hastened Koeman’s departure, given tension between the pair.
Messi’s exit to PSG in August was due to the club’s financial challenges, which has also impacted Koeman and their ability in the transfer market.