The pair took over the reins in 2013 and guided the Republic to the last 16 of the 2016 European Championships.
However, they did not win a competitive game in 2018 and were relegated from Uefa Nations League Group B4.
“I thank Martin, Roy, and the management team for the impact they had,” said Football Association of Ireland (FAI) chief John Delaney.
“There have been many highlights during Martins reign – none more so than Euro 2016 in France, which will live long in the memory of all Irish supporters.”
ONeill, who leaves alongside his backroom team of Keane, Steve Guppy and goalkeeping coach Seamus McDonagh, signed a contract extension in January to remain as manager until 2020.
After their World Cup 2018 qualification hopes ended in a humiliating 5-1 home defeat by Denmark, the Republics inaugural Nations League campaign was plagued by poor results and a string of off-field incidents.
In September, midfielder Harry Arter pulled out of the squad for games against Wales and Poland following a row with Keane.
Nottingham Forest legend Martin ONeill leaves Republic of Ireland role
ONeill defended his assistant after audio emerged of defender Stephen Ward apparently describing the incident which led to Arters withdrawal.
Although Arter would return to the squad for Octobers fixtures, highly-rated West Ham teenager Declan Rice has remained absent since September.
The London-born defender continues to consider whether to represent the Republic, for whom he has featured in three friendlies, or England.
Relegation to the third tier of Uefas new competition came as they collected just two points, scoring only once, in four games.
Speaking after Mondays scoreless draw in Denmark, ONeill indicated that he wanted to remain in the job.
Martin ONeill: Republic of Ireland managers future in doubt
“If I dont do that there Ill have a conversation with him in the next few days or whenever it may be and its always been the case.”
The 0-0 draw in Aarhus means ONeills team has now gone four matches without scoring a goal and the Republic, ranked 33rd in the world, failed to win a competitive match in 2018.
The Football Association of Ireland president Donal Conway said on the associations official website: “On behalf of the FAI Board, I would like to thank Martin, Roy, and the management team for their work with the Republic of Ireland team over the last five years.”
Martin ONeill quits as Republic of Ireland manager
The appointment of ONeill and former captain Keane in November 2013 was greeted by a wave of optimism among supporters following the decline of the team under Giovanni Trapattoni, which had seen the Republic slump to 67th place in the Fifa world rankings.
Home News Sport Entertainment Services Search Sport All Sport Football Grassroots Rugby Cricket Boxing Speedway Darts Commonwealth Games More UK & international sports Viral sports news Aston Villa Aston Villa > Fixtures Squad Advertising
Resigned: Martin ONeill and Roy Keane leave jobs with Republic of Ireland
The new management duo had almost a year to prepare for their first competitive fixture and successfully steered the side to Euro 2016 after a qualifying campaign that featured a 1-0 defeat of Germany before beating Bosnia-Herzegovina in a play-off.
Ex-Celtic duo Roy Keane and Martin ONeill sacked by Republic of Ireland after shocking run of form
A famous win against Italy at the European Championships secured a last 16 match against tournament hosts France but they were beaten 2-1 in Lyon.
He had previously claimed that he wanted to remain in his role following the goalless draw with Denmark earlier this week. He had failed to secure a competitive win for Ireland over the course of the calendar year.
Martin ONeill & Roy Keane axed Ireland suffer Nations League relegation
Veteran players Robbie Keane and Shay Given retired after the end of the tournament but the Republic made a promising start to their 2018 World Cup qualifiers before a scoreless draw against Wales triggered a run of four competitive fixtures without a win and the heavy play-off loss to the Danes, which ended their prospects of qualifying for Brazil.
In his 55 games in charge, ONeill had a win ratio of just over 34% with 19 wins, 20 draws and 16 defeats and he used a total of 65 players, including 28 international debutants.
ONeill had been at the helm since 2013, leading the country to the last 16 of Euro 2016, and stands down by mutual consent after a disappointing UEFA Nations League campaign.
The 66-year-old handed international debuts to 12 players this year as the former Leicester, Sunderland and Celtic manager looked to build a new-look squad for the start of the Euro 2020 qualifying campaign.
The Republic will be among the third seeds for the European qualifiers when the draw is made in Dublin on 2 December.
Read more ONeill has gone after the team finished bottom of their Nations League group, his final match being a goalless draw in Denmark this week in which Ireland did not have a shot on target.
The former Leicester and Celtic manager took the post in November 2013 and led Ireland to the last 16 of Euro 2016. ONeill departs amid growing disillusionment among fans, the side having gone 397 minutes without a goal and having won only one of their nine games in 2018.
The Football Association of Irelands chief executive, John Delaney, said: I would like to thank Martin, Roy, and the management team for the impact that they had with the Ireland team. There have been many highlights during Martins reign – none more so than Euro 2016 in France, which will live long in the memory of all Irish supporters.