There has been no “significant shift” by the government in cross-party Brexit talks, John McDonnell has said.
The shadow chancellor said Labour had not seen enough movement from ministers to reach a deal, especially on the issue of a customs union with the EU.
He insisted Labour had “compromised in some areas”, but added: “We’re not near what we want.”
Foreign Secretary Jeremy Hunt said a compromise was not impossible but talks could not continue “indefinitely”.
Following a cabinet meeting on Tuesday, BBC political correspondent Nick Eardley said ministers had agreed that talks with Labour would continue.
The discussions have been going on for weeks with little sign of progress.
Speaking at a Wall Street Journal event in London, Mr McDonnell criticised a letter from senior Tories to Theresa May urging her not to agree a deal with Labour that includes a customs union.
The letter has been signed by 13 former Tory cabinet ministers and Sir Graham Brady, chairman of the backbench 1922 Committee.
Mr McDonnell said “the customs union element is absolutely key to us”, but the letter gave Labour “no security” that any deal done would be honoured in the long-term – especially once Mrs May is replaced as Tory leader.
“We’ve gone into this in really good faith, we’ve tried to put party politics to one side…” hesaid.
“Our big problem now is if we’re going to march our troops in Parliament to the top of the hill to vote for a deal and then that’s overturned, literally, in weeks, I think that would be a cataclysmic act of bad faith.”
But speaking at the same event, Mr Hunt said there was “potential” for a deal because it was in the interests of both main parties to resolve the Brexit impasse.
“Both of us would be crucified by our base if we went into a general election having promised that we would respect the referendum result and not having respected it,” he added.
Attempts to find a cross-party compromise began after Theresa May’s Brexit deal was rejected three times by MPs.
The inability to agree on a way forward led the UK to miss its 29 March deadline for leaving the EU – the current date for departure is 31 October.
So far both sides have resisted calls to set a deadline on the negotiations.
But the prime minister’s official spokesman said the government believed it was “imperative” that the Withdrawal Agreement Bill – the legislation required to leave the EU – was brought to Parliament in time for it pass all its stages by the summer recess.
No date has so far been set for the summer recess, but Parliament usually rises towards the end of July.
Robbins in Brussels
The cabinet discussions came as the PM’s Brexit negotiator Olly Robbins travelled to Brussels to explore the scope for changes to the political declaration between the UK and the EU.
The document sets out the parameters for the future relationship, and Labour negotiators have insisted that any deal they strike with ministers must be reflected in changes to it.
They want a permanent and comprehensive customs union with the EU after Brexit, meaning there would be no internal tariffs (taxes) on goods sold between the UK and the rest of the bloc.
But it would mean the UK cannot negotiate its own trade deals on goods with other countries around the world, something many Brexit-supporting Tory MPs support.
A Downing Street source told BBC political editor Laura Kuenssberg that a compromise was being sought with Labour on customs “as an interim position or a stepping stone”.
“We will not sign up to a permanent customs union,” the source said.
Analysis: By Nick Eardley, BBC political correspondent
The big question at Westminster is how long can these talks go on for. The answer appears to be, a little while longer. But the odds are stacked against a Tory-Labour compromise.
Labour doesn’t think the government has moved far enough. They remain worried that Theresa May’s replacement will come in and decide they don’t like what has been agreed and try to rip it up.
On the other side, prominent Tories are seething at the idea a customs union could be the price of getting a Brexit deal through.
No 10 sources said this morning the PM won’t agree to a permanent customs union – but the idea of a temporary solution is exactly what frightens many on the Labour side.
In this process, taking a step to keep one group happy appears to mean you make another more annoyed.
Cabinet agreed today it was “imperative” that legislation to allow the UK’s withdrawal is brought back to Parliament in time to pass before the summer recess. That sets up the prospect of another set of big Brexit votes in the coming weeks.
But for now, there’s more talk than action.
Mr McDonnell also said Labour had told ministers they “may well have to concede that there is a public vote of some sort” to get a deal through Parliament.
At the weekend, shadow Brexit secretary Sir Keir Starmer said a “significant number, probably 120 if not 150” of Labour MPs would not back deal without a “confirmatory vote”.
On the prospect of another referendum, Mr McDonnell said: “My view is that you’d put the deal to the people, but you’d have to also have the option of the status quo.
“Deep in my heart, I’m still a Remainer, but I’ve got to try and bring together effectively what is a British compromise.”
Asked if Labour leader Jeremy Corbyn was also a Remainer in his heart, the shadow chancellor responded: “Yes.”
But Jeremy Hunt said another referendum or a general election were the “least likely outcomes” of the current Brexit stalemate.
“When approximately half your constituents have voted to leave the EU, just imagine their anger if you went on to support a second referendum where you’re basically saying ‘we think you got it wrong first time’.”