The controversial named person scheme is to be scrapped six years after it was first introduced, John Swinney has announced.
The scheme set out to appoint a single point of contact, such as a teacher or health visitor, to look out for the welfare of children up to the age of 18.
It was introduced in 2013 as part of the Scottish Government's Getting it right for every child (GIRFEC) policy.
Opponents of named person have branded it a "state guardian" scheme that intrudes on family life.
The deputy first minister and education secretary confirmed to MSPs on Thursday the scheme would be dropped after being hobbled by legal battles.
He faced calls to apologise to parents over the years-long attempts to force the policy through.
Critics said the policy was illiberal and a "snooper's charter" that would interfere with family rights.
It had been due to come into force in August 2016 but was delayed after campaigners argued successfully in the Supreme Court that the information-sharing elements of the scheme were contrary to parts of the European Convention on Human Rights.
An expert panel was set up by Swinney in November 2017 after the Scottish Parliament's education committee called for an "authoritative" code of practice on information-sharing.
A Scottish Government investigation has also been launched after details of Swinney's statement was leaked to the media earlier on Thursday.
Addressing MSPs, the deputy FM said the expert panel had not been able to produce a workable code of practice on information-sharing and that trying to impose one would be too complex to be "desirable".
He went on: "The mandatory named person scheme for every child - underpinned by law - will now not happen.
"We will withdraw our bill and repeal the relevant legislation.
"Instead, existing voluntary schemes that provide a point of contact for support will continue under current legal powers, where councils and health boards wish to provide them and parents wish to use them.
"In this way, we will support our children and young people so that they can thrive and rise to the challenges and opportunities that life brings."
He added: "Only through continued investment in our children's wellbeing will we achieve our vision of a prosperous country where everyone gets the chance to fulfil their potential, and no-one is left behind."
But opposition MSPs accused Scottish ministers of "poor government" and called on Swinney to apologise for the "fiasco" around the policy.
Liz Smith, Scottish Conservative shadow education secretary, said: "This is a complete humiliation for the SNP.
"Common sense should have told them years ago that this policy was both universally unpopular and unworkable.
"In the interim, millions of pounds of taxpayers' money has been wasted as has the time that should have been spent addressing the challenges facing our most vulnerable children.
"John Swinney should apologise to all the professionals on the front line who have been faced with endless bureaucracy around this policy and the anxiety of not knowing where their legal responsibilities lay."
She also said in the chamber that "no tears" would be shed following the axing of the "deeply unpopular, illiberal and unworkable" scheme.
Swinney responded: "I think it's really important that the government's intention is to put in place measures that will support and enhance the well-being of children in our society.
"I will not apologise for trying to find the best way to try to do that."
Scottish Labour's education spokesman Iain Gray said the government's handling of the policy since it was introduced in 2014 had been "a shambles".
He said: "The principle of the named person scheme was a good one but it has been destroyed by the incompetence of successive SNP ministers.
"They lost control of the policy, lost the confidence and support of practitioners, parents and the public, then lost the challenge in the supreme court...
"John Swinney was meant to be a 'safe pair of hands' but he has dropped the ball yet again."
He added: "The cabinet secretary talks about good intentions, but good intentions do not alone make for good government.
"And this has been very poor government indeed."
Activist Simon Calvert, of the No to Named Persons (NO2NP) campaign, said: "John Swinney has finally done the right thing - but not before time.
"I remember standing outside the UK Supreme Court in July 2016 when we had just won a court case against the named person scheme and against the data-sharing aspect of it.
"He's eventually done the right thing but he has done a lot of damage to public trust over the last couple of years by trying to prop up the named person scheme."
He went on: "Nevertheless, this is good news for families all across Scotland and it's a great relief for a lot of people.
"I think he still owes an apology to families for insisting on still trying to get legislation through that breached their human rights, and he has a big job of work to do to unpick all of the legally inaccurate training that his government has been giving to officials all across Scotland for years."