In my opinion the idea that we should only plant native plants and must eradicate invasives is comparable to ideas like allowing only white people and deporting brown people.
Plants, like people, have been migrating since ever, through different means of transport like long shoots extending the area of growth and seed dispersal by wind, water or animals. There is no point in history we could point to and say “This is the distribution of plant species over the planet that’s desirable and we should strive to return to that.” Just like there’s no point in history where human people were distributed on the planet in a perfect way we should strive to go back to.
When we talk about neophytes, from a biological perspective we’re referring to different species. People of different origins are (despite perhaps having different skin colors) one and the same species. Therefore, your comparison doesn’t hold up.
If we apply the human perspective to the neophyte debate, it would be like planting maple trees from Canada in a maple forest in the U.S. 200 km further south… which is a marginal ecological problem because they’re the same species.