What is the legal basis of this in the EU? Ignoring the fact they could end up stealing IP, it seems like the collected information could easily contain PII, and consent would have to be > freely given, specific, informed and unambiguous. In order to obtain freely given consent, it must be given on a voluntary basis.
It breaks GDPR easily: GDPR enforces you to comply with opt-out by default, no workaround by prefilling before hitting submit. While some think this applies only to personal data, then yes. But it takes only one line of code to use my phone number for testing while I test locally a register form in the application I'm developing. Once it gets sent to Copilot I can threaten with legal action if they are not taking it…
If you don't want to wait until your PII inevitably gets sent through, you can already now file a complaint to your local supervisory authority: https://www.edpb.europa.eu/about-edpb/about-edpb/members_en