Merge remainder of pyproject support

[scbedd]

I snipped this from #39100

Not mandating anything about keyvault, but wanted to get this merged to main for anyone else who wants to ship a pyproject.toml project.

[Copilot]

Pull Request Overview

This PR removes residual support for pyproject by updating various build and package discovery functions and improving logging and command interfaces. Key changes include:

• Adding a new CLI command "sdk_build_package" in setup.py.
• Updating package parsing in scenario/generation.py to work directly with pyproject paths.
• Adjusting tox and dependent package installation configurations to align with the new pyproject support.

Reviewed Changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tools/azure-sdk-tools/setup.py	Added new CLI entry for building packages.
tools/azure-sdk-tools/ci_tools/scenario/generation.py	Replaced direct setup.py path usage with target_setup for package parsing.
tools/azure-sdk-tools/ci_tools/functions.py	Updated logging message to reflect changes in requirement updates.
tools/azure-sdk-tools/ci_tools/build.py	Introduced build_package function and improved error checking in build commands.
eng/tox/verify_sdist.py	Updated glob search to be recursive to locate sdists.
eng/tox/tox.ini	Adjusted tox commands to use the new build_package interface.
eng/tox/install_depend_packages.py	Renamed parameter and updated path joining for clarity.
eng/scripts/get_package_properties.py	Enhanced package discovery to check for pyproject support.
eng/ci_tools.txt	Updated build version reference.
eng/test_tools.txt	Added additional package requirements for specific Python versions.

Comments suppressed due to low confidence (3)

tools/azure-sdk-tools/ci_tools/scenario/generation.py:70

• Switching from using a joined 'setup.py' path to passing 'target_setup' directly may affect how ParsedSetup locates the project configuration. Please verify that ParsedSetup.from_path accepts a directory containing pyproject.toml or setup.py as intended.

    target_package = ParsedSetup.from_path(target_setup)

tools/azure-sdk-tools/ci_tools/scenario/generation.py:106

• Removing the explicit 'setup.py' in the file path when parsing requirements might impact behavior if ParsedSetup expects a specific file. Please confirm that this modification is compatible with projects using both setup.py and pyproject.toml.

                requirements = ParsedSetup.from_path(os.path.join(os.path.abspath(target_setup))).requires

eng/tox/install_depend_packages.py:329

• [nitpick] The variable renaming from 'setup_py_path' to 'package_directory' should be reflected in the associated documentation/comments. Consider updating the docstring for filter_dev_requirements to clarify this change for better maintainability.

    dev_req_path = os.path.join(package_directory, DEV_REQ_FILE)