gh-73123: Add a keepempty argument to string, bytes and bytearray split methods

[MarkCBell]

This PR adds an optional keepempty argument to string.split (and similarly for bytes.split, bytearray.split and UserString.split). As described in issue bpo-28937:

• If keepempty is true then empty strings are never stripped out of the result array.
• If keepempty is false then empty strings are always stripped out of the result array.
• If keepempty is None (the default) then the current behaviour is followed in which empty strings are stripped out of the result array if and only if the separator string is None.

To do this it uses a new splitting algorithm which has been designed to be compatible with the existing maxsplit argument. This is roughly:

def split(string, sep=None, maxsplit=None, keepempty=None):

    prune = sep is None if keepempty is None else not keepempty
    if sep is None: sep = ' '
    # Ok, the real implementation actually matches on any whitespace,
    # but matching on ' ' is good enough for this toy example.

    results = []
    count = 0
    i = 0
    j = string.find(sep, i)
    while j >= 0:
        if j > i or not prune:
            if maxsplit is not None and count >= maxsplit:
                break
            results.append(string[i:j])
            count += 1
        i = j + len(sep)
        j = string.find(sep, i)

    if i < len(string) or not prune:
        results.append(string[i:])

    return results

A number of tests have been added to check the correct behaviour.

https://bugs.python.org/issue28937

• Issue: str.split(): allow removing empty strings (when sep is not None) #73123

[tiran]

PyUncode_Split() and PyUnicode_RSplit() are in the stable ABI and API. You cannot modify the function arguments. Instead you have to create additional functions.

[bedevere-bot]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[MarkCBell]

Thank you for letting me know about the API / ABI. I have changed this PR so that it leaves PyUncode_Split() and PyUnicode_RSplit() alone and created two new methods PyUncode_SplitWithKeepempty() and PyUnicode_RSplitWithKeepempty() that provide access to the new functionality instead.

I have decided that the C API interfaces add little extra value and so have removed them. If it is decided later that these are useful then they can be re-added by reverting commit f95b254.

Now that I have made the requested changes; please review again.

[bedevere-bot]

Thanks for making the requested changes!

@tiran: please review the changes made to this pull request.

[MarkCBell]

Could a maintainer approve running the last two jobs of the CI workflow for me please.

[akulakov]

I've left a comment in the issue tracker, I think there may be a potential problem with the API.

[github-actions]

This PR is stale because it has been open for 30 days with no activity.