Recursive right-to-left segmenting (tokenizing) of strings in Python

Here are some general comments:

1. Having a function mutate a global variable is a bad practice. In this case, it may seem easy enough to reason about what is going on, but I guarantee that at some point it will lead to an insidious bug that is hard to track down because you have no idea who is changing the global variable and when.
2. Recursion is not necessary here; it over-complicates the solution and may make it difficult to change/update the code.

The code itself is rather concise, but the weird data-flow out of the function makes it difficult to understand what's going on. This is complicated because the function is recursive and so you then have to keep track of the calling stack when figuring out how segmented_morphs is being changed.

I don't think the re module is really necessary here, I bet that it is heavier that just searching for strings and doesn't seem to offer any benefit. I've kept it for now, but I might offer another solution without it.

Here's how I would approach the problem:

import re

def segment_word(word, morphs):
    regex = re.compile('(' + '|'.join(morphs) + ')' + '$')
    splitter = lambda s: (regex.split(s) + [None])[:2]

    segments = []
    word, end = splitter(word)
    while end is not None:
        segments.append(end)
        word, end = splitter(word)

    if word:
        segments.append(word)

    return segments[::-1]

def segment_phrase(phrase, morphs):
    return [s for w in phrase.split() for s in segment_word(w, morphs)]

Here's a version without re, I'm not super happy about either. Neither version seems particularly elegant.

def segment_word(word, morphs):
    segments = []
    previous_length = -1
    while len(word) != previous_length:
        previous_length = len(word)
        for morph in morphs:
            if word.endswith(morph):
                segments.append(morph)
                word = word[:-len(morph)]

    if word:
        segments.append(word)

    return segments[::-1]

Your solution is rather awkward due to the weird recursion that mutates a global segmented_morphs variable. Well designed functions should never have side-effects like that.

For that matter, you should probably avoid capturing morphs as a global variable as well: it could be a parameter, or perhaps a member variable in an object.

I also don't like the use of re.split(), when you are really only interested in matching.

Regex searches produce the leftmost longest matches. Therefore, the trick to using regular expressions to find suffixes is to reverse everything:

• reverse the subject string,
• reverse the morphs that you are looking for,
• reverse the matches,
• reverse the text within each match.

Note the use of \S+ in the regex below: it catches any unrecognized prefix in each word. Also, it's good practice to escape strings when composing regexes, in case the constituent parts contain any regex metacharacters.

import re

def segment(suffixes, text):
    """
    Segment the text into a list of strings.

    >>> morphs = ['house', 'flower', 'system', 'collapse']
    >>> segment(morphs, 'greenhouse')
    ['green', 'house']
    >>> segment(morphs, 'flowering plants')
    ['flowering', 'plants']
    >>> segment(morphs, 'ecosystemcollapse')
    ['eco', 'system', 'collapse']
    """
    suffix_re = '|'.join(re.escape(s[::-1]) for s in suffixes) + r'|\S+'
    return [
        match.group()[::-1] for match in re.finditer(suffix_re, text[::-1])
    ][::-1]

If you intend to reuse the same morphs to analyze multiple phrases, here's an object-oriented design:

class Segmenter:
    def __init__(self, suffixes):
        self.re = re.compile(
            '|'.join(re.escape(s[::-1]) for s in suffixes) + r'|\S+'
        )

    def segment(self, text):
        return [
            match.group()[::-1] for match in self.re.finditer(text[::-1])
        ][::-1]

phrases = …
segmenter = Segmenter(['house', 'flower', 'system', 'collapse'])
for phrase in phrases:
    print(segmenter.segment(phrase))