curses.textpad.Textbox mishandles double-width characters

[fedonman]

Bug description:

Lib/curses/textpad.py shifts and reads the window one cell at a time, so a double-width (East Asian) character, which occupies two cells, is mishandled in three ways: insert mode duplicates it and drops a character from the line, gather() reports every one of them twice, and typing one whose second cell would fall in the last column raises curses.error.

import curses, sys
from curses.textpad import Textbox

def check(stdscr):
    win = curses.newwin(1, 12, 0, 0)
    box = Textbox(win, insert_mode=True)
    for ch in 'abc':
        box.do_command(ch)
    win.move(0, 0)
    box.do_command('漢')
    insert = win.instr(0, 0, 12).decode().rstrip()

    win = curses.newwin(1, 12, 0, 0)
    box = Textbox(win)
    for ch in '你好':
        box.do_command(ch)
    window, gathered = win.instr(0, 0, 12).decode().rstrip(), box.gather().rstrip()

    win = curses.newwin(1, 4, 0, 0)
    box = Textbox(win)
    try:
        for ch in '你好':
            box.do_command(ch)
        lineend = 'no exception'
    except curses.error as exc:
        lineend = 'curses.error: %s' % exc
    return insert, window, gathered, lineend

insert, window, gathered, lineend = curses.wrapper(check)
print(f"insert 漢 before 'abc': {insert!a}\n"
      f"typed '你好', window:  {window!a}\n"
      f"typed '你好', gather(): {gathered!a}\n"
      f"'你好' into 4 columns:  {lineend}", file=sys.stderr)

Output:

insert 漢 before 'abc': '\u6f22ac'
typed '你好', window:  '\u4f60\u597d'
typed '你好', gather(): '\u4f60\u4f60\u597d\u597d'
'你好' into 4 columns:  curses.error: wadd_wch() (called by addch()) returned ERR

Expected: '漢abc' from the insert, gather() equal to the window's own '你好', and no exception in the last case. Doc/library/curses.rst says of Textbox that "Entering and reading back the full Unicode range, including combining characters, is now supported when curses is built with wide-character support".

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-156207: Fix curses Textbox.gather() for double-width characters #156208