gh-143050: correct PyLong_FromString() to use _PyLong_Negate()

[skirpichev]

The long_from_string_base() might return a small integer, when the _pylong.py is used to do conversion. Hence, we must be careful here to not smash it "small int" bit by using the _PyLong_FlipSign().

Co-authored-by: Sam Gross colesbury@gmail.com

• Issue: Remove redundant decref from _PyLong_Negate #143050

[skirpichev]

Shouldn't affect users, so - no news.

CC @colesbury, @eendebakpt

[skirpichev]

_long_is_small_int() logic used also in Modules/_testcapi/immortal.c. What about moving this inlined function to the Include/internal/pycore_long.h as _PyLong_IsImmortal()?

[skirpichev]

I'm slightly worry that this pass will non-debug builds.

@corona10 (author of #103962), are you sure there is no other way to implement this helper? Return a different value, raise an error?

[eendebakpt]

Calling _PyLong_FlipSign this on the 0 small int caused the problems. Can we add an assert in _PyLong_FlipSign:

static inline void
_PyLong_FlipSign(PyLongObject *op) {
    assert(!__PyLong_IsSmallInt((PyObject *)op));
    unsigned int flipped_sign = 2 - (op->long_value.lv_tag & SIGN_MASK);
    op->long_value.lv_tag &= NON_SIZE_MASK;
    op->long_value.lv_tag |= flipped_sign;
}

The __PyLong_IsSmallInt does not exist yet, I used the following in testing:

static inline int
/// Return 1 if the object is one of the immortal small ints
_PyLong_IsSmallInt(PyObject *op)
{
    // slow, but safe version, for use in assertions and debugging.
    for(int i = 0; i < _PY_NSMALLPOSINTS + _PY_NSMALLNEGINTS; i++) {
        if (op == (PyObject *)&_PyLong_SMALL_INTS[i]) {
            return 1;
        }
    }
    return 0;
}

In a similar way we could add an assert to _PyLong_SetSignAndDigitCount.

[skirpichev]

Calling _PyLong_FlipSign this on the 0 small int caused the problems.

Yes, but any small int will trigger this. We can add test case for -1, for example.

Can we add an assert in _PyLong_FlipSign

My suggestion would be using assert(!_PyLong_IsImmortal((PyObject *)op));: as the "small int" bit will be cleared by the _PyLong_FlipSign(). (_PyLong_IsImmortal() is current _long_is_small_int().)

In a similar way we could add an assert to _PyLong_SetSignAndDigitCount.

This will affect all "setters" in Include/internal/pycore_long.h (i.e. also _PyLong_SetDigitCount).

[skirpichev]

I've added _PyLong_IsImmortal() helper.

In a similar way we could add an assert to _PyLong_SetSignAndDigitCount.

I tried to add assert here and in _PyLong_SetDigitCount(), but got:

$ make -s
_freeze_module: ./Include/internal/pycore_long.h:291: _PyLong_SetSignAndDigitCount: Assertion `!_PyLong_IsImmortal(op)' failed.
Aborted
make: *** [Makefile:1952: Python/frozen_modules/getpath.h] Error 134

[eendebakpt]

Also add the test case int('-' + '0' * 7000 + '123', 10)? That case was already fine on the main branch, but it would not hurt to add it I think.

[skirpichev]

That case was already fine on the main branch

No, this is not fine in the main branch: such case also trigger assertion. Any small int "works" in this sense. Keep in mind, that this test will pass on non-debug builds, see above: #145901 (comment)