Skip to content

Do not allow calling NotImplemented#2992

Open
Louisvranderick wants to merge 1 commit intofacebook:mainfrom
Louisvranderick:fix-2922-reject-dataclass-enum
Open

Do not allow calling NotImplemented#2992
Louisvranderick wants to merge 1 commit intofacebook:mainfrom
Louisvranderick:fix-2922-reject-dataclass-enum

Conversation

@Louisvranderick
Copy link
Copy Markdown
Contributor

Summary

NotImplemented is a singleton, not callable, but bundled stdlib stubs declare NotImplementedType with an Any base. Pyrefly then treated instances as implicitly callable when no call was found (has_base_any → implicit Any call target). This change skips that implicit-call path for types.NotImplementedType and builtins._NotImplementedType, so NotImplemented(...) is reported as not callable. Adds a regression test in pyrefly/lib/test/callable.rs.

Fixes #2918

Test Plan

cargo test -p pyrefly test_notimplemented_not_callable
cargo test -p pyrefly test_return_notimplemented (ensure return NotImplemented behavior unchanged)
cargo test -p pyrefly test_no_missing_return_for_stubs
python3 test.py --no-test --no-conformance (no generated conformance files touched for this change)

@meta-cla meta-cla bot added the cla signed label Apr 2, 2026
@github-actions github-actions bot added the size/m label Apr 2, 2026
@yangdanny97 yangdanny97 changed the title Fix 2922 reject dataclass enum Do not allow calling NotImplemented Apr 2, 2026
Copy link
Copy Markdown
Contributor

@yangdanny97 yangdanny97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit summary is right but the actual changes are for something else entirely, i think you got your commits mixed up

@Louisvranderick Louisvranderick force-pushed the fix-2922-reject-dataclass-enum branch from 01b5d24 to 73c25c2 Compare April 4, 2026 23:05
@github-actions github-actions bot added size/xs and removed size/m labels Apr 4, 2026
@Louisvranderick
Copy link
Copy Markdown
Contributor Author

Sorry about that I had stacked an unrelated commit on this branch. It’s clean now: only the #2918 NotImplemented change remains after rebasing on latest main and force-pushing.

@github-actions github-actions bot added size/xs and removed size/xs labels Apr 5, 2026
@Louisvranderick Louisvranderick force-pushed the fix-2922-reject-dataclass-enum branch from 5623998 to 99fa21b Compare April 6, 2026 19:12
Copy link
Copy Markdown
Contributor

@stroxler stroxler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the files (see the Changes view on the PR) are still not right.

It looks to me like there are multiple enum-related features mixed into the changes associated with this PR (for example, no @dataclass decorator on Enum)

@stroxler stroxler self-assigned this Apr 7, 2026
@iamPulakesh
Copy link
Copy Markdown

@stroxler are you working on this? or else i can work on it.

@stroxler
Copy link
Copy Markdown
Contributor

stroxler commented Apr 9, 2026

@iamPulakesh I assigned the PR review to myself, but I'm not working on the issue. I'm not sure whether @Louisvranderick is planning to do more work - at the moment I think there are some issues with unrelated code in the PR (and tests failing)

@iamPulakesh
Copy link
Copy Markdown

@iamPulakesh I assigned the PR review to myself, but I'm not working on the issue. I'm not sure whether @Louisvranderick is planning to do more work - at the moment I think there are some issues with unrelated code in the PR (and tests failing)

Thanks for the update @stroxler! I looked at the issue and i think the fix could be handled by overriding the stub to drop the Any base, though I need to verify something else first. If @Louisvranderick doesn't get back to it, i'll be happy to work on it. I'll open a PR on this tomorrow.

@iamPulakesh
Copy link
Copy Markdown

@iamPulakesh I assigned the PR review to myself, but I'm not working on the issue. I'm not sure whether @Louisvranderick is planning to do more work - at the moment I think there are some issues with unrelated code in the PR (and tests failing)

Applied my fixes #3101. Please take a look

NotImplementedType stubs inherit Any, so has_base_any made the singleton
callable via the implicit-Any call path. Exclude types.NotImplementedType
and builtins._NotImplementedType from that branch.

Tests: callable/calls regressions; return NotImplemented unchanged.
Made-with: Cursor
@Louisvranderick Louisvranderick force-pushed the fix-2922-reject-dataclass-enum branch from ed59b58 to df1fc7a Compare April 10, 2026 21:19
@Louisvranderick
Copy link
Copy Markdown
Contributor Author

Thanks @yangdanny97 @stroxler — you were right the branch had unrelated enum / LitEnum / dataclass-on-Enum and conformance churn. I have reset the PR so it only contains the #2918 fix: the has_base_any implicit-callable guard for NotImplementedType / _NotImplementedType, plus tests in callable.rs / calls.rs. No other semantic changes.

Happy for you to keep iterating on the broader enum / dataclass behavior separately; this PR is intentionally minimal so it can land without that scope. (There is also #3101 with another approach to the same issue — if maintainers prefer one implementation, I am fine closing this in favor of that.)

@github-actions github-actions bot added size/xs and removed size/l labels Apr 10, 2026
@github-actions
Copy link
Copy Markdown

Diff from mypy_primer, showing the effect of this PR on open source code:

scipy (https://github.com/scipy/scipy)
+ ERROR scipy/stats/tests/test_qmc.py:538:20-29: Expected a callable, got `NotImplementedType` [not-callable]
+ ERROR scipy/stats/tests/test_qmc.py:543:24-33: Expected a callable, got `NotImplementedType` [not-callable]

@github-actions
Copy link
Copy Markdown

Primer Diff Classification

❌ 1 regression(s) | 1 project(s) total | +2 errors

1 regression(s) across scipy. error kinds: not-callable.

Project Verdict Changes Error Kinds Root Cause
scipy ❌ Regression +2 not-callable pyrefly/lib/alt/call.rs
Detailed analysis

❌ Regression (1)

scipy (+2)

The QMCEngineTests class (line 519-690) uses qmce = NotImplemented as a placeholder (line 521) that subclasses are expected to override. All concrete test subclasses do override it: TestHalton.qmce = qmc.Halton (line 694), TestLHS.qmce = qmc.LatinHypercube (line 727), TestSobol.qmce = qmc.Sobol (line 812), TestPoisson.qmce = qmc.PoissonDisk (line 906). The engine() method at lines 529-543 calls self.qmce(...), which at runtime will always be one of these concrete callable classes. Pyrefly is looking at the base class definition and seeing NotImplementedType, then flagging the call. This is a common Python pattern for abstract-like base test classes. Neither mypy nor pyright flags this. The error is a false positive — the code is correct because self.qmce is always overridden before these methods are called.
Attribution: The change in pyrefly/lib/alt/call.rs in the AnswersSolver added special-casing for NotImplementedType to skip the has_base_any implicit callable path. This correctly prevents NotImplemented() from being treated as callable. However, pyrefly is now flagging self.qmce(...) calls in the base class QMCEngineTests because it sees qmce = NotImplemented at the class level and infers the type as NotImplementedType. It doesn't account for the fact that subclasses override this attribute with actual callable types. The PR's intent was correct (flag direct NotImplemented() calls), but the collateral damage on this pattern — using NotImplemented as a sentinel placeholder in a base class that subclasses override — is a false positive.

Suggested fixes

Summary: The PR correctly prevents NotImplemented() from being called, but causes false positives when NotImplemented is used as a sentinel placeholder in base classes that subclasses override with callable types.

1. In the callable checking logic in pyrefly/lib/alt/call.rs (around line 366-380), the issue is not with the NotImplementedType special-casing itself — that's correct for direct NotImplemented() calls. The real problem is that when pyrefly resolves self.qmce(...) in the base class QMCEngineTests, it only sees the base class attribute type NotImplementedType and doesn't consider that subclasses override it. This is a pre-existing limitation in how pyrefly resolves attribute types on self — it uses the declaring class's annotation rather than considering the full class hierarchy. The NotImplementedType change just made this visible because previously NotImplementedType was treated as callable (via has_base_any). A targeted fix: in the callable resolution path, when the callee type is NotImplementedType AND it comes from an attribute access on self (not a direct name lookup of NotImplemented), AND the containing class is not the class that will be instantiated at runtime (i.e., it could be a base class), consider falling back to a less strict check or suppressing the error. However, a simpler and more correct fix would be: only apply the NotImplementedType non-callable guard when the expression being called is literally the name NotImplemented (a direct reference to the builtin constant), not when it's accessed via self.attr. This could be done by checking the call expression's AST node — if it's a simple Name node resolving to NotImplemented, apply the guard; if it's an attribute access like self.qmce, don't apply the special NotImplementedType block. Concretely, pass information about whether the callee is a direct name reference vs an attribute access into the callable resolution, and only block NotImplementedType callability for direct references.

Files: pyrefly/lib/alt/call.rs
Confidence: medium
Affected projects: scipy
Fixes: not-callable
The 2 new errors in scipy are both pyrefly-only (0/2 mypy, 0/2 pyright). They arise because self.qmce in the base class QMCEngineTests has type NotImplementedType (from qmce = NotImplemented), but at runtime it's always overridden by subclasses with actual callable types. Before this PR, NotImplementedType was treated as callable via the has_base_any path, masking the issue. The fix should distinguish between calling NotImplemented directly (which should error) vs calling an attribute that happens to have NotImplementedType in a base class (which may be overridden). This eliminates 2 not-callable errors in scipy.


Was this helpful? React with 👍 or 👎

Classification by primer-classifier (1 LLM)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Calling NotImplemented (a constant, not a class)

4 participants