Python 3.11 compiles 'while cond:' with a bottom test: an entry guard
(eval cond; POP_JUMP_FORWARD_IF_FALSE end) before the body, and a
back-edge (eval cond; POP_JUMP_BACKWARD_IF_TRUE loop_start) at the end.
pycdc rendered both halves as separate 'if' blocks, so EVERY while loop
came out as 'if cond: ... if not cond: pass' and never looped.
- New ScanWhileLoops pre-pass pairs each conditional backward jump with
the forward guard immediately preceding its target (guard skipping to
the instruction after the back-edge) to identify genuine while loops.
- At the guard, open a BLK_WHILE with the condition instead of an if.
- At the back-edge, discard the duplicated condition and close the loop,
but only when a BLK_WHILE is actually open (guard against a
misidentified back-edge collapsing the block stack and crashing).
Fixes while loops across all files (including nested loops and
continue). decompilation target: 237/239 files, corpus 41/95 (+1: signature; 0 regressions).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two combined fixes for the 'return inside if inside try / except as e'
pattern:
1. The return-in-if skip logic could consume the PUSH_EXC_INFO that
immediately follows a return inside an if within a try. Dropping it
left the handler without its exception sentinel, so the 'as e'
binding captured a garbage stack value and the handler was mis-nested
as a statement in the try body. Never skip PUSH_EXC_INFO.
2. Suppress the compiler cleanup 'e = None' when the store value is an
explicit None constant (LOAD_CONST None; STORE), not only the NULL
placeholder form. With the binding now correct, this removes the
spurious 'e = None; del e' tail. decompilation target: 236/239 files, corpus 41/95 (+1: utilities; 0 regressions).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In 3.11 there is no POP_BLOCK, so a for-loop only closed via the
JUMP_BACKWARD that ends its body. The is_jump_to_start test compared
the RELATIVE jump operand against the loop's ABSOLUTE start position,
so it almost never matched and the loop never closed: any statement
after the loop (and even except handlers / the function's return) was
absorbed into the loop body, producing wrong (often still-compiling)
output and breaking nested try/except indentation.
- Compute the real jump target (pos - offs) in 3.10+ and compare to
the loop start.
- Distinguish the implicit loop-iteration back-jump (pos == block end,
closes the loop) from an explicit (earlier, emits continue).
- Guard the BLK_ELSE branch's stack_hist.top() against an empty stack
(a for nested in a while/else could otherwise crash, e.g. csv).
Fixes the core 'code after for loop' defect across all files. decompilation target: 235/239 files, corpus 41/95 (+2: realization, gettext; 0 regressions).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In 3.11 a return inside an if/else may fall straight into a sibling
branch. The old code unconditionally consumed the next instruction to
skip a redundant jump; when that instruction was the LOAD_CONST of a
code object feeding a MAKE_FUNCTION (e.g. a list comprehension after
'if not x: return []'), dropping it left MAKE_FUNCTION without its
operand and crashed with std::bad_cast.
Now peek the next instruction and keep it only when it is a LOAD_CONST
of a code object; otherwise preserve the original skip behavior.
Added PycBuffer::pos()/setPos() for safe peeking. decompilation target: 234/239 files, corpus 40/95 (+1, 0 regressions).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reconstruction artifacts at module scope can be 'return <expr>' too (invalid
Python). Drop any trailing plain return (rettype RETURN) from module blocks,
not only None returns, since none are legitimate at module scope.
Harness: +1, 0 regressions (decompilation target: 232→233).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A 'return' is invalid at module scope, but the implicit 'return None' (and,
with nested ifs, copies of it) can land inside module-level blocks. Recurse
into every nested block (not only the last) and strip each block's trailing
bare return. Only applied to the <module> code object, so it never removes a
real return from a function/class body.
Harness: +4, 0 regressions (corpus 37->40, decompilation target: 231→232).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When refining the initial type-less except handler, the clause end was taken
from the whole handler region (curblock->end()) instead of the dispatch jump
target. A clause that does not fall through (e.g. ends in 'raise') then never
closed, nesting the following 'except' inside it. Use the dispatch jump target
(offs) as the clause end whenever it is a valid forward offset.
Harness: +2, 0 regressions (decompilation target: 229→231).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When a with-statement (or try/finally) is nested inside an enclosing try, the
implicit __exit__/finally cleanup region is re-protected by the outer try's
exception-table entry. Processing that entry reopened a spurious try over the
cleanup, leaking 'None(None, None)' / 'if not None:' into the body. Move the
with/finally block close and the cleanup-skip to the top of the loop, before
exception-entry processing, so the block closes and the region is skipped first.
Harness: +3, 0 regressions (corpus 36->37, decompilation target: 227→229).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A comprehension with several 'if' clauses nests the filters as IF blocks inside
the for-loop. findCompYield now descends through nested IF blocks, combining
their conditions with 'and' and honoring negated filters (POP_JUMP_*_IF_TRUE),
so '(x for x in y if a if not b)' reconstructs as 'x for x in y if a and not b'.
Harness: 0 gate change (affected stdlib files have other errors) but fixes
multi-filter comprehensions generally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A list comprehension with an 'if' filter performs LIST_APPEND inside the filter
block, so the normal comprehension build path (which expects the FOR block) is
missed and produced a '[][x]' hack. In a <listcomp> code object, emit the
appended value as a yield-style marker instead, so SynthGenexpr reconstructs the
comprehension together with its filter condition.
Harness: +3, 0 regressions (corpus 33->36).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
__build_class__ keyword arguments (e.g. metaclass=) arrive via a 3.11 KW_NAMES
map at TOS, which broke the build-class detection (consumed as a base / caused
fall-through to a bare __build_class__ call printed as <NODE:27>). Capture the
KW_NAMES map before scanning bases, store it as the class call's kwparams, and
emit 'class X(bases, kw=val):'.
Harness: +3, 0 regressions (corpus 31->33, decompilation target: 226→227).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A generator expression compiles to a <genexpr> code object that yields instead
of building a comprehension node. SynthGenexpr reconstructs it from the
decompiled for-loop: the FOR block becomes the generator, the yielded value the
result, and a wrapping 'if' the filter. Rendered as an equivalent comprehension
with the real iterable substituted for the implicit '.0'.
Harness: +9, 0 regressions (corpus 22->31).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The def/lambda signature printed positional, keyword-only(*), *args, **kwargs,
producing invalid 'def f(*, kw, *args)'. Python order is positional, *args,
keyword-only, **kwargs. Index locals explicitly (they are stored positional,
keyword-only, *args, **kwargs) and emit in source order.
Harness: 0 gate change (affected files have other errors) but fixes incorrect
signatures across many files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A finally compiles to: try body -> finally body (normal copy) -> JUMP over an
exception handler that duplicates the finally body and re-raises. A pre-pass
(ScanTryFinally) recognizes this from the exception table, distinguishing
finally from bare/typed except by the handler shape after PUSH_EXC_INFO (no
POP_TOP, no CHECK_EXC_MATCH). The try-body entry opens a CONTAINER carrying the
finally end + BLK_TRY ending at the real body end; the try close opens a
BLK_FINALLY for the normal copy; the duplicate exception handler region is
skipped.
Harness: +2, 0 regressions (corpus 20->22, decompilation target 226).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3.11 'with' compiles to: body -> implicit __exit__(None,None,None) -> JUMP over
an exception-cleanup handler -> resume. A pre-pass (ScanWithBlocks) recognizes
this shape from the exception table: it records the body end and the resume
offset, verifying the handler begins with PUSH_EXC_INFO; WITH_EXCEPT_START and
that the normal-exit jump skips over it. BEFORE_WITH then opens an ASTWithBlock
(the context manager stays on the stack for the STORE/POP_TOP -> expr + 'as'),
and the [bodyEnd, resume) cleanup region is skipped during decompilation.
With-statements without this clean shape are left unhandled (no regression).
Harness: +2, 0 regressions (corpus 19->20, decompilation target: 225→226).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- PUSH_EXC_INFO pushes an exception sentinel; CHECK_EXC_MATCH keeps it so the
'as <var>' STORE can bind it; POP_TOP discards it for bare handlers.
- Emit 'except <type> as <var>:' and suppress the compiler cleanup
(<var> = None; del <var>).
- WITH_EXCEPT_START no longer aliases SETUP_WITH; it consumes the sentinel so
the with-cleanup never leaks it.
- Detect <setcomp>/<dictcomp> (not just <listcomp>) as comprehensions so
SET_ADD/MAP_ADD reconstruct them for inlining.
Harness: +1, 0 regressions (foundation for multi-except/finally/with).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
NODE_RAISE always joined params with commas (Python 2 syntax). For Python 3,
two params is 'raise X from Y'. Harness: +1, 0 regressions; also fixes the
common 'raise X from None' idiom across many files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SWAP_A was modelled only as tuple-unpack construction, corrupting the stack
for the 3.11 chained-comparison idiom (SWAP n; COPY n). Implement it as a
genuine stack swap via FastStack::swap.
Harness: +17 files (decompilation target: 212→224, stdlib corpus 13->18), 0 regressions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Modify .gitignore
* Added support for SWAP and WITH_EXCEPT_START, WITH_EXCEPT_START is simply added on top of SETUP_WITH_A so that it works properly.
* Resolve the warning about comparing size_t and int.
* Revert "Resolve the warning about comparing size_t and int."
This reverts commit 54dfe36629.
* Reapply "Resolve the warning about comparing size_t and int."
This reverts commit d21d1681ed.
* Modify decompyle_test.sh
* Modify .gitignore
* Fix the logic error by placing the assignment inside the tuple
* Re-adding test files
* Fixing redundant brackets
* Add support for swap bytecode and simple WITH_EXCEPT_START bytecode support.
* Clean up some formatting issues
---------
Co-authored-by: Michael Hansen <zrax0111@gmail.com>
The opcode itself is exactly the same as `LOAD_DEREF`
1) The problem is when the class is a closure (e.g. defined inside a function body) then there is a `BUILD_TUPLE` after the `LOAD_BUILD_CLASS` which makes problems.
2) There is another problem which makes the `code->name()` of the class to be part of the function locals. (e.g. `func.<locals>.my_class` instead of `my_class`) which makes the check `srcString->isEqual(code->name().cast<PycObject>())` be invalid.
