Commit graph

243 commits

Author SHA1 Message Date
polfg
3cea1b75c2 Fix function signature parameter order (*args before keyword-only)
The def/lambda signature printed positional, keyword-only(*), *args, **kwargs,
producing invalid 'def f(*, kw, *args)'. Python order is positional, *args,
keyword-only, **kwargs. Index locals explicitly (they are stored positional,
keyword-only, *args, **kwargs) and emit in source order.

Harness: 0 gate change (affected files have other errors) but fixes incorrect
signatures across many files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 17:18:41 +02:00
polfg
90c6906505 Reconstruct Python 3.11 try/finally
A finally compiles to: try body -> finally body (normal copy) -> JUMP over an
exception handler that duplicates the finally body and re-raises. A pre-pass
(ScanTryFinally) recognizes this from the exception table, distinguishing
finally from bare/typed except by the handler shape after PUSH_EXC_INFO (no
POP_TOP, no CHECK_EXC_MATCH). The try-body entry opens a CONTAINER carrying the
finally end + BLK_TRY ending at the real body end; the try close opens a
BLK_FINALLY for the normal copy; the duplicate exception handler region is
skipped.

Harness: +2, 0 regressions (corpus 20->22, decompilation target 226).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 17:15:29 +02:00
polfg
2f0e6180b4 Reconstruct Python 3.11 with-statement (common mid-block shape)
3.11 'with' compiles to: body -> implicit __exit__(None,None,None) -> JUMP over
an exception-cleanup handler -> resume. A pre-pass (ScanWithBlocks) recognizes
this shape from the exception table: it records the body end and the resume
offset, verifying the handler begins with PUSH_EXC_INFO; WITH_EXCEPT_START and
that the normal-exit jump skips over it. BEFORE_WITH then opens an ASTWithBlock
(the context manager stays on the stack for the STORE/POP_TOP -> expr + 'as'),
and the [bodyEnd, resume) cleanup region is skipped during decompilation.
With-statements without this clean shape are left unhandled (no regression).

Harness: +2, 0 regressions (corpus 19->20, decompilation target: 225→226).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 17:02:59 +02:00
polfg
27cc6c2a74 Reconstruct 'except ... as e' for Python 3.11 + dict/set comprehension inlining
- PUSH_EXC_INFO pushes an exception sentinel; CHECK_EXC_MATCH keeps it so the
  'as <var>' STORE can bind it; POP_TOP discards it for bare handlers.
- Emit 'except <type> as <var>:' and suppress the compiler cleanup
  (<var> = None; del <var>).
- WITH_EXCEPT_START no longer aliases SETUP_WITH; it consumes the sentinel so
  the with-cleanup never leaks it.
- Detect <setcomp>/<dictcomp> (not just <listcomp>) as comprehensions so
  SET_ADD/MAP_ADD reconstruct them for inlining.

Harness: +1, 0 regressions (foundation for multi-except/finally/with).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 12:00:11 +02:00
polfg
b5bbd70989 Render 'raise X from Y' for Python 3 (was Python-2 'raise X, Y')
NODE_RAISE always joined params with commas (Python 2 syntax). For Python 3,
two params is 'raise X from Y'. Harness: +1, 0 regressions; also fixes the
common 'raise X from None' idiom across many files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 11:50:29 +02:00
polfg
717b3d7d9c Fix SWAP_A as a real stack swap (chained comparisons, starred unpack)
SWAP_A was modelled only as tuple-unpack construction, corrupting the stack
for the 3.11 chained-comparison idiom (SWAP n; COPY n). Implement it as a
genuine stack swap via FastStack::swap.

Harness: +17 files (decompilation target: 212→224, stdlib corpus 13->18), 0 regressions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 11:43:44 +02:00
polfg
7baa5df5c3 Python 3.11 decompilation fixes
- MAKE_FUNCTION 3.6+ flag bitmask (defaults/kwdefaults/annotations/closure)
- decorator reconstruction for functions and classes (3.11 no-PUSH_NULL form)
- consume NULL before LOAD_BUILD_CLASS so decorated classes reconstruct
- strip 3.11 class artifacts (__classcell__ / return __class__)
- try/except inside loops (exception-table stack_depth > 0)
- inline list comprehensions called as code objects (substitute .0)
- guard extra RETURN_VALUE bytecode read against EOF (fixes empty bodies)
- strip implicit module-level 'return None'
- add opcodes: MAKE_CELL, COPY_FREE_VARS, LOAD_ASSERTION_ERROR, DICT_MERGE/UPDATE,
  MAP_ADD, LIST_TO_TUPLE, RETURN_GENERATOR, POP_JUMP_FORWARD/BACKWARD_IF_NONE/etc.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 23:32:11 +02:00
daiche330-png
413fbd83c9 Support Python 3.11 exception table try/except flow 2026-04-06 09:30:16 +09:00
daiche330-png
0b1f7cf070 Handle Python 3.11 exception opcodes in AST builder 2026-04-05 22:13:27 +09:00
Michael Hansen
577720302e Add basic protection aginst circular references in pycdas and pycdc.
This fixes the last case of fuzzer errors detected by #572.
2025-08-28 16:42:03 -07:00
Michael Hansen
0e7be40367 Add some extra guards against null dereference and empty std::stack pops
Fixes segfault cases of #572
2025-08-28 15:36:35 -07:00
Sahil Jain
8b0ea9450e
Fix RAISE_VARARGS bug 2025-07-04 19:16:04 +05:30
Michael Hansen
e64ea4bdec
Merge branch 'master' into new-opcodes 2025-07-02 07:56:28 -07:00
Sahil Jain
6e0089e01c
Update 2025-07-01 22:52:59 +05:30
Sahil Jain
5fe61462a2
Support COPY opcde 2025-07-01 09:25:49 +05:30
Sahil Jain
ad5f39db56
Support SLICE opcodes 2025-07-01 09:24:53 +05:30
Sahil Jain
97ec04789d
Add JUMP_BACKWARD + CACHE comments 2025-07-01 09:22:57 +05:30
Sahil Jain
6dae4e801f
Remove COPY opcode 2025-07-01 09:22:57 +05:30
Sahil Jain
040732920b
Add comment 2025-07-01 09:22:57 +05:30
Sahil Jain
a93fd14672
Add new loop tests 2025-07-01 09:22:57 +05:30
Sahil Jain
a4a6a24f3e
Support END_FOR opcode 2025-07-01 09:22:56 +05:30
samy kamkar
bbc19885f4 show INVALID opcode properly if < 0 2024-10-12 13:45:16 -07:00
Michael Hansen
b939aeb87c Update operand documentation for new opcodes and oparg changes.
Also extends the disassembly oparg decoding for new 3.13 additions.
2024-08-07 15:44:36 -07:00
Michael Hansen
48d1bfa59f Fix a null dereference.
Fixes #486
2024-08-06 08:21:12 -07:00
Michael Hansen
0b45b5fa07 Fix FORMAT_VALUE for values that have both a conversion and a format_spec.
Also output the conversion and flags in disassembly.
2024-08-01 13:28:43 -07:00
easyz
6ad3ceb67e
Add support for swap bytecode and simple WITH_EXCEPT_START bytecode support. (#488)
* Modify .gitignore

* Added support for SWAP and WITH_EXCEPT_START, WITH_EXCEPT_START is simply added on top of SETUP_WITH_A so that it works properly.

* Resolve the warning about comparing size_t and int.

* Revert "Resolve the warning about comparing size_t and int."

This reverts commit 54dfe36629.

* Reapply "Resolve the warning about comparing size_t and int."

This reverts commit d21d1681ed.

* Modify decompyle_test.sh

* Modify .gitignore

* Fix the logic error by placing the assignment inside the tuple

* Re-adding test files

* Fixing redundant brackets

* Add support for swap bytecode and simple WITH_EXCEPT_START bytecode support.

* Clean up some formatting issues

---------

Co-authored-by: Michael Hansen <zrax0111@gmail.com>
2024-06-23 11:59:30 -07:00
Alex
8367a8e0ab Return back a fix for Centos6/7 compilation issues not related to shadow ones 2024-03-12 22:58:43 +02:00
Alex
68a697dfc1 Revert "Fix for Centos6/7 compilation issues"
This reverts commit f80b662f77.
2024-03-12 22:38:03 +02:00
Alex
f80b662f77 Fix for Centos6/7 compilation issues 2024-03-08 14:27:36 +02:00
Michael Hansen
1f30136e21 Merge branch 'master' of github.com:ncaklovic/pycdc 2024-02-28 15:43:58 -08:00
Michael Hansen
8e48bf2194
Merge pull request #462 from greenozon/master
Aligning some opcodes for Python 3.11, 3.12: LOAD_GLOBAL, LOAD_ATTR
2024-02-28 15:40:02 -08:00
ncaklovic
32c1ca10bb
Update ASTree.cpp
Co-authored-by: Michael Hansen <zrax0111@gmail.com>
2024-02-28 12:34:32 +01:00
Alex
035f963f3d Aligning some opcodes for Python 3.11, 3.12: LOAD_GLOBAL, LOAD_ATTR 2024-02-27 23:12:03 +02:00
MrDakik
e8be65b2f3 add support for WITH_CLEANUP_START,WITH_CLEANUP_FINISH 2024-02-27 10:54:07 +02:00
MrDakik
00d4b02d1e Added support for LOAD_CLASSDEREF
The opcode itself is exactly the same as `LOAD_DEREF`
1) The problem is when the class is a closure (e.g. defined inside a function body) then there is a `BUILD_TUPLE` after the `LOAD_BUILD_CLASS` which makes problems.
2) There is another problem which makes the `code->name()` of the class to be part of the function locals. (e.g. `func.<locals>.my_class` instead of `my_class`) which makes the check `srcString->isEqual(code->name().cast<PycObject>())` be invalid.
2024-02-26 16:52:31 +02:00
Michael Hansen
787090e0a5 Merge github.com:kako57/pycdc 2024-02-14 21:31:17 -08:00
Nenad Čaklović
5f225caf52 LOAD_ATTR operand changes in 3.12 2024-01-05 21:32:53 +01:00
Nenad Čaklović
830dd13228 COMPARE_OP operand changes in 3.12 2024-01-04 23:49:07 +01:00
Michael Hansen
9c00554efb Add basic support for some Instrumented opcodes which should match their
non-instrumented versions.
2023-11-18 17:59:24 -08:00
Jack
5936412a6f
fix #396 Unsupported Node type: 27 (#397)
* fix #396 Unsupported Node type: 27

* Add test file

* Modify a little comment

* Modify the test pyc file name

* Delete redundant pyc files

* retest

* Add newline at EOF

---------

Co-authored-by: Michael Hansen <zrax0111@gmail.com>
2023-11-11 10:22:49 -08:00
Michael Hansen
9b384ad1fa Add basic bytecode and disassembly support for Python 3.12 2023-11-09 12:17:03 -08:00
TheHellTower
b32f231e33 Add GEN_START support (#392)
* Apparently should be enough ?

* Add `GEN_START` test

* Add `GEN_START` test compiled

* Add `GEN_START` test tokenized

* Smaller test since only POP is needed.

* Smaller test since only POP is needed.

* Smaller test since only POP is needed.

* dos2unix
2023-10-20 13:43:14 -07:00
MrDakik
da859f5a2b Added support for opcodes: (in DESC)
`KW_NAMES`
`POP_JUMP_FORWARD_IF_FALSE`
`POP_JUMP_FORWARD_IF_TRUE`
`LOAD_GLOBAL`
2023-07-18 18:33:03 +03:00
Michael Hansen
753d42d94a Simplify string object printing to reduce code duplication 2023-06-09 09:09:03 -07:00
Michael Hansen
bf3599c87a Move some output stream parameters forward.
This allows us to avoid removing parameter defaults for these functions.
2023-06-05 13:56:25 -07:00
Perceval Wajsbürt
c4c35fc531 Use C++ streams over C style IO 2023-06-02 01:12:59 +02:00
Michael Hansen
409f175827 Reduce code duplication for pre-3.11 binary ops, based on #348 2023-06-01 13:55:17 -07:00
Michael Hansen
90815b47ca Squash merge of #348
(Plus some whitespace cleanup)

commit ca30277743
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Thu Jun 1 14:55:26 2023 -0400

    Test

commit 1764514e1d
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Thu Jun 1 14:46:49 2023 -0400

    Semi-Support

commit 104fa62355
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Thu Jun 1 14:46:30 2023 -0400

    Semi-Support

commit 0809a71c72
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Thu Jun 1 14:46:11 2023 -0400

    Semi-Support

commit bdf1695a57
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Thu Jun 1 00:06:34 2023 -0400

    Updated Test

commit ea69a0b3be
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Thu Jun 1 00:06:23 2023 -0400

    Updated Test

commit ecde214620
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Thu Jun 1 00:06:12 2023 -0400

    Updated Test

commit d5bbbe6224
Author: TheHellTower <thehelltower@tuta.io>
Date:   Wed May 31 23:59:46 2023 -0400

    Feat: @zrax

    Co-authored-by: Michael Hansen <zrax0111@gmail.com>

commit f8d2299d14
Author: TheHellTower <thehelltower@tuta.io>
Date:   Wed May 31 23:59:03 2023 -0400

    Feat: @zrax

    Co-authored-by: Michael Hansen <zrax0111@gmail.com>

commit 3fa8691f4c
Author: TheHellTower <thehelltower@tuta.io>
Date:   Wed May 31 23:58:53 2023 -0400

    Feat: @zrax

    Co-authored-by: Michael Hansen <zrax0111@gmail.com>

commit 6af17f0cff
Author: TheHellTower <thehelltower@tuta.io>
Date:   Wed May 31 23:58:45 2023 -0400

    Feat: @zrax

    Co-authored-by: Michael Hansen <zrax0111@gmail.com>

commit c7c76097fc
Author: TheHellTower <thehelltower@tuta.io>
Date:   Wed May 31 16:55:20 2023 -0400

    Remove `iostream`

commit 2edaec022f
Author: TheHellTower <thehelltower@tuta.io>
Date:   Wed May 31 16:54:48 2023 -0400

    Mistake

commit a36d4c5e62
Author: TheHellTower <thehelltower@tuta.io>
Date:   Wed May 31 16:52:12 2023 -0400

    Remove `iostream`

commit 1c3a5b02c9
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Wed May 31 16:43:30 2023 -0400

    Should be a bit better

commit 11603f1bb2
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Wed May 31 04:07:47 2023 -0400

    Update

commit 689b544fc1
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Wed May 31 04:07:38 2023 -0400

    Update

commit 20bc5b820f
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Wed May 31 04:07:24 2023 -0400

    Update

commit c01e8251fa
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Wed May 31 04:05:28 2023 -0400

    Format problem ?

commit f35c494dd0
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Wed May 31 04:02:03 2023 -0400

    Fix

commit 7f3265e419
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Mon May 29 00:57:01 2023 -0400

    BINARY_OP | Simplified

commit 045a6041f7
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Mon May 29 00:56:53 2023 -0400

    BINARY_OP | Simplified

commit 0bfb76280e
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Mon May 29 00:56:41 2023 -0400

    BINARY_OP | Simplified

commit 31159d69e8
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Mon May 29 00:50:19 2023 -0400

    Fix Matching...

commit e4a978aead
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Mon May 29 00:50:06 2023 -0400

    Fix Matching...

commit d7a1b25b8f
Author: TheHellTower <thehelltower@tuta.io>
Date:   Mon May 29 00:41:43 2023 -0400

    Add `BINARY_OP` support

commit 6527299a67
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Mon May 29 00:40:32 2023 -0400

    Add `BINARY_OP` test tokenized

commit c053cab789
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Mon May 29 00:40:11 2023 -0400

    Add `BINARY_OP` test

commit e32f006f90
Author: TheHellTower <dontbeshybuddy@gmail.com>
Date:   Mon May 29 00:39:53 2023 -0400

    Add `BINARY_OP` test compiled

Closes #333
Closes #348

Co-authored-by: TheHellTower <dontbeshybuddy@gmail.com>
2023-06-01 13:21:23 -07:00
Michael Hansen
0710ea07f1 Fix missing includes, which fixes build on GCC 13+ 2023-05-07 13:28:10 -07:00
Dre Cura
a04b6c0258
Merge branch 'zrax:master' into master 2023-04-24 17:37:18 -04:00