Discussion:
[issue11822] Improve disassembly to show embedded code objects
Raymond Hettinger
2011-04-10 20:25:52 UTC
Permalink
dis('[x**2 for x in range(3)]')
1 0 LOAD_CONST 0 (<code object <listcomp> at 0x1005d1e88, file "<dis>", line 1>)
3 MAKE_FUNCTION 0
6 LOAD_NAME 0 (range)
9 LOAD_CONST 1 (3)
12 CALL_FUNCTION 1
15 GET_ITER
16 CALL_FUNCTION 1
19 RETURN_VALUE

I propose that dis() build-up a queue undisplayed code objects and then disassemble each of those after the main disassembly is done (effectively making it recursive and displaying code objects in the order that they are first seen in the disassembly). For example, the output shown above would be followed by a disassembly of its internal code object:

<code object <listcomp> at 0x1005d1e88, file "<dis>", line 1>:
1 0 BUILD_LIST 0
3 LOAD_FAST 0 (.0)
6 FOR_ITER 16 (to 25)
9 STORE_FAST 1 (x)
12 LOAD_FAST 1 (x)
15 LOAD_CONST 0 (2)
18 BINARY_POWER
19 LIST_APPEND 2
22 JUMP_ABSOLUTE 6
25 RETURN_VALUE
----------
components: Library (Lib)
messages: 133478
nosy: rhettinger
priority: normal
severity: normal
status: open
title: Improve disassembly to show embedded code objects
type: feature request
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Alexander Belopolsky
2011-04-11 14:29:18 UTC
Permalink
Changes by Alexander Belopolsky <belopolsky at users.sourceforge.net>:


----------
nosy: +belopolsky

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Alexander Belopolsky
2011-04-11 14:42:04 UTC
Permalink
Alexander Belopolsky <belopolsky at users.sourceforge.net> added the comment:

Would you like to display lambdas as well?
dis('lambda x: x**2')
1 0 LOAD_CONST 0 (<code object <lambda> at 0x1005c9ad0, file "<dis>", line 1>)
3 MAKE_FUNCTION 0
6 RETURN_VALUE

<code object <lambda> at 0x1005cb140, file "<dis>", line 1>:
1 0 LOAD_FAST 0 (x)
3 LOAD_CONST 1 (2)
6 BINARY_POWER
7 RETURN_VALUE


I like the idea, but would rather see code objects expanded in-line, possibly indented rather than at the end.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Antoine Pitrou
2011-04-11 21:08:07 UTC
Permalink
Antoine Pitrou <pitrou at free.fr> added the comment:

I think it should be enabled with an optional argument. Otherwise in some cases you'll get lots of additional output while you're only interested in the top-level code.

----------
nosy: +pitrou

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Raymond Hettinger
2011-04-11 21:13:10 UTC
Permalink
Raymond Hettinger <raymond.hettinger at gmail.com> added the comment:

If you disassemble a function, you typically want to see all the code in that function. This isn't like pdb where you're choosing to step over or into another function outside the one being viewed.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Antoine Pitrou
2011-04-11 21:21:03 UTC
Permalink
Post by Raymond Hettinger
If you disassemble a function, you typically want to see all the code
in that function.
That depends on the function. If you do event-driven programming (say,
Twisted deferreds with addCallback()), you don't necessarily want to see
the disassembly of the callbacks that are passed to the various
framework functions. Also, if you do so recursively, it might become
*very* unwieldy.

So I don't think there's anything "typical" here. It depends on what you
intend to focus on.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Alexander Belopolsky
2011-04-11 21:48:24 UTC
Permalink
Alexander Belopolsky <belopolsky at users.sourceforge.net> added the comment:

On Mon, Apr 11, 2011 at 5:21 PM, Antoine Pitrou <report at bugs.python.org> wrote:
..
Raymond>> If you disassemble a function, you typically want to see all the code
Raymond>> [defined] in that function.

+1 (with clarification in [])

If the function calls a function defined elsewhere, I don't want to
see the called function disassembly when I disassemble the caller. In
this case it is very easy to disassemble interesting functions with
separate dis() calls. In the case like the following, however:

def f():
def g(x):
return x**2
dis(f)
2 0 LOAD_CONST 1 (<code object g at
0x10055ce88, file "x.py", line 2>)
3 MAKE_FUNCTION 0
6 STORE_FAST 0 (g)
...

when I see '<code object g at 0x10055ce88, ..>', I have to do
something unwieldy such as

3 0 LOAD_FAST 0 (x)
3 LOAD_CONST 1 (2)
6 BINARY_POWER
7 RETURN_VALUE
Post by Antoine Pitrou
That depends on the function. If you do event-driven programming (say,
Twisted deferreds with addCallback()), you don't necessarily want to see
the disassembly of the callbacks that are passed to the various
framework functions. Also, if you do so recursively, it might become
*very* unwieldy.
Can you provide some examples of this? Nested functions are typically
short and even if they are long, the size disassembly would be
proportional to the line count of the function being disassembled,
which is expected.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Raymond Hettinger
2011-04-13 02:28:27 UTC
Permalink
Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
assignee: -> rhettinger

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Nick Coghlan
2011-06-01 05:10:24 UTC
Permalink
Nick Coghlan <ncoghlan at gmail.com> added the comment:

Note that Yaniv Aknin (author of the Python's Innards series of blog posts) has a recursive dis variant that may be useful for inspiration:

https://bitbucket.org/yaniv_aknin/pynards/src/c4b61c7a1798/common/blog.py

As shown there, this recursive behaviour can also be useful for code_info/show_code.

----------
nosy: +ncoghlan

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Torsten Landschoff
2011-12-10 23:34:20 UTC
Permalink
Torsten Landschoff <t.landschoff at gmx.net> added the comment:

I offer the attached patch as a starting point to fulfill this feature request.

The patch changes the output to insert the disassembly of local code objects on the referencing line.

As that made the output unreadable to me, I added indentation for the nested code (by 4 spaces, hoping that nobody will nest code 10 levels deep :-)
from dis import dis
dis('[x**2 for x in range(3)]')
1 0 LOAD_CONST 0 (<code object <listcomp> at 0x7f24a67dde40, file "<dis>", line 1>)
1 0 BUILD_LIST 0
3 LOAD_FAST 0 (.0)
6 FOR_ITER 16 (to 25)
9 STORE_FAST 1 (x)
12 LOAD_FAST 1 (x)
15 LOAD_CONST 0 (2)
18 BINARY_POWER
19 LIST_APPEND 2
22 JUMP_ABSOLUTE 6
25 RETURN_VALUE
3 LOAD_CONST 1 ('<listcomp>')
6 MAKE_FUNCTION 0
9 LOAD_NAME 0 (range)
12 LOAD_CONST 2 (3)
15 CALL_FUNCTION 1
18 GET_ITER
19 CALL_FUNCTION 1
22 RETURN_VALUE

----------
keywords: +patch
nosy: +torsten
Added file: http://bugs.python.org/file23908/issue11822.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Raymond Hettinger
2011-12-10 23:44:24 UTC
Permalink
Raymond Hettinger <raymond.hettinger at gmail.com> added the comment:

Thank you :-)

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Raymond Hettinger
2013-07-12 07:35:54 UTC
Permalink
Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
stage: -> needs patch
versions: +Python 3.4 -Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Josh Lee
2014-08-12 16:20:50 UTC
Permalink
Changes by Josh Lee <jleedev at gmail.com>:


----------
nosy: +jleedev

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Raymond Hettinger
2014-08-12 19:49:09 UTC
Permalink
Changes by Raymond Hettinger <raymond.hettinger at gmail.com>:


----------
stage: needs patch -> patch review
versions: +Python 3.5 -Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
STINNER Victor
2014-08-16 12:42:16 UTC
Permalink
Changes by STINNER Victor <victor.stinner at gmail.com>:


----------
nosy: +haypo

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Nick Coghlan
2014-09-07 01:01:46 UTC
Permalink
Nick Coghlan added the comment:

Sorry for the long delay in doing anything with this patch. Unfortunately, trunk has moved on quite a bit since this patch was submitted, and it's no longer directly applicable.

However, the basic principle is sound, so this is a new patch that aligns with the changes made in 3.4 to provide an iterator based bytecode introspection API. It also changes the indenting to be based on the structure of the bytecode disassembly - nested lines start aligned with the opcode *name* on the preceding line. This will get unreadable with more than two or three levels of nesting, but at that point, hard to read disassembly for the top level function is the least of your worries. (A potentially useful option may to be add a flag to turn off the implicit recursion, easily restoring the old single level behaviour. I'd like the recursive version to be the default though, since it's far more useful given that Python 3 comprehensions all involve a nested code object)

A descriptive header makes the new output more self-explanatory. Note that I did try repeating the code object repr from the LOAD_CONST opcode in the new header - it was pretty unreadable, and redundant given the preceding line of disassembly.

Two examples, one showing Torsten's list comprehension from above, and another showing that the nested line numbers work properly.

This can't be applied as is - it's still missing tests, docs, and fixes to disassembly output tests that assume the old behaviour.
dis.dis('[x**2 for x in range(3)]')
1 0 LOAD_CONST 0 (<code object <listcomp> at 0x7f459ec4a0c0, file "<dis>", line 1>)
Disassembly for nested code object
1 0 BUILD_LIST 0
3 LOAD_FAST 0 (.0)
6 FOR_ITER 16 (to 25)
9 STORE_FAST 1 (x)
12 LOAD_FAST 1 (x)
15 LOAD_CONST 0 (2)
18 BINARY_POWER
19 LIST_APPEND 2
22 JUMP_ABSOLUTE 6
25 RETURN_VALUE
3 LOAD_CONST 1 ('<listcomp>')
6 MAKE_FUNCTION 0
9 LOAD_NAME 0 (range)
12 LOAD_CONST 2 (3)
15 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
18 GET_ITER
19 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
22 RETURN_VALUE
... print("Hello")
... def g():
... for x in range(10):
... yield x
... return g
...
dis.dis(f)
2 0 LOAD_GLOBAL 0 (print)
3 LOAD_CONST 1 ('Hello')
6 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
9 POP_TOP

3 10 LOAD_CONST 2 (<code object g at 0x7f459ec4a540, file "<stdin>", line 3>)
Disassembly for nested code object
4 0 SETUP_LOOP 25 (to 28)
3 LOAD_GLOBAL 0 (range)
6 LOAD_CONST 1 (10)
9 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
12 GET_ITER
13 FOR_ITER 11 (to 27)
16 STORE_FAST 0 (x)

5 19 LOAD_FAST 0 (x)
22 YIELD_VALUE
23 POP_TOP
24 JUMP_ABSOLUTE 13
27 POP_BLOCK
28 LOAD_CONST 0 (None)
31 RETURN_VALUE
13 LOAD_CONST 3 ('f.<locals>.g')
16 MAKE_FUNCTION 0
19 STORE_FAST 0 (g)

6 22 LOAD_FAST 0 (g)
25 RETURN_VALUE

----------
Added file: http://bugs.python.org/file36565/issue11822_nested_disassembly.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Nick Coghlan
2014-09-07 05:30:36 UTC
Permalink
Nick Coghlan added the comment:

I didn't want to add a second argument to turn off the new behaviour, so I changed it such that passing a value < 0 for "nested" turns off the new feature entirely. Levels >= 0 enable it, defining which level to start with. The default level is "0" so there's no implied prefix, and nested code objects are displayed by default. This picks up at least comprehensions, lambda expressions and nested functions. I haven't checked how it handles nested classes yet.

I used this feature to get the old tests passing again by turning off the recursion feature. New tests for the new behaviour are still needed.

I also tweaked the header to show the *name* of the code object. The full repr is to noisy, but the generic message was hard to read when there were multiple nested code objects.

----------
Added file: http://bugs.python.org/file36566/issue11822_nested_disassembly_with_off_switch.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Berker Peksag
2015-01-26 01:28:24 UTC
Permalink
Changes by Berker Peksag <***@gmail.com>:


----------
nosy: +berker.peksag

_______________________________________
Python tracker <***@bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Stéphane Wirtel
2015-11-06 21:16:34 UTC
Permalink
Stéphane Wirtel added the comment:

Hi all,

For this feature, I have an other output:

***@sg1 /tmp> python3 dump_bytecode.py
<module>
--------
3 0 LOAD_BUILD_CLASS
1 LOAD_CONST 0 (<code object User at 0x10b830270, file "<show>", line 3>)
4 LOAD_CONST 1 ('User')
7 MAKE_FUNCTION 0
10 LOAD_CONST 1 ('User')
13 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
16 STORE_NAME 0 (User)

8 19 LOAD_NAME 0 (User)
22 LOAD_CONST 2 ('user')
25 LOAD_CONST 3 ('password')
28 CALL_FUNCTION 2 (2 positional, 0 keyword pair)
31 STORE_NAME 1 (user)
34 LOAD_CONST 4 (None)
37 RETURN_VALUE

<module>.User
-------------
3 0 LOAD_NAME 0 (__name__)
3 STORE_NAME 1 (__module__)
6 LOAD_CONST 0 ('User')
9 STORE_NAME 2 (__qualname__)

4 12 LOAD_CONST 1 (<code object __init__ at 0x10b824270, file "<show>", line 4>)
15 LOAD_CONST 2 ('User.__init__')
18 MAKE_FUNCTION 0
21 STORE_NAME 3 (__init__)
24 LOAD_CONST 3 (None)
27 RETURN_VALUE

<module>.User.__init__
----------------------
5 0 LOAD_FAST 1 (email)
3 LOAD_FAST 0 (self)
6 STORE_ATTR 0 (email)

6 9 LOAD_FAST 2 (password)
12 LOAD_FAST 0 (self)
15 STORE_ATTR 1 (password)
18 LOAD_CONST 0 (None)
21 RETURN_VALUE

----------
nosy: +matrixise
versions: +Python 3.6 -Python 3.5

_______________________________________
Python tracker <***@bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________
Raymond Hettinger
2015-11-10 07:27:13 UTC
Permalink
Changes by Raymond Hettinger <***@gmail.com>:


----------
assignee: rhettinger ->

_______________________________________
Python tracker <***@bugs.python.org>
<http://bugs.python.org/issue11822>
_______________________________________

Loading...