Guppy-PE

A Python Programming Environment


Guppy A fish swimming in Python
Heapy Heap Analysis Toolset
GSL Guppy Specification Language
Documentation
Download
Credits
Contact

This is the home page for Guppy-PE , a programming environment providing object and heap memory sizing, profiling and analysis. It includes a prototypical specification language that can be used to formally specify aspects of Python programs and generate tests and documentation from a common source.

Guppy

Guppy is an umbrella package combining Heapy and GSL with support utilities such as the Glue module that keeps things together.

The name guppy was choosen because I found it in a backward-dictionary as a word ending with py and I thought it was cute enough and that it would not so likely conflict with some other package name. It was to be a general name since all kinds of packages should fit under this top level name.

The name guppy-pe is because there was another project named guppy in Sourceforge when I was about to register guppy. The other guppy was not in Python, so I added -pe which means Programming Environment. The Python package is just guppy.

Heapy

The aim of Heapy is to support debugging and optimization regarding memory related issues in Python programs.

Such issues can make a program use too much memory, making it slow by itself as well as slowing down an entire server, or it may fail to run at all in a limited memory device such as a mobile phone.

The primary motivation for Heapy is that there has been a lack of support for the programmer to get information about the memory usage in Python programs. Heapy is an attempt to improve this situation. A project with a similar intent is PySizer.

The problem situation has a number of aspects, which I think can be characterised, for example, as follows.

As Heapy has evolved, with considerations like this in mind, it currently provides the following features.

Data gathering

Data processing

Presentation

Portability aspects

System aspects

Heapy has been used during development of itself and of the other parts of Guppy. It has been used to tell how much memory the parts of compound objects use, to see what could be worthwhile to optimize. It was used to find a memory leak in the Heapy profile browser, and to find out the cause, which as far as I can tell was due to a bug in a library routine which I have reported.

Example

The following example shows

  1. How to create the session context: hp=hpy()
  2. How to show the reachable objects in the heap: hp.heap()
  3. How to create and show a set of objects: hp.iso(1,[],{})
  4. How to show the shortest paths from the root to x: hp.iso(x).shpaths
>>> from guppy import hpy; hp=hpy()
>>> hp.heap()
Partition of a set of 48477 objects. Total size = 3265516 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0  25773  53  1612820  49   1612820  49 str
     1  11699  24   483960  15   2096780  64 tuple
     2    174   0   241584   7   2338364  72 dict of module
     3   3478   7   222592   7   2560956  78 types.CodeType
     4   3296   7   184576   6   2745532  84 function
     5    401   1   175112   5   2920644  89 dict of class
     6    108   0    81888   3   3002532  92 dict (no owner)
     7    114   0    79632   2   3082164  94 dict of type
     8    117   0    51336   2   3133500  96 type
     9    667   1    24012   1   3157512  97 __builtin__.wrapper_descriptor
<76 more rows. Type e.g. '_.more' to view.>
>>> hp.iso(1,[],{})
Partition of a set of 3 objects. Total size = 176 bytes.
 Index  Count   %     Size   % Cumulative  % Kind (class / dict of class)
     0      1  33      136  77       136  77 dict (no owner)
     1      1  33       28  16       164  93 list
     2      1  33       12   7       176 100 int
>>> x=[]
>>> hp.iso(x).shpaths
 0: Src.i0_modules['__main__'].__dict__['x']
>>> 

GSL

The Guppy Specification Language is an evolving specification language. I started experimenting with this language because I felt the need to have a way to specify documentation and tests from the same source. GSL can describe aspects of a system, especially its API, in a way that can be automatically converted to tests as well as to documents. The documents generated have a formal structure for describing the formal aspects of the specification, complemented with descriptive text from the same source documents. A language that is similar in intent is the Assertion Definition Language .

Specifications written in GSL can be used for:

GSL has been used to generate the documentation for this Guppy distribution. Some part of the specification has been checked against the implementation using the generated tests, which did reveal some discrepancies that were subsequently corrected.

The documents generated by GSL use a formal syntax to describe parameter modes. This document contains examples of such parameter descriptions and explains what they mean.

Documentation

The documentation is included when the source code is downloaded. It is also available here via the following links.

Document example Explains the meaning of some aspects of the documents.
Guppy Specification of guppy , the top level module.
Profile Browser How to use the graphical heap profile browser.
Screenshot Example showing the graphical heap profile browser in action.
GSL The Guppy Specification Language.
heapyc Specification of the heapyc extension module. Note that this is an internal interface and may be subject to change.
sets Specification of the interface to the setsc extension module which contains bitsets and nodesets.

Download

Source code in gzip tar format.

guppy-0.1.2.tar.gz Bugfix version. Pointer comparison bugs and test portability problems were fixed. See changelog in source distribution.
guppy-0.1.1.tar.gz The C source code for the extension modules was changed to be ANSI compatible and I also changed some help text that had become outdated.
guppy-0.1.tar.gz Original version. Extension modules could not be compiled using strict ANSI C compilers.

Credits

Contact

The author, Sverker Nilsson, may be contacted at:
svenil@users.sourceforge.net
I have registered a mailing list for discussions, questions, announcements etc. The list information, subscription form and archives are available at:
http://lists.sourceforge.net/mailman/listinfo/guppy-pe-list
Please let me know of problems, either by mailing me directly, or via the mailing list mentioned above or the SourceForge bug tracking system:
http://sourceforge.net/tracker/?group_id=105577&atid=641821
The Sourceforge project summary page is:
http://sourceforge.net/projects/guppy-pe

Generated by GSL 0.1 on Wed Jan 18 01:39:45 2006