Wednesday, May 26, 2010

Google Summer of Code 2010 - fact ctypes

So, this is my third time with GSoC program. Last year I was chosen by Umit Project and PyPy (through Python Software Foundation) and I had to choose where to go. I decided to go with Umit Project and finish my project which was born during GSoC 2007 edition. The main reason was that I needed it to my Master Thesis.

So, this year I gave a try with PyPy again and fortunately I was chosen again! Thanks guys!
And here is my project proposal for this year...


Project Name

PyPy - fast ctypes

Introduction

The PyPy project aims at producing a flexible and fast Python implementation. The guiding idea is to translate a Python-level description of the Python language itself to lower level languages.

PyPy project's main part is not Python interpreter implementation, but its configurable translator. It provides a good way to avoid writing interpreter for every language and platform. It provides tool-chain for translating front-end dynamic language into one of the back-ends (e.g. C, CLI).

Since 1.2 release[1], PyPy has provided Just-In-Time compiler (actually it's a 5th generation, also called pyjitpl5)[2]. Currently the speed up for Python applications is massive[3], however it doesn't mean that the PyPy has reached the limits. One of the way to give a boost is to speed up ctypes[4]. And this is a main goal of the proposal.

Project Details

Current implementation of pypy/lib/_ctypes uses libffi[5] library through _rawffi module. The idea is to give a chance to JIT in supporting ctypes. This can be reached by rewriting ctypes in RPython and avoiding to use libffi.

The rewriting process ain't that simple due to restrictions in RPython language[6]. One of the obvious issues which has to be solved in the project is dynamic amount of arguments in calling functions. Python allows to use dynamic amount of arguments, so does JIT. Unfortunately, there is RPython in the middle which is enough restricted and static to forbid this. The issue to solve is how to transport dynamic amount of arguments through RPython.

In fact that the goal is to speed up ctypes especially by using JIT, special attention is needed during the porting process. From the beginning, RPython's ctypes should be written with the JIT in mind. The JIT should be able to trace ctypes since first incarnation of the new ctypes. This fact will impact on some implementation details.


References

[1] http://codespeak.net/pypy/trunk/pypy/doc/release-1.2.0.html
[2] http://codespeak.net/pypy/trunk/pypy/doc/jit/pyjitpl5.html
[3] http://speed.pypy.org
[4] http://python.net/crew/theller/ctypes/
[5] http://sourceware.org/libffi/
[6] http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html#restricted-python

The Blog

Seems like I was more or less forced to start a technical blog. So, here I am :)

Mostly, I will use this blog for status updates of my GSoC project. But we will see what future will bring :-)

Enjoy hacking!