General Abstract Meta Programming for Python
Médéric Hurier 75533a04ed updated project 2 months ago
covers updated project 2 months ago
devs updated project 2 months ago
editors updated project 2 months ago
formats updated project 2 months ago
gampy updated project 2 months ago
hooks initial commit 4 months ago
lints updated project 2 months ago
packages updated project 2 months ago
pytags updated project 2 months ago
shells updated project 2 months ago
tests updated project 2 months ago
types updated project 2 months ago
vulns updated project 2 months ago
.coveragerc updated structure 3 months ago
.ctags updated project 2 months ago
.gitignore initial commit 4 months ago
.python-version updated project 2 months ago
LICENSE.txt Update 'LICENSE.txt' 3 months ago
Makefile updated project 2 months ago
README.md updated readme 4 months ago
black.ini updated structure 3 months ago
ipython.py updated structure 3 months ago
jupyterlab.py updated structure 3 months ago
mypy.ini updated structure 3 months ago
pylintrc updated structure 3 months ago
pytest.ini updated structure 3 months ago
requirements.txt initial commit 4 months ago
setup.py updated structure 3 months ago
tags updated project 2 months ago

README.md

Gampy

Gampy provides experimental constructs to improve your Python programs.

Pipeline

While functional programming is a great paradigm to create data pipelines, some operations remains hard to express:

  • wrap every function of a data pipeline with an external function (e.g. logging, safe exception handling …)
  • concatenate and transform data pipelines with algebra operators (e.g. convert and compare a pipeline …)
  • apply the compose and partial operators on a data pipeline to create a single callable function

Gampy Pipeline is a data structure created to address these problems.

Here is an example of a simple pipeline:

from gampy.structures import Pipeline

pipeline = Pipeline([
    (map, [lambda x: x + 1], {}),
    (filter, [lambda x: x % 2 == 0]),
    (list,)
])

Each step of the pipeline is represented as a 3-tuple: (function, arguments, keyword arguments). While function is mandatory, arguments and keyword argument will be replaced by list() and dict() respectively if they are missing. This structure allows the creation of unevaluated expression, that can be further transformed prior to their execution.

The most interesting operations over a pipeline are () (call) and @ (matmul).

Call converts the pipeline into a single function. This process is divided in two steps:

  • functools.partial is applied on each step arguments to create a single function per step
  • functions are composed two by two with functools.reduce to create a single function per pipeline
>>> f = pipeline()
>>> f(range(10))
30

Matmul applies an advice to each function of the pipeline. This allows the expression of cross concern aspects.

In the snippet below, any exception raised by a pipeline function will return None.

from gampy.advices import exceptional

safepipe = pipeline @ exceptional(None)

An advice is similar to a parametrized decorator, which create a function that takes a function and replaced it by a new function. The purpose is to extend the behavior of the original function.

def exceptional(x: Any = None, on: Type[Exception] = Exception) -> Advice:
    """Return x when f raises an exception."""

    def advice(f):
        @wraps(f)
        def wrapped(*args, **kwargs):
            try:
                return f(*args, **kwargs)
            except on:
                return x

        return wrapped

    return advice