This doc is incomplete, check back soon!

Transformations in subgrounds operate via layers, where one is applied after another. There are two types of transforms:


This transform operates on a DataRequest and DataResponse which represent the total query being made via one operation of _execute.


This transform operates on a Document and DocumentResponse which (ignoring pagination) represent a part of the total query that will be made. Each Document cooresponds to a single subgraph server.

Each layer contains two important functions:

transform_request() / transform_document()

This transforms requests on-the-way towards pagination.

transform_response() / transform_response()

This transforms responses backwards from pagination.

When fully assembled, the layers act like an onion.

graph LR A[`execute`] --> |DataRequest| B[RequestTransform] B --> |DataRequest.documents| C(["DocumentTransform(s)"]) C --> |Document| D[[`paginate`]] D --> |DocumentResponse| C C --> |"DataResponse.responses"| B B --> |"DataResponse"| A

A top-level showcase on how a request turns into a response#

The Pipeline#

Subgrounds supports any number of transformations for every query produced — something that ends up being quite complex to fufill! Generally, here are the steps we take when executing the transforms:

  1. Convert all DocumentTransforms to DocumentRequestTransform.

    • This greatly simplifies the transform pipeline.

  2. Convert all transformation layers into a generator sandwich.

    • These "sandwiches" essentially create an generator that hold the context of DataRequest making it easier for us to transform both DataRequest and DataResponse.

    • Essentially, we store these in a stack allowing us to easily iterate through them.

  3. Waterfall the first DataRequest through all of the generators in the stack.

    • The first request gets transformed through each generators within the stack.

  4. Forward the final transformed DataRequest to pagination (and eventually execution).

  5. Receive the raw DataResponse from pagination / execution.

  6. In reverse, waterfall this DataResponse up the generator stack.

  7. Return this DataResponse as the result of the transformation pipeline.

Visual Guide#

This is the general flow of how a request gets tranformed through the pipeline.