Strategies#

Subgrounds pagination strategies

This module implements functions and data structures used to implement the two PaginationStrategies that Subgrounds offers.

For both strategies, pagination is done in four steps:

  1. Generate one or many PaginationNode objects per query document. These are tree-like data structures that extract all information about pagination fields (i.e.: fields that return lists of entities) while maintaining the nestedness relationship between each pagination field (i.e.: which is nested in which).

  2. The query document is normalized such that every pagination field in the query has:

    1. An ordering (i.e.: orderBy and orderDirection are specified)

    2. A first argument set to the firstN variable

    3. A skip argument set to the skipN variable

    4. A where filter with the filter name derived from the ordering and the value being a variable named lastOrderingValueN

    In other words, the query will be transformed in a form which allows Subgrounds to paginate automatically by simply setting the set of pagination variables (i.e.: firstN, skipN and lastOrderingValueN) to different values. Each field that requires pagination (i.e.: each field that yields a list) will have its own set of variables, hence the N post-fix.

    Example: The initial query

    query {
  items(
    orderBy: timestamp,
    orderDirection: desc,
    first: 10000
  ) {
    foo
  }
}

    will be transformed to

    query($first0: Int, $skip0: Int, $lastOrderingValue0: BigInt) {
  items(
    orderBy: timestamp,
    orderDirection: desc,
    first: $first0,
    skip: $skip0,
    where: {
      timestamp_lt: $lastOrderingValue0
    }
  ) {
    foo
  }
}

  3. For each data page, generate the values for the pagination variables (i.e.: firstN, skipN and `lastOrderingValueN`)

  4. If some variables are undefined (i.e.: they are present in the query document, but not given a value as part of step 3), then the document is pruned and all selections (and sub-selections) containing undefined variables are removed.

Depending on the strategy, the variable values computed at step 3 will change.

exception subgrounds.pagination.strategies.StopPagination#

Raise within PaginationStategy when pagination is completed, usually within step().

Causes paginate() to be stopped.

exception subgrounds.pagination.strategies.SkipPagination#

Raise within PaginationStategy should be skipped, usually within __init___().

Causes paginate() to be skipped.

class subgrounds.pagination.strategies.PaginationStrategy(*args, **kwargs)#
step(page_data=None)#

Returns the new query document and its variables which will be executed to get the next page of data.

If this is the first query made as part of the pagination strategy, then

page_data will be None.

If pagination should be interupted (e.g.: if enough entities have been queried), then this method should raise a StopPagination exception.

Parameters:

page_data (Optional[dict[str, Any]]) -- The previous query's response data. If this is the first query (i.e.: the first page of data), then it will be None. Defaults to None.

Returns:

A tuple (doc, vars) where doc is the query document that will be

executed to fetch the next page of data and vars are the variables for that document.

Return type:

tuple

class subgrounds.pagination.strategies.LegacyStrategyArgGenerator(pagination_nodes: 'list[PaginationNode]')#
class Cursor(page_node: 'PaginationNode')#
update(data)#

Moves self cursor forward according to previous response data data

Parameters:

data (dict[str, Any]) -- Previous response data

Raises:

StopIteration -- _description_

step(data)#

Updates either self cursor or inner state machine depending on whether the inner state machine has reached its limit

Parameters:

data (dict[str, Any]) -- _description_

args()#
Returns the pagination arguments for the current state of the

state machine

Returns:

_description_

Return type:

dict[str, Any]

reset()#

Reset state machine

class subgrounds.pagination.strategies.ShallowStrategyArgGenerator(pagination_nodes: 'list[PaginationNode]')#
class Cursor(page_node, inner, inner_idx=0, filter_value=None, queried_entities=0, page_count=0)#

Class used to generate the pagination variables for a given tree of PaginationNode objects.

page_node#

The PaginationNode object which this cursor is iterating through

Type:

subgrounds.pagination.preprocess.PaginationNode

inner#

The cursors for nested PaginationNodes, if any

Type:

list[subgrounds.pagination.strategies.ShallowStrategyArgGenerator.Cursor]

inner_idx#

The index of the inner PaginationNode through which this cursor iterating

Type:

int

filter_value#

The previous page's index value used to query the next data page Depends on page_node.filter_field, e.g.: if page_node.filter_field is timestamp_gt, then filter_value will be the highest timestamp the entities returned in the previous data page.

Type:

Any

queried_entities#

Counter for the total number of queried entities

Type:

int

stop#

Flag indicating whether or not to stop the cursor

page_count#

Counter keeping track of the total number data pages queried

Type:

int

keys#

Set of keys of all queried entities to avoid duplicates

# TODO: Add GreedyStrategy doc

class subgrounds.pagination.strategies.SkipStrategy(schema, document)#
This strategy always raises a SkipPagination

when constructed or if somehow step is called.

This is the default if None is passed to the strategy (default for non-subgraphs).