subgrounds package#

Subpackages#

Submodules#

Module contents#

class subgrounds.FieldPath(subgraph: 'Subgraph', root_type: 'TypeRef.T', type_: 'TypeRef.T', path: 'list[Tuple[Optional[dict[str, Any]], TypeMeta.FieldMeta]]')#

Bases: FieldOperatorMixin

class subgrounds.Subgraph(url: 'str', schema: 'SchemaMeta', transforms: 'list[DocumentTransform]' = [<subgrounds.transform.TypeTransform object at 0x7f8be06e3a00>, <subgrounds.transform.TypeTransform object at 0x7f8be06e1000>], is_subgraph: 'bool' = True)#

Bases: object

class subgrounds.Subgrounds(headers: dict[str, typing.Any] = <factory>, global_transforms: list[subgrounds.transform.RequestTransform] = <factory>, subgraphs: dict[str, subgrounds.subgraph.subgraph.Subgraph] = <factory>)#

Bases: object

headers: dict[str, Any]#
global_transforms: list[subgrounds.transform.RequestTransform]#
subgraphs: dict[str, subgrounds.subgraph.subgraph.Subgraph]#
classmethod from_pg_key(key)#
load(url, save_schema=False, cache_dir='schemas/', is_subgraph=True)#
load_subgraph(url, save_schema=False, cache_dir='schemas/')#

Performs introspection on the provided GraphQL API url to get the schema, stores the schema if save_schema is True and returns a generated class representing the subgraph with all its entities.

Parameters:

  • url (str) -- The url of the API

  • save_schema (bool, optional) -- Flag indicating whether or not the schema should be cached to disk. Defaults to False.

  • cache_dir (str, optional) -- If save_schema == True, then subgraph schemas will be stored under cache_dir. Defaults to schemas/

Returns:

A generated class representing the subgraph and its entities

Return type:

Subgraph

load_api(url, save_schema=False, cache_dir='schemas/')#

Performs introspection on the provided GraphQL API url to get the schema, stores the schema if save_schema is True and returns a generated class representing the GraphQL endpoint with all its entities.

Parameters:

  • url (str) -- The url of the API

  • save_schema (bool, optional) -- Flag indicating whether or not the schema should be saved to disk. Defaults to False.

Returns:

A generated class representing the subgraph and its entities

Return type:

Subgraph

mk_request(fpaths)#

Creates a DataRequest object by combining one or more FieldPath objects.

Parameters:

fpaths (FieldPath | list[FieldPath]) -- One or more FieldPath objects that should be included in the request

Returns:

A new DataRequest object

Return type:

DataRequest

execute(req, pagination_strategy=<class 'subgrounds.pagination.strategies.LegacyStrategy'>)#

Executes a DataRequest object, sending the underlying query(ies) to the server and returning a data blob (list of Python dictionaries, one per actual query).

Parameters:

  • req (DataRequest) -- The DataRequest object to be executed

  • pagination_strategy (Optional[Type[PaginationStrategy]], optional) -- A Class implementing the PaginationStrategy Protocol. If None, then automatic pagination is disabled. Defaults to LegacyStrategy.

Returns:

The reponse data

Return type:

list[dict]

execute_iter(req, pagination_strategy=<class 'subgrounds.pagination.strategies.LegacyStrategy'>)#

Same as execute, except that an iterator is returned which will iterate the data pages.

Parameters:

  • req (DataRequest) -- The DataRequest object to be executed

  • pagination_strategy (Optional[Type[PaginationStrategy]], optional) -- A Class implementing the PaginationStrategy Protocol. If None, then automatic pagination is disabled. Defaults to LegacyStrategy.

Returns:

An iterator over the reponse data pages

Return type:

Iterator[dict]

query_json(fpaths, pagination_strategy=<class 'subgrounds.pagination.strategies.LegacyStrategy'>)#

Equivalent to Subgrounds.execute(Subgrounds.mk_request(fpaths), pagination_strategy).

Parameters:

  • fpaths (FieldPath | list[FieldPath]) -- One or more FieldPath objects that should be included in the request.

  • pagination_strategy (Optional[Type[PaginationStrategy]], optional) -- A Class implementing the PaginationStrategy Protocol. If None, then automatic pagination is disabled. Defaults to LegacyStrategy.

Returns:

The reponse data

Return type:

list[dict[str, Any]]

query_json_iter(fpaths, pagination_strategy=<class 'subgrounds.pagination.strategies.LegacyStrategy'>)#

Same as query_json except an iterator over the response data pages is returned.

Parameters:

  • fpaths (FieldPath | list[FieldPath]) -- One or more FieldPath objects that should be included in the request.

  • pagination_strategy (Optional[Type[PaginationStrategy]], optional) -- A Class implementing the PaginationStrategy Protocol. If None, then automatic pagination is disabled. Defaults to LegacyStrategy.

Returns:

The reponse data

Return type:

list[dict[str, Any]]

query_df(fpaths, columns=None, concat=False, pagination_strategy=<class 'subgrounds.pagination.strategies.LegacyStrategy'>)#

Same as Subgrounds.query() but formats the response data into a Pandas DataFrame. If the response data cannot be flattened to a single query (e.g.: when querying multiple list fields that return different entities), then multiple dataframes are returned

fpaths is a list of FieldPath objects that indicate which data must be queried.

columns is an optional argument used to rename the dataframes(s) columns. The length of columns must be the same as the number of columns of all returned dataframes.

concat indicates whether or not the resulting dataframes should be concatenated together. The dataframes must have the same number of columns, as well as the same column names and types (the names can be set using the columns argument).

Parameters:

  • fpaths (FieldPath | list[FieldPath]) -- One or more FieldPath objects that should be included in the request.

  • columns (Optional[list[str]], optional) -- The column labels. Defaults to None.

  • merge (bool, optional) -- Whether or not to merge resulting dataframes.

  • pagination_strategy (Optional[Type[PaginationStrategy]], optional) -- A Class implementing the PaginationStrategy Protocol. If None, then automatic pagination is disabled. Defaults to LegacyStrategy.

Returns:

A DataFrame containing the reponse data

Return type:

pd.DataFrame | list[pd.DataFrame]

Example:

>>> from subgrounds import Subgrounds
>>> sg = Subgrounds()
>>> univ3 = sg.load_subgraph('https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v3')

# Define price SyntheticField
>>> univ3.Swap.price = abs(univ3.Swap.amount0) / abs(univ3.Swap.amount1)

# Query last 10 swaps from the ETH/USDC pool
>>> eth_usdc = univ3.Query.swaps(
...     orderBy=univ3.Swap.timestamp,
...     orderDirection='desc',
...     first=10,
...     where=[
...         univ3.Swap.pool == '0x8ad599c3a0ff1de082011efddc58f1908eb6e6d8'
...     ]
... )
>>> sg.query_df([
...     eth_usdc.timestamp,
...     eth_usdc.price
... ])
  swaps_timestamp  swaps_price
0       1643213811  2618.886394
1       1643213792  2618.814281
2       1643213792  2617.500494
3       1643213763  2615.458495
4       1643213763  2615.876574
5       1643213739  2615.352390
6       1643213678  2615.205713
7       1643213370  2614.115746
8       1643213210  2613.077301
9       1643213196  2610.686563
query_df_iter(fpaths, pagination_strategy=<class 'subgrounds.pagination.strategies.LegacyStrategy'>)#

Same as query_df except an iterator over the response data pages is returned :param fpaths: One or more FieldPath objects that

should be included in the request

Parameters:

  • columns (Optional[list[str]], optional) -- The column labels. Defaults to None.

  • merge (bool, optional) -- Whether or not to merge resulting dataframes.

  • pagination_strategy (Optional[Type[PaginationStrategy]], optional) -- A Class implementing the PaginationStrategy Protocol. If None, then automatic pagination is disabled. Defaults to LegacyStrategy.

Returns:

An iterator over the response data pages, each as a DataFrame

Return type:

Iterator[pd.DataFrame]

query(fpaths, unwrap=True, pagination_strategy=<class 'subgrounds.pagination.strategies.LegacyStrategy'>)#

Executes one or multiple FieldPath objects immediately and return the data (as a tuple if multiple FieldPath objects are provided).

Parameters:

  • fpaths (FieldPath | list[FieldPath]) -- One or more FieldPath object(s) to query.

  • unwrap (bool, optional) -- Flag indicating whether or not, in the case where the returned data is a list of one element, the element itself should be returned instead of the list. Defaults to True.

  • pagination_strategy (Optional[Type[PaginationStrategy]], optional) -- A Class implementing the PaginationStrategy Protocol. If None, then automatic pagination is disabled. Defaults to LegacyStrategy.

Returns:

The FieldPath object(s) data

Return type:

[type]

Example:

>>> from subgrounds import Subgrounds
>>> sg = Subgrounds()
>>> univ3 = sg.load_subgraph('https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v3')

# Define price SyntheticField
>>> univ3.Swap.price = abs(univ3.Swap.amount0) / abs(univ3.Swap.amount1)

# Construct FieldPath to get price of last swap on ETH/USDC pool
>>> eth_usdc_last = univ3.Query.swaps(
...     orderBy=univ3.Swap.timestamp,
...     orderDirection='desc',
...     first=1,
...     where=[
...         univ3.Swap.pool == '0x8ad599c3a0ff1de082011efddc58f1908eb6e6d8'
...     ]
... ).price

# Query last price FieldPath
>>> sg.query(eth_usdc_last)
2628.975030015892
query_iter(fpaths, unwrap=True, pagination_strategy=<class 'subgrounds.pagination.strategies.LegacyStrategy'>)#

Same as query except an iterator over the resonse data pages is returned.

Parameters:

  • fpath (FieldPath | list[FieldPath]) -- One or more FieldPath object(s) to query.

  • unwrap (bool, optional) -- Flag indicating whether or not, in the case where the returned data is a list of one element, the element itself should be returned instead of the list. Defaults to True.

  • pagination_strategy (Optional[Type[PaginationStrategy]], optional) -- A Class implementing the PaginationStrategy Protocol. If None, then automatic pagination is disabled. Defaults to LegacyStrategy.

Returns:

An iterator over the FieldPath object(s)' data pages

Return type:

Iterator[type]

class subgrounds.SyntheticField(f: 'Callable', type_: 'TypeRef.T', deps: 'list[FieldPath | SyntheticField] | FieldPath | SyntheticField', default: 'Any' = None)#

Bases: FieldOperatorMixin

STRING: ClassVar[TypeRef.Named] = Named(name_='String', kind='SCALAR')#
INT: ClassVar[TypeRef.Named] = Named(name_='Int', kind='SCALAR')#
FLOAT: ClassVar[TypeRef.Named] = Named(name_='Float', kind='SCALAR')#
BOOL: ClassVar[TypeRef.Named] = Named(name_='Boolean', kind='SCALAR')#
static default_of_type(type_)#
static constant(value)#

Returns a constant SyntheticField with value value. Useful for injecting additional static data to a schema or merging entities.

Parameters:

value (str | int | float | bool) -- The constant field's value

Returns:

The constant SyntheticField

Return type:

SyntheticField

Example:

>>> from subgrounds.subgrounds import Subgrounds
>>> from subgrounds.subgraph import SyntheticField
>>> sg = Subgrounds()
>>> univ3 = sg.load_subgraph(
...     'https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v3'
... )

# Create constant SyntheticFields
>>> univ3.Mint.tx_type = SyntheticField.constant('MINT')
>>> univ3.Burn.tx_type = SyntheticField.constant('BURN')

# Last 10 mints and burns
>>> mints = univ3.Query.mints(
...     first=10,
...     orderBy=univ3.Mint.timestamp,
...     orderDirection='desc'
... )
>>> burns = univ3.Query.burns(
...     first=10,
...     orderBy=univ3.Burn.timestamp,
...     orderDirection='desc'
... )

# Query mints and burns. Notice that we merge the two entity tables by
# setting `concat=True` and overwriting the column names (columns must
# match the `FieldPaths`)
>>> df = sg.query_df([
...     mints.transaction.id,
...     mints.timestamp,
...     mints.tx_type,
...     mints.origin,
...     mints.amountUSD,
...     burns.transaction.id,
...     burns.timestamp,
...     burns.tx_type,
...     burns.origin,
...     burns.amountUSD,
... ], columns=['tx_hash', 'timestamp', 'tx_type', 'origin', 'amount_USD'], concat=True)

# Sort the DataFrame
>>> df.sort_values(by=['timestamp'], ascending=False)
                                              tx_hash   timestamp tx_type                                      origin    amount_USD
0   0xcbe1bacacc1e64fe613ae5eef2063563bd0057d1e3df...  1656016553    MINT  0x3435e7946d40b1a83c0cf154326fc6b3ca908952  7.879784e+03
1   0xdddaaddf59e5a3abff4feadef83b3ceb023a74424ea7...  1656016284    MINT  0xc747962e7e416e2a582813b1d7ad59eb83077fa6  5.110840e+04
10  0xa7671452c34a3b083083ef81e364489c2c9ee801a3b8...  1656016284    BURN  0xd40db77990bbb30514276b5ac17c3ce5cc9c951f  2.804573e+05
2   0xc132e73975e77c2c2c91fcf332018dfb01aac0ca9471...  1656015853    MINT  0xc747962e7e416e2a582813b1d7ad59eb83077fa6  5.122569e+04
3   0x1444744f4021a2046787c1b7b88cd9ac21f071c65acc...  1656015773    MINT  0xd11aa2e3a000275ed12b87515c9ac0d67b32e7b9  8.897983e+03
4   0x3315617d426fc2b0db5d8dbccd549efaa8f1ae2969ca...  1656015693    MINT  0xb7dd4d134b1794ee848e1af1a62b85d7b2ea9301  0.000000e+00
11  0xcc713daa2dc58cd1f2218c8f438e7fcf04d2e9c7c83d...  1656015278    BURN  0xa7c43e2057d89b6946b8865efc8bee3a4ea7d28d  1.254942e+06
5   0x7bbfae86f0c3c983651bd0671557cd851fc301317c06...  1656015111    MINT  0xac56cee8ccd00d0c1d72ce3415140874552e80f4  3.432075e+04
12  0xea21c3a68a8f0c6a2721a3072e0c8b2edc77f4d2f0d9...  1656014785    BURN  0x0709b103d46d71458a71e5d81230dd688809a53d  2.059106e+04
6   0x3bd369bf45c55cab40c62db81bb3e0684fd85fe2b662...  1656014120    MINT  0x509984bfc0fb24e2d1377cfec224d3afec4c341e  2.517578e+03
13  0x1ea59da77c442479af8fb51501a931260d473e249de7...  1656014018    BURN  0x509984bfc0fb24e2d1377cfec224d3afec4c341e  0.000000e+00
7   0xb9d31ef78b8bf786b422d948dd1fba246710078abff8...  1656013998    MINT  0x22dfec183294d257f80c15d3c9cd47495bdc728c  8.365750e+04
14  0xc5e3ec84a2860e3c3a055ccdced435a67b4aff4dd3be...  1656013946    BURN  0xac56cee8ccd00d0c1d72ce3415140874552e80f4  3.363809e+04
8   0x7c736255d9a4ebf4781069a1b2a929ad89100f1af980...  1656013913    MINT  0x4ce6aea89f059915ae5efbf34a2a8adc544ae09e  4.837287e+04
15  0x95cf56b9eb69aa45048a9b7b3e472df5bc3bfad591cd...  1656013728    BURN  0x4ce6aea89f059915ae5efbf34a2a8adc544ae09e  5.110010e+04
9   0x76dd2bbf43485c224471dd823c2992178f031f27194b...  1656013599    MINT  0x234a644868c419ce0dcdd9fd539762eba47f3759  5.363896e+03
16  0x47e595b02fdcb51ff42a5008e53be7ee3bdf8063b580...  1656013580    BURN  0xaf0fdd39e5d92499b0ed9f68693da99c0ec1e92e  0.000000e+00
17  0xe20ec9702f455d74b3cc1f54fe2f3450604ca5037a72...  1656013455    BURN  0xaf0fdd39e5d92499b0ed9f68693da99c0ec1e92e  0.000000e+00
18  0xac3e95666be3a45fdfbbfa513a114136ea6ecffb9de2...  1656013237    BURN  0x665d2d2444f2384fb3d96aaa0ea3536b92984dce  2.254100e+05
19  0x01c3424a48c36104ea388482723347f15c0bc1bb1a80...  1656013034    BURN  0x0084ee6c8893c01e252198b56ec127443dc27464  0.000000e+00
static datetime_of_timestamp(timestamp)#

Returns a SyntheticField that will transform the FieldPath timestamp into a human-readable ISO8601 string.

Parameters:

timestamp (FieldPath | SyntheticField) -- A FieldPath representing a Unix timestamp field.

Returns:

An ISO8601 datetime string SyntheticField.

Return type:

SyntheticField

Example:

>>> from subgrounds.subgrounds import Subgrounds
>>> from subgrounds.subgraph import SyntheticField
>>> sg = Subgrounds()
>>> univ3 = sg.load_subgraph(
...     'https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v3'
... )

# Create datetime SyntheticField
>>> univ3.Swap.datetime = SyntheticField.datetime_of_timestamp(univ3.Swap.timestamp)

# Query 100 swaps
>>> sg.query_df([
...     univ3.Query.swaps.timestamp,
...     univ3.Query.swaps.datetime,
... ])
    swaps_timestamp       swaps_datetime
0        1625105710  2021-06-30 22:15:10
1        1629253724  2021-08-17 22:28:44
2        1647333277  2022-03-15 04:34:37
3        1630801974  2021-09-04 20:32:54
4        1653240241  2022-05-22 13:24:01
..              ...                  ...
95       1646128326  2022-03-01 04:52:06
96       1646128326  2022-03-01 04:52:06
97       1626416555  2021-07-16 02:22:35
98       1626416555  2021-07-16 02:22:35
99       1625837291  2021-07-09 09:28:11
static map(dict, type_, fpath, default=None)#

Returns a SyntheticField that will map the values of fpath using the key value pairs in dict. If a value is not in the dictionary, then default will be used instead.

Parameters:

  • dict (dict[Any, Any]) -- The dictionary containing the key value pairs used to map fpath's values

  • type (TypeRef.T) -- The type of the resulting field

  • fpath (FieldPath | SyntheticField) -- The FieldPath whose values will be mapped using the dictionary

  • default (Optional[Any]) -- Default value used when a value is not in the provided dictionary

Returns:

A map SyntheticField

Return type:

SyntheticField

Example:

>>> from subgrounds.subgrounds import Subgrounds
>>> from subgrounds.subgraph import SyntheticField
>>> sg = Subgrounds()
>>> univ3 = sg.load_subgraph(
...     'https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v3'
... )

# Hand-crafted mapping of pool addresses to symbol
>>> pooladdr_symbol_map = {
...     '0x5777d92f208679db4b9778590fa3cab3ac9e2168': 'DAI/USDC-001',
...     '0x6c6bc977e13df9b0de53b251522280bb72383700': 'DAI/USDC-005',
...     '0x8ad599c3a0ff1de082011efddc58f1908eb6e6d8': 'USDC/ETH-030',
...     '0x88e6a0c2ddd26feeb64f039a2c41296fcb3f5640': 'USDC/ETH-005',
... }

# Create map SyntheticField using our dictionary with 'UNKNOWN' as the
# default value
>>> univ3.Pool.symbol = SyntheticField.map(
...     pooladdr_symbol_map,
...     SyntheticField.STRING,
...     univ3.Pool.id,
...     'UNKNOWN'
... )

# Query top 10 pools by TVL
>>> pools = univ3.Query.pools(
...     first=10,
...     orderBy=univ3.Pool.totalValueLockedUSD,
...     orderDirection='desc'
... )
>>> sg.query_df([
...     pools.id,
...     pools.symbol
... ])
                                     pools_id  pools_symbol
0  0xa850478adaace4c08fc61de44d8cf3b64f359bec       UNKNOWN
1  0x5777d92f208679db4b9778590fa3cab3ac9e2168  DAI/USDC-001
2  0x6c6bc977e13df9b0de53b251522280bb72383700  DAI/USDC-005
3  0x8ad599c3a0ff1de082011efddc58f1908eb6e6d8  USDC/ETH-030
4  0x88e6a0c2ddd26feeb64f039a2c41296fcb3f5640  USDC/ETH-005
5  0x3416cf6c708da44db2624d63ea0aaef7113527c6       UNKNOWN
6  0xcbcdf9626bc03e24f779434178a73a0b4bad62ed       UNKNOWN
7  0xc63b0708e2f7e69cb8a1df0e1389a98c35a76d52       UNKNOWN
8  0x4585fe77225b41b697c938b018e2ac67ac5a20c0       UNKNOWN
9  0x4e68ccd3e89f51c3074ca5072bbac773960dfa36       UNKNOWN