HTTP#

Making requests#

Hint

The finished example can be found here

Note

We will use await at the top level here as its easier to explain. For your own code, please use a async function.

If you want a example of how it can be done in a async function, see the full example.

Setting up#

from os import environ

from discord_typings import ChannelData
from nextcore.http import BotAuthentication, HTTPClient, Route

# Constants
AUTHENTICATION = BotAuthentication(environ["TOKEN"])
CHANNEL_ID = environ["CHANNEL_ID"]

# HTTP Client
http_client = HTTPClient()
await http_client.setup()

Note

You will need to set environment variables on your system for this to work.

Bash

export TOKEN="..."
export CHANNEL_ID="..."

PowerShell

$env:TOKEN = "..."
$env:CHANNEL_ID = "..."

Creating a `Route`#

First you need to find what route you are going to implement. A list can be found on https://discord.dev

For this example we are going to use Get Channel

For the first parameter, this will be the HTTP method.

route = Route("GET", ...)

The second parameter is the “route”. This is the path of the request, without any parameters included. To get this, you take the path (/channels/{channel.id}) and replace . with _ It will look something like this.

route = Route("GET", "/channels/{channel_id}", ...)

Warning

This should not be a f-string!

The kwargs will be parameters to the path.

route = Route("GET", "/channels/{channel_id}", channel_id=CHANNEL_ID)

Doing the request#

To do a request, you will need to use HTTPClient.request().

response = await http_client.request(route,
     rate_limit_key=AUTHENTICATION.rate_limit_key,
     headers=AUTHENTICATION.headers
)

This will return a aiohttp.ClientResponse for you to process.

Getting the response data#

We can use aiohttp.ClientResponse.json() to get the JSON response data.

channel = await response.json()

And finally, get the channel name

print(channel.get("name"))  # DM channels do not have names

Cleaning up#

When you are finished with all requests, you can close the HTTP client gracefully.

await http_client.close()

HTTP reference#

class nextcore.http.HTTPClient(*, trust_local_time=True, timeout=60, max_rate_limit_retries=10)#

The HTTP client to interface with the Discord API.

Example usage

http_client = HTTPClient()
await http_client.setup()

# This can be found on https://discord.dev/topics/gateway#get-gateway
route = Route("GET", "/gateway")

# No authentication is used, so rate_limit_key None here is equivilent of your IP.
response = await http_client.request(route, rate_limit_key=None)
gateway: GetGatewayData = await response.json()

print(gateway["url"])

await http_client.close()

Parameters:

trust_local_time (bool) – Whether to trust local time. If this is not set HTTP rate limiting will be a bit slower but may be a bit more accurate on systems where the local time is off.
timeout (float) – The default request timeout in seconds.
max_rate_limit_retries (int) – How many times to attempt to retry a request after rate limiting failed.

trust_local_time#

If this is enabled, the rate limiter will use the local time instead of the discord provided time. This may improve your bot’s speed slightly.

Warning

If your time is not correct, and this is set to True, this may result in more rate limits being hit.

Ubuntu

You can check if your clock is synchronized by running the following command:

timedatectl

If it is synchronized, it will show “System clock synchronized: yes” and “NTP service: running”

If the system clock is not synchronized but the ntp service is running you will have to wait a few minutes for it to sync.

To enable the ntp service run the following command:

sudo timedatectl set-ntp on

This will automatically sync the system clock every once in a while.

Arch

You can check if your clock is synchronized by running the following command:

timedatectl

If it is synchronized, it will show “System clock synchronized: yes” and “NTP service: running”

If the system clock is not synchronized but the ntp service is running you will have to wait a few minutes for it to sync.

To enable the ntp service run the following command:

sudo timedatectl set-ntp on

This will automatically sync the system clock every once in a while.

Windows

This can be turned on by going to Settings -> Time & language -> Date & time and turning on Set time automatically.

timeout#: The default request timeout in seconds.

default_headers#: The default headers to pass to every request.

max_retries#: How many times to attempt to retry a request after rate limiting failed.

Note

This does not retry server errors.

rate_limit_storages#

Classes to store rate limit information.

The key here is the rate_limit_key (often a user ID).

dispatcher#: Events from the HTTPClient. See the events

async close()#

Clean up internal state

Return type:: None

async connect_to_gateway(*, version=UndefinedType.UNDEFINED, encoding=UndefinedType.UNDEFINED, compress=UndefinedType.UNDEFINED)#

Connects to the gateway

Example usage:

ws = await http_client.connect_to_gateway()

Parameters:

version (Literal[6, 7, 8, 9, 10] | UndefinedType) –
The major API version to use

Hint

It is a good idea to pin this to make sure something doesn’t unexpectedly change
encoding (Literal['json', 'etf'] | UndefinedType) – Whether to use json or etf for payloads
compress (Literal['zlib-stream'] | UndefinedType) – Payload compression from data sent from Discord.

Raises:

RuntimeError – HTTPClient.setup() was not called yet.
RuntimeError – HTTPClient was closed.

Returns:

The gateway websocket

Return type:

aiohttp.ClientWebSocketResponse

async request(route, rate_limit_key, *, headers=None, bucket_priority=0, global_priority=0, wait=True, **kwargs)#

Requests a route from the Discord API

Parameters:

route (Route) – The route to request
rate_limit_key (str | None) –
A ID used for differentiating rate limits. This should be a bot or oauth2 token.

Note

This should be None for unauthenticated routes or webhooks (does not include modifying the webhook via a bot).
headers (dict[str, str] | None) – Headers to mix with HTTPClient.default_headers to pass to aiohttp.ClientSession.request()
bucket_priority (int) – The request priority to pass to Bucket. Lower priority will be picked first.
global_priority (int) –
The request priority for global requests. Lower priority will be picked first.

Warning

This may be ignored by your BaseGlobalRateLimiter.
wait (bool) –
Wait when rate limited.

This will raise RateLimitedError if set to False and you are rate limited.
kwargs (Any) – Keyword arguments to pass to aiohttp.ClientSession.request()

Returns:

The response from the request.

Return type:

ClientResponse

Raises:

RuntimeError – HTTPClient.setup() was not called yet.
RuntimeError – HTTPClient was closed.
RateLimitedError – You are rate limited, and wait was set to False
CloudflareBanError – You have been temporarily banned from the Discord API for 1 hour due to too many requests. Read the documentation for more information.
BadRequestError – The request data was invalid.
UnauthorizedError – No token was provided on a endpoint that requires a token.
ForbiddenError – The token was valid but you do not have permission to use do this.
NotFoundError – The endpoint you requested was not found or a route parameter was invalid.
InternalServerError – Discord is having issues. Try again later.
HTTPRequestStatusError – A non-200 status code was returned.

async setup()#

Sets up the HTTP session

Warning

This has to be called before HTTPClient._request() or HTTPClient.connect_to_gateway()

Raises:: RuntimeError – This can only be called once
Return type:: None

class nextcore.http.Route(method, path, *, ignore_global=False, guild_id=None, channel_id=None, webhook_id=None, webhook_token=None, **parameters)#

Metadata about a discord API route

Example usage

route = Route("GET", "/guilds/{guild_id}", guild_id=1234567890)

Parameters:

method (Literal['GET', 'HEAD', 'POST', 'PUT', 'DELETE', 'CONNECT', 'OPTIONS', 'TRACE', 'PATCH']) – The HTTP method of the route
path (LiteralString) – The path of the route. This can include python formatting strings ({var_here}) from kwargs
ignore_global (bool) – If this route bypasses the global rate limit.
guild_id (Snowflake | None) – Major parameters which will be included in parameters and count towards the rate limit.
channel_id (Snowflake | None) – Major parameters which will be included in parameters and count towards the rate limit.
webhook_id (Snowflake | None) – Major parameters which will be included in parameters and count towards the rate limit.
webhook_token (str | None) – Major parameters which will be included in parameters and count towards the rate limit.
parameters (Snowflake) –
The parameters of the route. These will be used to format the path.

This will be included in Route.bucket

method#: The HTTP method of the route

route#: The path of the route. This can include python formatting strings ({var_here}) from kwargs.

path#: The formatted version of Route.route

ignore_global#

If this route bypasses the global rate limit.

This is always True for unauthenticated routes.

bucket#

The rate limit bucket this fits in.

This is created from Route.guild_id, Route.channel_id, Route.webhook_id, Bucket.method and Route.path

class nextcore.http.RateLimitStorage#

Storage for rate limits for a user.

One of these should be created for each user.

Note

This will register a gc callback to clean up the buckets.

global_lock#: The users per user global rate limit.

async close()#

Clean up before deletion.

Warning

If this is not called before you delete this or it goes out of scope, you will get a memory leak.

Return type:: None

async get_bucket_by_discord_id(discord_id)#

Get a rate limit bucket from the Discord bucket hash.

This can be obtained via the X-Ratelimit-Bucket header.

Parameters:: discord_id (str) – The Discord bucket hash
Return type:: Bucket | None

async get_bucket_by_nextcore_id(nextcore_id)#

Get a rate limit bucket from a nextcore created id.

Parameters:: nextcore_id (str) – The nextcore generated bucket id. This can be gotten by using Route.bucket
Return type:: Bucket | None

async get_bucket_metadata(bucket_route)#

Get the metadata for a bucket from the route.

Parameters:: bucket_route (str) – The bucket route.
Return type:: BucketMetadata | None

async store_bucket_by_discord_id(discord_id, bucket)#

Store a rate limit bucket by the discord bucket hash.

This can be obtained via the X-Ratelimit-Bucket header.

Parameters:

discord_id (str) – The Discord bucket hash
bucket (Bucket) – The bucket to store.

Return type:

None

async store_bucket_by_nextcore_id(nextcore_id, bucket)#

Store a rate limit bucket by nextcore generated id.

Parameters:

nextcore_id (str) – The nextcore generated id of the
bucket (Bucket) – The bucket to store.

Return type:

None

async store_metadata(bucket_route, metadata)#

Store the metadata for a bucket from the route.

Parameters:

bucket_route (str) – The bucket route.
metadata (BucketMetadata) – The metadata to store.

Return type:

None

Authentication#

class nextcore.http.BaseAuthentication#

A wrapper around discord credentials.

Warning

This is a base class. You should probably use BotAuthentication or BearerAuthentication instead.

Example implementation

class BotAuthentication(BaseAuthentication):
    """A wrapper around bot token authentication.

    **Example usage**

    .. code-block:: python3

        authentication = BotAuthentication(os.environ["TOKEN"])

        route = Route("GET", "/gateway/bot")
        await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers)

    Parameters
    ----------
    token:
        The bot token.

    Attributes
    ----------
    prefix:
        The prefix of the token.
    token:
        The bot token
    """

    __slots__: tuple[str, ...] = ()

    def __init__(self, token: str) -> None:
        self.prefix: Literal["Bot"] = "Bot"
        self.token: str = token

    @property
    def rate_limit_key(self) -> str:
        """The key used for rate limiting

        This will be in the format ``Bot AABBCC.DDEEFF.GGHHII``

        **Example usage**

        .. code-block:: python3

            await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...)
        """
        return f"{self.prefix} {self.token}"

    @property
    def headers(self) -> dict[str, str]:
        """Headers for doing a authenticated request.

        This will return a dict with a ``Authorization`` field.

        **Example usage**

        .. code-block:: python3

            await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...)
        """

        return {"Authorization": f"{self.prefix} {self.token}"}

prefix#: The prefix of the authentication.

token#: The bot’s token.

abstract property headers: dict[str, str]#

Headers used for making a authenticated request.

This may return a empty dict if headers is not used for authenticating this type of authentication.

Example usage

await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...)

abstract property rate_limit_key: str | None#

The key used for rate limiting

This is usually the prefix + token for for example Bot AABBCC.DDEEFF.GGHHII

Example usage

await http_client.request(route, rate_limit_key=authentication.rate_limit_key, ...)

class nextcore.http.BotAuthentication(token)#

A wrapper around bot token authentication.

Example usage

authentication = BotAuthentication(os.environ["TOKEN"])

route = Route("GET", "/gateway/bot")
await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers)

Parameters:: token (str) – The bot token.

prefix#: The prefix of the token.

token#: The bot token

property headers: dict[str, str]#

Headers for doing a authenticated request.

This will return a dict with a Authorization field.

Example usage

await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...)

property rate_limit_key: str#

The key used for rate limiting

This will be in the format Bot AABBCC.DDEEFF.GGHHII

Example usage

await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...)

class nextcore.http.BearerAuthentication(token)#

A wrapper around OAuth2 Bearer Token authentication.

Parameters:: token (str) – The bearer token.

prefix#: The prefix of the token.

token#: The bearer token

property headers: dict[str, str]#

Headers for doing a authenticated request.

This will return a dict with a Authorization field.

Example usage

await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...)

property rate_limit_key: str#

The key used for rate limiting

This will be in the format Bearer AABBCCDDEEFFGGHHII

Example usage

await http_client.request(route, rate_limit_key=authentication.rate_limit_key, headers=authentication.headers, ...)

Bucket rate limiting#

class nextcore.http.Bucket(metadata)#

A discord rate limit implementation around a bucket.

Example usage

bucket_metadata = BucketMetadata()
bucket = Bucket(bucket_metadata)

async with bucket.acquire():
    # Do request
    await bucket.update(remaining, reset_after_seconds, unlimited=False)

Parameters:: metadata (BucketMetadata) – The metadata for the bucket.

metadata#

The metadata for the bucket.

This should also be updated with info that applies to all buckets like limit and if it is unlimited.

reset_offset_seconds#

How much the resetting should be offset to account for processing/networking delays.

This will be added to the reset time, so for example a offset of 1 will make resetting 1 second slower.

acquire(*, priority=0, wait=True)#

Use a spot in the rate limit.

Example usage

async with bucket.acquire():
    # Do request
    await bucket.update(remaining, reset_after_seconds, unlimited=False)

Parameters:

priority (int) – The priority of a request. A lower number means it will be executed faster.
wait (bool) –
Wait for a spot in the rate limit.

If this is set to False, this will raise RateLimitedError if no spot is available right now.

Raises:

RateLimitedError – You are rate limited and wait was set to False

Return type:

AsyncIterator[None]

async close()#

Cleanup this instance.

This should be done when this instance is never going to be used anymore

Warning

Continued use of this instance will result in instability

property dirty: bool#

Whether the bucket is currently any different from a clean bucket created from a BucketMetadata.

This can be for example if any requests is being made, or if the bucket is waiting for a reset.

class nextcore.http.BucketMetadata(limit=None, *, unlimited=False)#

Metadata about a discord bucket.

Example usage

bucket_metadata = BucketMetadata()
bucket = Bucket(bucket_metadata)

async with bucket.acquire():
    ...

    bucket_metadata.limit = 5 # This can be found in the response headers from discord.
    bucket_metadata.unlimited = False

Parameters:

limit (int | None) – The maximum number of requests that can be made in the given time period.
unlimited (bool) – Whether the bucket has an unlimited number of requests. If this is True, limit has to be None.

limit#: The maximum number of requests that can be made in the given time period.

Note

This will be None if BucketMetadata.unlimited is True.

This will also be None if no limit has been fetched yet.

unlimited#: Wheter the bucket has no rate limiting enabled.

class nextcore.http.RequestSession(*, priority=0, unlimited=False)#

A metadata class about a pending request. This is used by Bucket

Parameters:

priority (int) – The priority of the request. Lower is better!
unlimited (bool) –
If this request was made when the bucket was unlimited.

This exists to make sure that there is no bad state when switching between unlimited and limited.

pending_future#: The future that when set will execute the request.

priority#: The priority of the request. Lower is better!

unlimited#

If this request was made when the bucket was unlimited.

This exists to make sure that there is no bad state when switching between unlimited and limited.

Global rate limiting#

class nextcore.http.BaseGlobalRateLimiter#

A base implementation of a rate-limiter for global-scoped rate-limits.

Warning

This does not contain any implementation!

You are probably looking for LimitedGlobalRateLimiter or UnlimitedGlobalRateLimiter

abstract acquire(*, priority=0, wait=True)#

Use a spot in the rate-limit.

Parameters:

priority (int) –

Warning

This can safely be ignored.

The request priority. Lower number means it will be requested earlier.
wait (bool) –
Whether to wait for a spot in the rate limit.

If this is set to False, this will raise a RateLimitedError

Returns:

A context manager that will wait in __aenter__ until a request should be made.

Return type:

typing.AsyncContextManager

abstract async close()#

Cleanup this instance.

This should be done when this instance is never going to be used anymore

Warning

Continued use of this instance may result in instability

Return type:: None

abstract update(retry_after)#

Updates the rate-limiter with info from a global scoped 429.

Parameters:

retry_after (float) –

The time from the retry_after field in the JSON response or the retry_after header.

Hint

The JSON field has more precision than the header.

Return type:

None

class nextcore.http.LimitedGlobalRateLimiter(limit=50)#

A limited global rate-limiter.

Parameters:: limit (int) – The amount of requests that can be made per second.

update(retry_after)#

A function that gets called whenever the global rate-limit gets exceeded

This just makes a warning log.

Parameters:: retry_after (float) –
Return type:: None

class nextcore.http.UnlimitedGlobalRateLimiter#

A global rate-limiting implementation

This works by allowing infinite requests until one fail, and when one fails stop further requests from being made until retry_after is done.

Warning

This may cause a lot of 429’s. Please use LimitedGlobalRateLimiter unless you are sure you need this.

Warning

This is slower than other implementations due to using Discord’s time.

There is some extra delay due to ping due to this.

acquire(*, priority=0, wait=True)#

Acquire a spot in the rate-limit

Parameters:

priority (int) –

Warning

Request priority currently does nothing.
wait (bool) –
Whether to wait for a spot in the rate limit.

If this is False, this will raise RateLimitedError instead.

Raises:

RateLimitedError – wait was set to False and we are rate limited.

Returns:

A context manager that will wait in __aenter__ until a request should be made.

Return type:

typing.AsyncContextManager

async close()#

Cleanup this instance.

This should be done when this instance is never going to be used anymore

Warning

Continued use of this instance will result in instability

Return type:: None

update(retry_after)#

Updates the rate-limiter with info from a global scoped 429.

Parameters:

retry_after (float) –

The time from the retry_after field in the JSON response or the retry_after header.

Hint

The JSON field has more precision than the header.

Return type:

None

HTTP errors#

exception nextcore.http.RateLimitingFailedError(max_retries, response)#

When rate limiting has failed more than HTTPClient.max_retries times

Hint

This can be due to a un-syncronized clock.

You can change HTTPClient.trust_local_time to False to disable using your local clock, or you could sync your clock.

Ubuntu

You can check if your clock is synchronized by running the following command:

timedatectl

If it is synchronized, it will show “System clock synchronized: yes” and “NTP service: running”

If the system clock is not synchronized but the ntp service is running you will have to wait a few minutes for it to sync.

To enable the ntp service run the following command:

sudo timedatectl set-ntp on

This will automatically sync the system clock every once in a while.

Arch

You can check if your clock is synchronized by running the following command:

timedatectl

If it is synchronized, it will show “System clock synchronized: yes” and “NTP service: running”

If the system clock is not synchronized but the ntp service is running you will have to wait a few minutes for it to sync.

To enable the ntp service run the following command:

sudo timedatectl set-ntp on

This will automatically sync the system clock every once in a while.

Windows

This can be turned on by going to Settings -> Time & language -> Date & time and turning on Set time automatically.

Parameters:

max_retries (int) – How many retries the request used that failed.
response (ClientResponse) – The response to the last request that failed.

Return type:

None

max_retries#: How many retries the request used that failed.

response#: The response to the last request that failed.

exception nextcore.http.CloudflareBanError#

A error for when you get banned by cloudflare

This happens due to getting too many 401, 403 or 429 responses from discord. This will block your access to the API temporarily for an hour.

See the documentation for more info.

exception nextcore.http.HTTPRequestStatusError(error, response)#

A base error for receiving a status code the library doesn’t expect.

Parameters:

error (HTTPErrorResponseData) – The error json from the body.
response (ClientResponse) – The response to the request.

Return type:

None

response#: The response to the request.

error_code#: The error code.

message#: The error message.

error#: The error json from the body.

exception nextcore.http.BadRequestError(error, response)#

A 400 error.

Parameters:

error (HTTPErrorResponseData) –
response (ClientResponse) –

Return type:

None

exception nextcore.http.NotFoundError(error, response)#

A 404 error.

Parameters:

error (HTTPErrorResponseData) –
response (ClientResponse) –

Return type:

None

exception nextcore.http.UnauthorizedError(error, response)#

A 401 error.

Parameters:

error (HTTPErrorResponseData) –
response (ClientResponse) –

Return type:

None

exception nextcore.http.ForbiddenError(error, response)#

A 403 error.

Parameters:

error (HTTPErrorResponseData) –
response (ClientResponse) –

Return type:

None

exception nextcore.http.InternalServerError(error, response)#

A 5xx error.

Parameters:

error (HTTPErrorResponseData) –
response (ClientResponse) –

Return type:

None

HTTP#

Making requests#

Setting up#

Creating a Route#

Doing the request#

Getting the response data#

Cleaning up#

HTTP reference#

Authentication#

Bucket rate limiting#

Global rate limiting#

HTTP errors#

Creating a `Route`#