granicus_archiver.downloader

exception granicus_archiver.downloader.DownloadError[source]

Bases: Exception

Raised if the downloaded content size does not match the “Content-Length” from the response headers

exception granicus_archiver.downloader.StupidZeroContentLengthError[source]

Bases: Exception

Raised when the reported “Content-Length” is zero

exception granicus_archiver.downloader.ThisShouldBeA404ErrorButItsNot(request_info: None, history: Tuple[None, ...], *, code: int | None = None, status: int | None = None, message: str = '', headers: MultiMapping[str] | None = None)[source]

Bases: ClientResponseError

Exception raised when Granicus redirects you to a warning page saying a file doesn’t exist even though that is something built into the HTTP protocol yes this is a runon sentence in a docstring header line but I am beyond the point of caring

Yes, another lovely edge case discovered - shocking.

The response can be detected by a 302 "Found" redirect with "/Confirmation.aspx" as the url path and M1=Gone in the query.

Parameters:

  • request_info (None)

  • history (Tuple[None, ...])

  • code (int | None)

  • status (int | None)

  • message (str)

  • headers (MultiMapping[str] | None)

Return type:

None

classmethod detect(response: ClientResponse) → bool[source]

Check whether this exception should be raised from the given response

Parameters:

response (ClientResponse)

Return type:

bool

classmethod detect_and_raise(response: ClientResponse) → None[source]

Check and raise this exception if its pattern is found on response

Parameters:

response (ClientResponse)

Return type:

None

class granicus_archiver.downloader.DownloadRequest[source]

Bases: TypedDict

Keyword arguments to create FileDownload instances

url: URL

URL for the download

filename: Path

Local filename for the download

chunk_size: NotRequired[int]

Chunk size for streaming download segments

timeout: NotRequired[ClientTimeout]

Custom ClientTimeout specification

class granicus_archiver.downloader.DownloadResult[source]

Bases: TypedDict

Result type used for FileDownload.result

url: URL

The url of the request

filename: Path

The filename of the request

meta: FileMeta

Metadata from the response headers as a FileMeta instance

class granicus_archiver.downloader.FileDownload(session: ClientSession, **kwargs: Unpack[DownloadRequest])[source]

Bases: object

Downloads a single file

Parameters:
session: ClientSession

The current aiohttp.ClientSession

progress: float

Current download progress (from 0.0 to 1.0)

property meta: FileMeta

Metadata from the response headers

property result: DownloadResult

The completed DownloadResult

class granicus_archiver.downloader.Downloader(session: ClientSession, scheduler: Scheduler | None = None)[source]

Bases: object

Manager for scheduling FileDownload jobs

Parameters:

  • session (ClientSession)

  • scheduler (Scheduler | None)

session: ClientSession

The current aiohttp.ClientSession

scheduler: Scheduler | None

The Scheduler to place download jobs on

default_chunk_size: int

Default to use for FileDownload.chunk_size

default_timeout: ClientTimeout

Default to use for FileDownload.timeout

async spawn(**kwargs: Unpack[DownloadRequest]) → Job[FileDownload][source]

Create a FileDownload and run it on the scheduler

Parameters:

**kwargs (DownloadRequest) – Keyword arguments to initialize the FileDownload instance

Return type:

Job[FileDownload]

async download(**kwargs: Unpack[DownloadRequest]) → FileDownload[source]

Create a FileDownload and begin downloading immediately (bypassing scheduler)

Parameters:

**kwargs (DownloadRequest) – Keyword arguments to initialize the FileDownload instance

Return type:

FileDownload