granicus_archiver.web.s3client

class granicus_archiver.web.s3client.DataFileMetadata[source]

Bases: TypedDict

Metadata for a data file

e_tag: str

The ETag of the file

sha1: SHA1Hash

The SHA1 hash of the file

class granicus_archiver.web.s3client.S3Client(app: Application)[source]

Bases: ClientBase

S3 client for downloading data files and assets

Parameters:

app (web.Application)

data_dirs: DataFiles

Remote data directories

data_files_local: DataFiles

Local data files

data_files_remote: DataFiles

Remote data files

metadata_file: Path

Filename to store cahced metadata

data_file_metadata: dict[DataFileType, DataFileMetadata | None]

Cached metadata for data files

async get_search_index_dir() → None[source]

Download the search index directory

Existing files will be overwritten.

Return type:

None

async get_data_files() → bool[source]

Download data files if they have changed remotely

Return type:

bool

async download_data_file(key: Literal['clips', 'legistar', 'legistar_rguid'], remote_metadata: DataFileMetadata | None = None) → bool[source]

Download a data file if it has changed remotely

Parameters:
Return type:

bool

async get_data_file_remote_meta(key: Literal['clips', 'legistar', 'legistar_rguid']) → DataFileMetadata[source]

Get the remote metadata for a data file

Parameters:

key (Literal['clips', 'legistar', 'legistar_rguid'])

Return type:

DataFileMetadata

load_data_file_metadata() → dict[Literal['clips', 'legistar', 'legistar_rguid'], DataFileMetadata | None][source]

Load the data file metadata from disk

Return type:

dict[Literal[‘clips’, ‘legistar’, ‘legistar_rguid’], ~granicus_archiver.web.s3client.DataFileMetadata | None]

save_data_file_metadata() → None[source]

Save the data file metadata to disk

Return type:

None

granicus_archiver.web.s3client.S3ClientKey = <AppKey(granicus_archiver.web.s3client.S3Client, type=granicus_archiver.web.s3client.S3Client)>

App key for the s3client.S3Client instance