granicus_archiver.clips.model

class granicus_archiver.clips.model.CLIP_ID

Unique id for a Clip

alias of str

granicus_archiver.clips.model.ClipFileKey

Key to for file types in ParseClipLinks and ClipFiles

alias of Literal[‘agenda’, ‘minutes’, ‘audio’, ‘video’]

granicus_archiver.clips.model.ClipFileUploadKey

Key for file types in ClipFiles

alias of Literal[‘agenda’, ‘minutes’, ‘audio’, ‘video’] | Literal[‘chapters’, ‘agenda_packet’]

class granicus_archiver.clips.model.Location

The “Location” (or folder) of a clip

alias of str

Bases: Serializable

Links for clip assets

Parameters:

  • agenda (URL | None)

  • minutes (URL | None)

  • audio (URL | None)

  • video (URL | None)

agenda: URL | None = None

Agenda link

minutes: URL | None = None

Minutes link

audio: URL | None = None

MP3 link

video: URL | None = None

MP4 link

merge(other: ParseClipLinks) → bool[source]

Merge any data missing in self from other

Parameters:

other (ParseClipLinks)

Return type:

bool

class granicus_archiver.clips.model.ParseClipData(id: CLIP_ID, location: Location, name: str, date: int, duration: int, original_links: ParseClipLinks, actual_links: ParseClipLinks | None = None, player_link: URL | None = None)[source]

Bases: Serializable

Data for a clip parsed from granicus

Parameters:
id: CLIP_ID

The (assumingly) primary key of the clip

location: Location

The “Location” (category or folder would be better terms)

name: str

The clip name

date: int

POSIX timestamp of the clip

duration: int

Duration of the clip (in seconds)

The asset links as reported by granicus. Some will be actually be redirects to a PDF viewer which will need to be resolved

The original_links after the redirects have been resolved

URL for the popup video player with agenda view

property datetime: datetime

The clip’s datetime (derived from the date)

property title_name: str

Combination of the clip’s formatted datetime, name and id

property unique_name: str

A unique name for the clip

Iterate over links existing in original_links but missing from actual_links

Return type:

Iterator[tuple[Literal[‘agenda’, ‘minutes’, ‘audio’, ‘video’], ~yarl.URL]]

Check if any links in original_links are missing from actual_links

build_fs_dir(root_dir: Path | None, replace_invalid: bool = True) → Path[source]

Create a path for the clip within the given root_dir

If replace_invalid is True, forward slashes (“/”) will be replaced with colons (“:”) to prevent invalid and unexpected path names

Parameters:

  • root_dir (Path | None)

  • replace_invalid (bool)

Return type:

Path

check(other: Self) → None[source]

Check other for any missing data in self

Parameters:

other (Self)

Return type:

None

exception granicus_archiver.clips.model.CheckError(clip: Clip, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], msg: str | None = None)[source]

Bases: Exception

Base exception for ClipFiles.check()

Parameters:

  • clip (Clip)

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

  • msg (str | None)

Return type:

None

exception granicus_archiver.clips.model.NoMetaError(clip: Clip, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], msg: str | None = None)[source]

Bases: CheckError

Raised when a file exists locally with no stored FileMeta

Parameters:

  • clip (Clip)

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

  • msg (str | None)

Return type:

None

exception granicus_archiver.clips.model.ContentLengthError(clip: Clip, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], msg: str | None = None)[source]

Bases: CheckError

Raised when the stored content_length does not match the local filesize

Parameters:

  • clip (Clip)

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

  • msg (str | None)

Return type:

None

exception granicus_archiver.clips.model.ContentTypeError(clip: Clip, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], msg: str | None = None)[source]

Bases: CheckError

Raised when the content_type disagrees with the file extension

Parameters:

  • clip (Clip)

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

  • msg (str | None)

Return type:

None

classmethod check_and_raise(clip: Clip, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], meta: FileMeta, filename: PathLike) → None[source]

Check the metadata against the file extension using mimetypes and raise this exception if they do not match

Parameters:
Return type:

None

exception granicus_archiver.clips.model.NoFileError(clip: Clip, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], msg: str | None = None)[source]

Bases: CheckError

Raised when there is metadata for a file that does not exist locally

Parameters:

  • clip (Clip)

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

  • msg (str | None)

Return type:

None

exception granicus_archiver.clips.model.FileShouldNotExist(clip: Clip, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], msg: str | None = None)[source]

Bases: CheckError

Raised when the stored metadata for a file is zero-length but the file exists locally

Parameters:

  • clip (Clip)

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

  • msg (str | None)

Return type:

None

exception granicus_archiver.clips.model.FilesizeMagicNumber(clip: Clip, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], msg: str | None = None)[source]

Bases: CheckError

Raised when the filesize is exactly 1245 bytes

Don’t ask me why, but this happens with some pdf downloads. It usually means the file can be re-downloaded because the Content-Length in the header shows a different value.

(I stopped asking why their systems are the way they are a long time ago)

Parameters:

  • clip (Clip)

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

  • msg (str | None)

Return type:

None

classmethod is_magic_number(item: Path | FileMeta | int) → bool[source]

Check if the value equals 1245

This seriously feels a bit like the is-thirteen package.

Parameters:

item (Path | FileMeta | int)

Return type:

bool

classmethod check_and_raise(clip: Clip, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], item: Path | FileMeta | int, msg: str | None = None) → None[source]

Check the given item against is_magic_number() and raise this exception if true

Parameters:

  • clip (Clip)

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

  • item (Path | FileMeta | int)

  • msg (str | None)

Return type:

None

class granicus_archiver.clips.model.ClipFiles(clip: ~granicus_archiver.clips.model.Clip, agenda: ~pathlib._local.Path | None, minutes: ~pathlib._local.Path | None, audio: ~pathlib._local.Path | None, video: ~pathlib._local.Path | None, chapters: ~pathlib._local.Path | None = None, agenda_packet: ~pathlib._local.Path | None = None, metadata: dict[~typing.Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], ~granicus_archiver.types.FileMeta] = <factory>)[source]

Bases: Serializable

File information for a Clip

Parameters:

  • clip (Clip)

  • agenda (Path | None)

  • minutes (Path | None)

  • audio (Path | None)

  • video (Path | None)

  • chapters (Path | None)

  • agenda_packet (Path | None)

  • metadata (dict[Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], ~granicus_archiver.types.FileMeta])

clip: Clip

The parent Clip

agenda: Path | None

Agenda filename

minutes: Path | None

Minutes filename

audio: Path | None

MP3 filename

video: Path | None

MP4 filename

chapters: Path | None = None

WebVTT chapters filename (built by AgendaTimestamps.build_vtt())

agenda_packet: Path | None = None

Agenda packet (parsed from legistar.detail_page.DetailPageResult)

metadata: dict[Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], FileMeta]

FileMeta for each file (if available)

classmethod from_parse_data(clip: Clip, parse_data: ParseClipData) → Self[source]

Create an instance from a ParseClipData instance

Parameters:
Return type:

Self

classmethod build_path(root_dir: Path, key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet']) → Path[source]

Build the filename for the given file type with root_dir prepended

Parameters:

  • root_dir (Path)

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

Return type:

Path

ensure_path(key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet']) → None[source]

Ensure path for key is set on the instance

Raises:

ValueError – If the file does not exist

Parameters:

key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

Return type:

None

get_metadata(key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet']) → FileMeta | None[source]

Get the FileMeta for the given file type (if available)

Parameters:

key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

Return type:

FileMeta | None

set_metadata(key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], meta: FileMeta | MultiMapping[str] | dict[str, str]) → FileMeta[source]

Set the FileMeta for the given file type from request headers

Parameters:
Return type:

FileMeta

check_chapters_file() → bool[source]

Check for an existing chapters file

If chapters is not set and the expected filename for it exists, the filename and its metadata will be added.

Return type:

bool

check()[source]

Check local files against the stored metadata

Raises:

CheckError – A subclass of CheckError if any errors are found

ensure_local_hashes(check_existing: bool = False) → bool[source]

Ensure that all local files have an sha1 hash stored in metadata

Parameters:

check_existing (bool) – If True, the hash of the local file will be checked against the stored hash

Returns:

True if any hashes were generated or updated

Return type:

bool

iter_existing(for_download: bool = True) → Iterator[tuple[Literal['agenda', 'minutes', 'audio', 'video'], Path]][source]
iter_existing(for_download: bool = False) → Iterator[tuple[Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], Path]]

Iterate over existing filenames

Parameters:

for_download – If True (the default), only the filenames expected to be on the Granicus server will be yielded. If False, locally-generated files will be included (such as chapters).

Yields:
class granicus_archiver.clips.model.AgendaTimestamp(seconds: int, text: str)[source]

Bases: Serializable

A timestamped agenda item

Parameters:
seconds: int

The timestamp in seconds

text: str

Agenda item text

property time_str: str

seconds formatted as HH:MM:SS

class granicus_archiver.clips.model.AgendaTimestamps(clip_id: CLIP_ID, items: list[AgendaTimestamp])[source]

Bases: Serializable

Collection of AgendaTimestamp for a Clip

Parameters:
clip_id: CLIP_ID

The associated clip’s id

items: list[AgendaTimestamp]

Timestamps for the clip

build_vtt(clip: Clip) → str[source]

Generate WebVTT-formatted chapters with timestamp/text data

Parameters:

clip (Clip) – The associated Clip instance. This is needed for the end timestamp of last cue in the VTT track.

Return type:

str

class granicus_archiver.clips.model.AgendaTimestampCollection(clips: dict[~granicus_archiver.clips.model.CLIP_ID, ~granicus_archiver.clips.model.AgendaTimestamps] = <factory>)[source]

Bases: Serializable

Container for AgendaTimestamps

Parameters:

clips (dict[CLIP_ID, AgendaTimestamps])

clips: dict[CLIP_ID, AgendaTimestamps]
classmethod load(filename: PathLike) → Self[source]

Loads an instance from previously saved data

Parameters:

filename (PathLike)

Return type:

Self

save(filename: PathLike, indent: int | None = 2) → None[source]

Saves all data as JSON to the given filename

Parameters:
Return type:

None

add(item: AgendaTimestamps) → None[source]

Add an AgendaTimestamps instance

Parameters:

item (AgendaTimestamps)

Return type:

None

get(key: CLIP_ID | Clip) → AgendaTimestamps | None[source]

Get an AgendaTimestamps object if it exists

The key can be a Clip instance or the clip’s id

Parameters:

key (CLIP_ID | Clip)

Return type:

AgendaTimestamps | None

class granicus_archiver.clips.model.Clip(parse_data: ParseClipData, root_dir: Path, parent: ClipCollection)[source]

Bases: Serializable

Stores all information for a single clip

Parameters:
root_dir: Path

Path for the clip (relative to its parent)

files: ClipFiles

The clip’s file information

parent: ClipCollection

The parent ClipCollection

property id: CLIP_ID

Alias for ParseClipData.id

property name: str

Alias for ParseClipData.name

property unique_name: str

Alias for ParseClipData.unique_name

property location: Location

Alias for ParseClipData.location

property root_dir_abs: Path

The root_dir with its parent prepended

get_file_path(key: Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], absolute: bool = False) → Path[source]

Get the relative or absolute path for the given file type

Parameters:

  • key (Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'])

  • absolute (bool)

Return type:

Path

classmethod from_parse_data(parent: ClipCollection, parse_data: ParseClipData) → Self[source]

Create an instance from a ParseClipData instance

Parameters:
Return type:

Self

iter_url_paths(actual: bool = True, absolute: bool = True) → Iterator[tuple[Literal['agenda', 'minutes', 'audio', 'video'], URL, Path]][source]

Iterate over the clip’s file types, url’s and filenames

Parameters:

  • actual (bool) – If True, only yields file types with valid URL’s

  • absolute (bool) – If True, the root_dir_abs is prepended to each filename

Return type:

Iterator[tuple[Literal[‘agenda’, ‘minutes’, ‘audio’, ‘video’], ~yarl.URL, ~pathlib._local.Path]]

iter_paths(absolute: bool = True, for_download: bool = False) → Iterator[tuple[Literal['agenda', 'minutes', 'audio', 'video', 'chapters', 'agenda_packet'], Path]][source]
iter_paths(absolute: bool = True, for_download: bool = True) → Iterator[tuple[Literal['agenda', 'minutes', 'audio', 'video'], Path]]

Iterate over paths in files

Parameters:

  • absolute – Whether to yield absolute paths

  • for_download – If True (the default), only the filenames expected to be on the Granicus server will be yielded. If False, locally-generated files will be included (such as ClipFiles.chapters).

Yields:
class granicus_archiver.clips.model.ClipCollection(base_dir: ~pathlib._local.Path, clips: dict[~granicus_archiver.clips.model.CLIP_ID, ~granicus_archiver.clips.model.Clip] = <factory>)[source]

Bases: Serializable

Container for Clips

Parameters:
base_dir: Path

Root filesystem path for the clip assets

classmethod load(filename: PathLike) → Self[source]

Loads an instance from previously saved data

Parameters:

filename (PathLike)

Return type:

Self

save(filename: PathLike, indent: int | None = 2) → None[source]

Saves all clip data as JSON to the given filename

Parameters:
Return type:

None

add_clip(parse_data: ParseClipData) → Clip[source]

Parse a Clip from the ParseClipData and add it to the collection

Parameters:

parse_data (ParseClipData)

Return type:

Clip

merge(other: ClipCollection) → ClipCollection[source]

Merge the clips in this instance with another

Parameters:

other (ClipCollection)

Return type:

ClipCollection

class granicus_archiver.clips.model.ClipIndex(id: CLIP_ID, location: Location, name: str, datetime: datetime, data_file: Path)[source]

Bases: Serializable

Model for only essential Clip data to be included in ClipsIndex

Parameters:
id: CLIP_ID

Clip.id

location: Location

Clip.location

name: str

Clip.name

datetime: datetime

Clip.datetime

data_file: Path

Path to the full Clip data file within its root_dir

classmethod from_clip(clip: Clip, root_dir: Path) → Self[source]

Create an instance from a Clip

Parameters:
Return type:

Self

write_data(clip: Clip | ClipCollection, exist_ok: bool = False, indent: int | None = 2) → None[source]

Serialize the clip data and save it to data_file

Parameters:
Return type:

None

class granicus_archiver.clips.model.ClipsIndex(clips: dict[CLIP_ID, ClipIndex], root_dir: Path)[source]

Bases: Serializable

An index of all clips containing a minimal amount of data

When serialized, this data will contain only basic clip information with relative paths to each clip’s full data representation.

This is intended for web services to use in order to avoid fetching a large amount of unnecessary data.

Parameters:
clips: dict[CLIP_ID, ClipIndex]

Mapping of ClipIndex using the ClipIndex.id as keys

root_dir: Path

Relative parent directory of the ClipCollection.base_dir

classmethod from_clip_collection(clips: ClipCollection, root_dir: PathLike) → Self[source]

Create an instance from a ClipCollection

Parameters:
Return type:

Self

write_data(clip_collection: ClipCollection | None = None, exist_ok: bool = False, indent: int | None = 2) → None[source]

Write the root index data and each clip's full data

The root index will be stored as “clip-index.json” within the root_dir. All items in clips will be flattened as a list of the serialized form of ClipIndex.

Parameters:

  • clip_collection (ClipCollection | None) – If provided, the write_data() method will be called on each item in clips using the clip collection’s data. If None, only the root index data will be saved.

  • exist_ok (bool) – If False and the data file exists, it will not be overwritten and an exception will be raised. This will also be passed when calling ClipIndex.write_data().

  • indent (int | None) – Indentation parameter to pass to json.dumps()

Return type:

None