The Hugging Face module

HFSourceBase

Bases: SourceBase, ABC

TODO: Add docstrings

__init__(env, repo_id, repo_type, root_dir=None)

A generic interface to a HuggingFace repository.

Parameters:

Name	Type	Description	Default
env	Env	An Env object containing loaded configuration options.	required
repo_id	str	HuggingFace repo ID.	required
repo_type	str	HuggingFace repo type.	required
root_dir	Path | None	An optional custom root directory. Defaults to None.	None

get_file(filename)

Retrieve a single (potentially cached) file from the Huggingface stored repo.

Parameters:

Name	Type	Description	Default
filename	str | Path	The file to retrieve.	required

Returns:

Type	Description
Path	A Path object.

get_files(filenames)

Retrieve multiple (potentially cached) files from the HuggingFace stored repo.

Parameters:

Name	Type	Description	Default
filenames	list[str | Path]	The files to retrieve.	required

Returns:

Type	Description
list[Path]	A list of Path objects.

load_dataset(criteria=None, columns=None) abstractmethod

Load and return a dataset.

Args:

criteria:
    Optional criteria used to create a subset.

columns:
    The columns to keep or retrieve.

Returns:

Type	Description
Table	An Ibis table.

GlobalStreetscapesSource

Bases: HFSourceBase

TODO: Add docstrings

__init__(env, root_dir=None)

An interface to the Global Streetscapes repository.

Parameters:

Name	Type	Description	Default
env	Env	An Env object containing loaded configuration options.	required
root_dir	str | Path | None	An optional custom root directory. Defaults to None.	None

dowload_images(table, mp, kv)

Download images from Mapillary and KartaView.

fetch_image_urls(table, mp, kv)

Fetch image URLs from Mapillary and KartaView.

load_csv(filename, root=None)

Load a CSV file from the Global Streetscapes repository.

Parameters:

Name	Type	Description	Default
filename	str | Path	Name of the CSV file.	required
root	str | Path	Optional root directory. Defaults to None.	None

Returns:

Type	Description
Table	An Ibis table.

load_dataset(criteria=None, columns=None)

Load and return a dataset.

Args:

criteria:
    Optional criteria used to create a subset.

columns:
    The columns to keep or retrieve.

Returns:

Type	Description
Table	An Ibis table.

load_parquet(filename, root=None)

Load a Parquet file from the Global Streetscapes repository.

Parameters:

Name	Type	Description	Default
filename	str | Path	A Parquet file to load.	required
root	str | Path	Optional root directory. Defaults to None.	None

Returns:

Type	Description
	An Ibis table.