Reference

eso_downloader

A shell command and python package to make it easier to download large amounts of data from the ESO Science Archive.

eso_downloader.auth

Helper functions and classes to manage authenticated access to the ESO archive.

class eso_downloader.auth.ESOJWTAuth(*, username, service='eso-downloader')

Bases: AuthBase

requests.auth.AuthBase subclass which automatically gets the JWT for ESO archive access.

This uses the keyring package to acquire the user’s password, and if the jwt package is installed, can automatically refresh the JWT as needed.

Parameters:

• username (str) – The ESO portal username to get the token for.

• service (str, optional) – The service name to use when querying keyring.

__annotate_func__ = None

__annotations_cache__ = {}

__call__(r)

Call self as a function.

__dict__ = mappingproxy({'__module__': 'eso_downloader.auth', '__firstlineno__': 57, '__doc__': "\n:class:`requests.auth.AuthBase` subclass which automatically gets the JWT\nfor ESO archive access.\n\nThis uses the ``keyring`` package to acquire the user's password,\nand if the ``jwt`` package is installed, can automatically refresh the JWT\nas needed.\n\nParameters\n----------\nusername: str\n    The ESO portal username to get the token for.\nservice: str, optional\n    The service name to use when querying ``keyring``.\n", '__init__': <function ESOJWTAuth.__init__>, '_get_token': <function ESOJWTAuth._get_token>, 'token_contents': <property object>, '__call__': <function ESOJWTAuth.__call__>, '__static_attributes__': ('_service_name', '_token_content', 'token', 'username'), '__annotations_cache__': {}})

__firstlineno__ = 57

__init__(*, username, service='eso-downloader')

__module__ = 'eso_downloader.auth'

__static_attributes__ = ('_service_name', '_token_content', 'token', 'username')

__weakref__

list of weak references to the object

property token_contents

The parsed contents of the JWT. May be None if parsing failed.

eso_downloader.auth.get_user_session(*, username, session_retries=None, **kwargs)

Get a requests.Session wrapping a ESOJWTAuth instance for authenticated access to the ESO archive.

Parameters:

• username (str) – The ESO portal username to get the token for.

• session_retries (varies, optional) – Passed directly to urllib3, see it for retry details.

• **kwargs (dict, optional) – Extra arguments to ESOJWTAuth creation.

Returns:

A requests.Session instance with authentication configured and with useful headers set (see code for specifics).

Return type:

requests.Session

eso_downloader.auth.store_eso_password(*, username, password, service_name='eso-downloader')

Store the provided username/password pair in the keyring.

Parameters:

• username (str) – The ESO portal username to store.

• password (str) – The ESO portal password to store.

• service (str, optional) – The service name to use when querying keyring.

eso_downloader.cli

The CLI entrypoints and config parsers.

class eso_downloader.cli.Config(cfg)

Bases: object

__annotate_func__ = None

__annotations_cache__ = {}

__dict__ = mappingproxy({'__module__': 'eso_downloader.cli', '__firstlineno__': 49, '__init__': <function Config.__init__>, 'program_requests_from_program_ids': <function Config.program_requests_from_program_ids>, '__static_attributes__': ('_cfg', '_default_base_dir', '_default_dir_structure', '_default_username', '_programs'), '__dict__': <attribute '__dict__' of 'Config' objects>, '__weakref__': <attribute '__weakref__' of 'Config' objects>, '__doc__': None, '__annotations_cache__': {}})

__firstlineno__ = 49

__init__(cfg)

__module__ = 'eso_downloader.cli'

__static_attributes__ = ('_cfg', '_default_base_dir', '_default_dir_structure', '_default_username', '_programs')

__weakref__

list of weak references to the object

program_requests_from_program_ids(program_ids)

exception eso_downloader.cli.ConfigError

Bases: Exception

__cause__

__context__

__dict__ = mappingproxy({'__module__': 'eso_downloader.cli', '__firstlineno__': 45, '__static_attributes__': (), '__weakref__': <attribute '__weakref__' of 'ConfigError' objects>, '__doc__': None, '__annotations_cache__': {}})

__firstlineno__ = 45

__init__(*args, **kwargs)

__module__ = 'eso_downloader.cli'

classmethod __new__(*args, **kwargs)

__reduce__()

Helper for pickle.

__repr__()

Return repr(self).

__setstate__(state, /)

__static_attributes__ = ()

__str__()

Return str(self).

__suppress_context__

__traceback__

__weakref__

list of weak references to the object

add_note(note, /)

Add a note to the exception

args

with_traceback(tb, /)

Set self.__traceback__ to tb and return self.

eso_downloader.cli.main(args=None)

eso_downloader.cli.parse_args(args)

eso_downloader.cli.parse_config(path)

eso_downloader.datalink

Helper classes and functions related to ESO’s datalink services.

class eso_downloader.datalink.DatalinkCalibrationMap

Bases: Mapping

Datalink result handler which collects together rows of the same kind of calibration.

Row classification is based on the "eso_category" column.

__abstractmethods__ = frozenset({})

__annotate_func__ = None

__annotations_cache__ = {}

classmethod __class_getitem__(args, /)

Represent a PEP 585 generic type

E.g. for t = list[int], t.__origin__ is list and t.__args__ is (int,).

__contains__(item)

__dict__ = mappingproxy({'__module__': 'eso_downloader.datalink', '__firstlineno__': 143, '__doc__': '\nDatalink result handler which collects together rows of the same kind of\ncalibration.\n\nRow classification is based on the ``"eso_category"`` column.\n', '__init__': <function DatalinkCalibrationMap.__init__>, 'add_row': <function DatalinkCalibrationMap.add_row>, '__len__': <function DatalinkCalibrationMap.__len__>, '__iter__': <function DatalinkCalibrationMap.__iter__>, '__getitem__': <function DatalinkCalibrationMap.__getitem__>, '__contains__': <function DatalinkCalibrationMap.__contains__>, 'get_datalink_urls': <function DatalinkCalibrationMap.get_datalink_urls>, '__static_attributes__': ('_mapping',), '__dict__': <attribute '__dict__' of 'DatalinkCalibrationMap' objects>, '__weakref__': <attribute '__weakref__' of 'DatalinkCalibrationMap' objects>, '__abstractmethods__': frozenset(), '_abc_impl': <_abc._abc_data object>, '__annotations_cache__': {}})

__eq__(other)

Return self==value.

__firstlineno__ = 143

__getitem__(item)

__hash__ = None

__init__()

__iter__()

__len__()

__module__ = 'eso_downloader.datalink'

__reversed__ = None

__slots__ = ()

__static_attributes__ = ('_mapping',)

classmethod __subclasshook__(C)

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object

add_row(row)

Add a row from a ESO datalink service to the mapping, based on the category of the file.

Parameters:

row – A row from an ESO datalink service.

get(k[, d]) → D[k] if k in D, else d.  d defaults to None.

get_datalink_urls()

Get an iterator over the datalink URLs for the rows held by this wrapper.

Yields:

datalink_url (str) – The datalink URLs for the files

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

values() → an object providing a view on D's values

class eso_downloader.datalink.DatalinkResult(science_file_url, night_log_url, raw_cal_selector_url, processed_cal_selector_url)

Bases: object

A helper class for loading and parsing a datalink result from the ESO archive.

The semantics column is consulted to perform the discovery of relevant rows and their contents.

Parameters:

• science_file_url (str) – The url to the original science file.

• night_log_url (str) – The url to the night log associated with the original science file.

• raw_cal_selector_url (str) – The url to the calibration selector service containing the raw calibrations for the original science file.

• processed_cal_selector_url (str) – The url to the calibration selector service containing the reduced calibrations (and raw if no reduced calibrations are available) for the original science file.

__annotations_cache__ = {}

__attrs_attrs__ = (Attribute(name='science_file_url', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='science_file_url'), Attribute(name='night_log_url', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='night_log_url'), Attribute(name='raw_cal_selector_url', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='raw_cal_selector_url'), Attribute(name='processed_cal_selector_url', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='processed_cal_selector_url'))

__attrs_own_setattr__ = False

__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe>, field_transformer=None)

__eq__(other)

Method generated by attrs for class DatalinkResult.

__firstlineno__ = 41

__getstate__()

Automatically created by attrs.

__hash__ = None

__init__(science_file_url, night_log_url, raw_cal_selector_url, processed_cal_selector_url) → None

Method generated by attrs for class DatalinkResult.

__match_args__ = ('science_file_url', 'night_log_url', 'raw_cal_selector_url', 'processed_cal_selector_url')

__module__ = 'eso_downloader.datalink'

__ne__(other)

Check equality and either forward a NotImplemented or return the result negated.

__replace__(**changes)

Method generated by attrs for class DatalinkResult.

__repr__()

Method generated by attrs for class DatalinkResult.

__setstate__(state)

Automatically created by attrs.

__slots__ = ('science_file_url', 'night_log_url', 'raw_cal_selector_url', 'processed_cal_selector_url', '__weakref__')

__static_attributes__ = ()

__weakref__

classmethod from_datalink_url(*, session, datalink_url)

Create a eso_downloader.datalink.DatalinkResult from a URL to ESO’s datalink service.

Parameters:

• session (requests.Session) – A session containing authentication details allowing access to the data.

• datalink_url (str) – A URL pointing to a ESO’s datalink service for a specific science file.

Returns:

The parsed datalink result.

Return type:

eso_downloader.datalink.DatalinkResult

get_night_log(*, session)

Get the parsed night log for this datalink result.

Parameters:

session (requests.Session) – A session containing authentication details allowing access to the data.

Returns:

The parsed night log for this datalink result.

Return type:

eso_downloader.night_log.NightLog

get_processed_cal_table(*, session)

get_raw_cal_table(*, session)

night_log_url

processed_cal_selector_url

raw_cal_selector_url

science_file_url

class eso_downloader.datalink.DatalinkSemanticMap(mapping)

Bases: Mapping

__abstractmethods__ = frozenset({})

__annotate_func__ = None

__annotations_cache__ = {}

classmethod __class_getitem__(args, /)

Represent a PEP 585 generic type

E.g. for t = list[int], t.__origin__ is list and t.__args__ is (int,).

__contains__(item)

__dict__ = mappingproxy({'__module__': 'eso_downloader.datalink', '__firstlineno__': 193, '__init__': <function DatalinkSemanticMap.__init__>, 'from_table': <classmethod(<function DatalinkSemanticMap.from_table>)>, 'from_datalink_url': <classmethod(<function DatalinkSemanticMap.from_datalink_url>)>, '__len__': <function DatalinkSemanticMap.__len__>, '__iter__': <function DatalinkSemanticMap.__iter__>, '__getitem__': <function DatalinkSemanticMap.__getitem__>, '__contains__': <function DatalinkSemanticMap.__contains__>, 'get_first': <function DatalinkSemanticMap.get_first>, 'get_first_access_url': <function DatalinkSemanticMap.get_first_access_url>, 'calibrations': <property object>, 'this_access_url': <property object>, 'this_row': <property object>, '__static_attributes__': ('_mapping',), '__dict__': <attribute '__dict__' of 'DatalinkSemanticMap' objects>, '__weakref__': <attribute '__weakref__' of 'DatalinkSemanticMap' objects>, '__doc__': None, '__abstractmethods__': frozenset(), '_abc_impl': <_abc._abc_data object>, '__annotations_cache__': {}})

__eq__(other)

Return self==value.

__firstlineno__ = 193

__getitem__(item)

__hash__ = None

__init__(mapping)

__iter__()

__len__()

__module__ = 'eso_downloader.datalink'

__reversed__ = None

__slots__ = ()

__static_attributes__ = ('_mapping',)

classmethod __subclasshook__(C)

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

__weakref__

list of weak references to the object

property calibrations

classmethod from_datalink_url(*, session, url)

classmethod from_table(table)

get(k[, d]) → D[k] if k in D, else d.  d defaults to None.

get_first(item)

get_first_access_url(item)

items() → a set-like object providing a view on D's items

keys() → a set-like object providing a view on D's keys

property this_access_url

property this_row

values() → an object providing a view on D's values

class eso_downloader.datalink.DatalinkSemanticsLabel(*values)

Bases: Enum

An enum.Enum representing the various possible values that the ESO datalink service provide for the semantics column.

CALIBRATION = '#calibration'

CURRENT_FILE = '#this'

DERIVATION = '#derivation'

MASTER_CAL = 'http://archive.eso.org/rdf/datalink/eso#calSelector_raw2master'

NIGHT_LOG = 'http://archive.eso.org/rdf/datalink/eso#night_log'

RAW_CAL = 'http://archive.eso.org/rdf/datalink/eso#calSelector_raw2raw'

classmethod __contains__(value)

Return True if value is in cls.

value is in cls if: 1) value is a member of cls, or 2) value is the value of one of the cls’s members. 3) value is a pseudo-member (flags)

classmethod __getitem__(name)

Return the member matching name.

classmethod __iter__()

Return members in definition order.

classmethod __len__()

Return the number of members (no aliases)

__members__ = mappingproxy({'CURRENT_FILE': <DatalinkSemanticsLabel.CURRENT_FILE: '#this'>, 'NIGHT_LOG': <DatalinkSemanticsLabel.NIGHT_LOG: 'http://archive.eso.org/rdf/datalink/eso#night_log'>, 'RAW_CAL': <DatalinkSemanticsLabel.RAW_CAL: 'http://archive.eso.org/rdf/datalink/eso#calSelector_raw2raw'>, 'MASTER_CAL': <DatalinkSemanticsLabel.MASTER_CAL: 'http://archive.eso.org/rdf/datalink/eso#calSelector_raw2master'>, 'CALIBRATION': <DatalinkSemanticsLabel.CALIBRATION: '#calibration'>, 'DERIVATION': <DatalinkSemanticsLabel.DERIVATION: '#derivation'>})

__module__ = 'eso_downloader.datalink'

__name__ = 'DatalinkSemanticsLabel'

__qualname__ = 'DatalinkSemanticsLabel'

classmethod from_str(s)

Return a DatalinkSemanticsLabel if it exists, or the string if it doesn’t.

eso_downloader.datalink.dp_id_from_datalink_id(link_id)

Get the dp_id from the datalink ID.

eso_downloader.night_log

Helper tools and functions for parsing the night log file produced for ESO observations.

class eso_downloader.night_log.Grade(*values)

Bases: Enum

A = 'A'

B = 'B'

C = 'C'

D = 'D'

NOT_FOUND = None

X = 'X'

classmethod __contains__(value)

Return True if value is in cls.

value is in cls if: 1) value is a member of cls, or 2) value is the value of one of the cls’s members. 3) value is a pseudo-member (flags)

classmethod __getitem__(name)

Return the member matching name.

classmethod __iter__()

Return members in definition order.

classmethod __len__()

Return the number of members (no aliases)

__members__ = mappingproxy({'A': <Grade.A: 'A'>, 'B': <Grade.B: 'B'>, 'C': <Grade.C: 'C'>, 'D': <Grade.D: 'D'>, 'X': <Grade.X: 'X'>, '_': <Grade._: '_'>, 'NOT_FOUND': <Grade.NOT_FOUND: None>})

__module__ = 'eso_downloader.night_log'

__name__ = 'Grade'

__qualname__ = 'Grade'

class eso_downloader.night_log.NightLog(grade)

Bases: object

__annotations_cache__ = {}

__attrs_attrs__ = (Attribute(name='grade', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=<enum 'Grade'>, kw_only=False, inherited=False, on_setattr=None, alias='grade'),)

__attrs_own_setattr__ = True

__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe>, field_transformer=None)

__eq__(other)

Method generated by attrs for class NightLog.

__firstlineno__ = 34

__getstate__()

Automatically created by attrs.

__hash__ = None

__init__(grade) → None

Method generated by attrs for class NightLog.

__match_args__ = ('grade',)

__module__ = 'eso_downloader.night_log'

__ne__(other)

Check equality and either forward a NotImplemented or return the result negated.

__replace__(**changes)

Method generated by attrs for class NightLog.

__repr__()

Method generated by attrs for class NightLog.

__setattr__(name, val)

Method generated by attrs for class NightLog.

__setstate__(state)

Automatically created by attrs.

__slots__ = ('grade', '__weakref__')

__static_attributes__ = ()

__weakref__

classmethod from_file(file)

classmethod from_path(path)

classmethod from_str(s)

grade

property is_acceptable

property is_completed

eso_downloader.pooled_writer

A basic threadpool-based downloader and writer building upon eso_downloader.writer.FileWriterBase for the ESO science archive.

class eso_downloader.pooled_writer.DirectoryStructure(*values)

Bases: Enum

The directory structure to use when saving files.

DECOMPOSED = 'decomposed'

GROUP_OB = 'group_ob'

GROUP_OB_ID = 'group_ob_id'

MAUVE_GECKOS = 'mauve_geckos'

classmethod __contains__(value)

Return True if value is in cls.

value is in cls if: 1) value is a member of cls, or 2) value is the value of one of the cls’s members. 3) value is a pseudo-member (flags)

classmethod __getitem__(name)

Return the member matching name.

classmethod __iter__()

Return members in definition order.

classmethod __len__()

Return the number of members (no aliases)

__members__ = mappingproxy({'DECOMPOSED': <DirectoryStructure.DECOMPOSED: 'decomposed'>, 'GROUP_OB': <DirectoryStructure.GROUP_OB: 'group_ob'>, 'GROUP_OB_ID': <DirectoryStructure.GROUP_OB_ID: 'group_ob_id'>, 'MAUVE_GECKOS': <DirectoryStructure.MAUVE_GECKOS: 'mauve_geckos'>})

__module__ = 'eso_downloader.pooled_writer'

__name__ = 'DirectoryStructure'

__qualname__ = 'DirectoryStructure'

class eso_downloader.pooled_writer.DownloadResult(dp_id, ob_id, science_dp_id, saved_filename=None, error=None)

Bases: object

__annotations_cache__ = {}

__attrs_attrs__ = (Attribute(name='dp_id', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='dp_id'), Attribute(name='ob_id', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='ob_id'), Attribute(name='science_dp_id', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='science_dp_id'), Attribute(name='saved_filename', default=None, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='saved_filename'), Attribute(name='error', default=None, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='error'))

__attrs_own_setattr__ = False

__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe>, field_transformer=None)

__eq__(other)

Method generated by attrs for class DownloadResult.

__firstlineno__ = 118

__getstate__()

Automatically created by attrs.

__hash__ = None

__init__(dp_id, ob_id, science_dp_id, saved_filename=None, error=None) → None

Method generated by attrs for class DownloadResult.

__match_args__ = ('dp_id', 'ob_id', 'science_dp_id', 'saved_filename', 'error')

__module__ = 'eso_downloader.pooled_writer'

__ne__(other)

Check equality and either forward a NotImplemented or return the result negated.

__replace__(**changes)

Method generated by attrs for class DownloadResult.

__repr__()

Method generated by attrs for class DownloadResult.

__setstate__(state)

Automatically created by attrs.

__slots__ = ('dp_id', 'ob_id', 'science_dp_id', 'saved_filename', 'error', '__weakref__')

__static_attributes__ = ()

__weakref__

dp_id

error

log_download_result()

ob_id

saved_filename

science_dp_id

class eso_downloader.pooled_writer.Downloader(dp_id, ob_id, ob_name, science_dp_id, url, session, base_dir, dir_structure, saved_filename=None, dry_run=False)

Bases: object

__annotations_cache__ = {}

__attrs_attrs__ = (Attribute(name='dp_id', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='dp_id'), Attribute(name='ob_id', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='ob_id'), Attribute(name='ob_name', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='ob_name'), Attribute(name='science_dp_id', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='science_dp_id'), Attribute(name='url', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='url'), Attribute(name='session', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='session'), Attribute(name='base_dir', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='base_dir'), Attribute(name='dir_structure', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='dir_structure'), Attribute(name='saved_filename', default=None, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='saved_filename'), Attribute(name='dry_run', default=False, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='dry_run'))

__attrs_own_setattr__ = False

__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe>, field_transformer=None)

__call__()

Call self as a function.

__eq__(other)

Method generated by attrs for class Downloader.

__firstlineno__ = 144

__getstate__()

Automatically created by attrs.

__hash__ = None

__init__(dp_id, ob_id, ob_name, science_dp_id, url, session, base_dir, dir_structure, saved_filename=None, dry_run=False) → None

Method generated by attrs for class Downloader.

__match_args__ = ('dp_id', 'ob_id', 'ob_name', 'science_dp_id', 'url', 'session', 'base_dir', 'dir_structure', 'saved_filename', 'dry_run')

__module__ = 'eso_downloader.pooled_writer'

__ne__(other)

Check equality and either forward a NotImplemented or return the result negated.

__replace__(**changes)

Method generated by attrs for class Downloader.

__repr__()

Method generated by attrs for class Downloader.

__setstate__(state)

Automatically created by attrs.

__slots__ = ('dp_id', 'ob_id', 'ob_name', 'science_dp_id', 'url', 'session', 'base_dir', 'dir_structure', 'saved_filename', 'dry_run', '__weakref__')

__static_attributes__ = ('saved_filename',)

__weakref__

base_dir

dir_structure

do_download()

dp_id

dry_run

fail_download(error)

ob_id

ob_name

pass_download()

saved_filename

science_dp_id

session

url

class eso_downloader.pooled_writer.PooledWriter(*, session, base_dir, executor=None, dir_structure)

Bases: FileWriterBase

__annotate_func__ = None

__annotations_cache__ = {}

__dict__ = mappingproxy({'__module__': 'eso_downloader.pooled_writer', '__firstlineno__': 42, '__init__': <function PooledWriter.__init__>, 'save': <function PooledWriter.save>, '_file_exists': <function PooledWriter._file_exists>, '_start_download': <function PooledWriter._start_download>, 'finalise': <function PooledWriter.finalise>, '__static_attributes__': ('_base_dir', '_dir_structure', '_executor', '_futures'), '__doc__': None, '__annotations_cache__': {}})

__enter__()

__exit__(exc_type, exc_val, exc_tb)

__firstlineno__ = 42

__init__(*, session, base_dir, executor=None, dir_structure)

__module__ = 'eso_downloader.pooled_writer'

__static_attributes__ = ('_base_dir', '_dir_structure', '_executor', '_futures')

__weakref__

list of weak references to the object

filter_calibrations(initial_calibrations)

finalise(*, timeout=None)

handle_calibrations(*, cal_table, science_dp_id, science_ob_id, science_ob_name, dry_run=False)

handle_tap_result(*, result, use_processed_calibrations=False, dry_run=False)

save(*, dp_id, download_url, ob_id, science_dp_id=None, ob_name, dry_run=False)

eso_downloader.pooled_writer.get_save_dir(*, base_dir, ob_id, ob_name, science_dp_id, dir_structure)

eso_downloader.program_request

Helper classes and functions for handling a single program request.

class eso_downloader.program_request.ProgramRequest(program_id, username, base_dir, dir_structure)

Bases: object

Container class for a single program request.

__annotations_cache__ = {}

__attrs_attrs__ = (Attribute(name='program_id', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='program_id'), Attribute(name='username', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='username'), Attribute(name='base_dir', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='base_dir'), Attribute(name='dir_structure', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='dir_structure'))

__attrs_own_setattr__ = False

__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe>, field_transformer=None)

__eq__(other)

Method generated by attrs for class ProgramRequest.

__firstlineno__ = 11

__getstate__()

Automatically created by attrs.

__hash__ = None

__init__(program_id, username, base_dir, dir_structure) → None

Method generated by attrs for class ProgramRequest.

__match_args__ = ('program_id', 'username', 'base_dir', 'dir_structure')

__module__ = 'eso_downloader.program_request'

__ne__(other)

Check equality and either forward a NotImplemented or return the result negated.

__replace__(**changes)

Method generated by attrs for class ProgramRequest.

__repr__()

Method generated by attrs for class ProgramRequest.

__setstate__(state)

Automatically created by attrs.

__slots__ = ('program_id', 'username', 'base_dir', 'dir_structure', '__weakref__')

__static_attributes__ = ()

__weakref__

base_dir

dir_structure

download_files(*, writer=None, session=None, writer_cls=None, **kwargs)

get_science_results(*, session=None)

get_session(**kwargs)

get_writer(*, session=None, writer_cls=None, **kwargs)

log_end()

log_start()

program_id

username

eso_downloader.tap

Helper classes and functions for querying the ESO archive TAP service.

class eso_downloader.tap.ScienceQueryResult(dp_id, datalink_url, ob_id, ob_name)

Bases: object

__annotations_cache__ = {}

__attrs_attrs__ = (Attribute(name='dp_id', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='dp_id'), Attribute(name='datalink_url', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='datalink_url'), Attribute(name='ob_id', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='ob_id'), Attribute(name='ob_name', default=NOTHING, validator=None, repr=True, eq=True, eq_key=None, order=True, order_key=None, hash=None, init=True, metadata=mappingproxy({}), type=None, converter=None, kw_only=False, inherited=False, on_setattr=None, alias='ob_name'))

__attrs_own_setattr__ = False

__attrs_props__ = ClassProps(is_exception=False, is_slotted=True, has_weakref_slot=True, is_frozen=False, kw_only=<KeywordOnly.NO: 'no'>, collected_fields_by_mro=True, added_init=True, added_repr=True, added_eq=True, added_ordering=False, hashability=<Hashability.UNHASHABLE: 'unhashable'>, added_match_args=True, added_str=False, added_pickling=True, on_setattr_hook=<function pipe.<locals>.wrapped_pipe>, field_transformer=None)

__eq__(other)

Method generated by attrs for class ScienceQueryResult.

__firstlineno__ = 14

__getstate__()

Automatically created by attrs.

__hash__ = None

__init__(dp_id, datalink_url, ob_id, ob_name) → None

Method generated by attrs for class ScienceQueryResult.

__match_args__ = ('dp_id', 'datalink_url', 'ob_id', 'ob_name')

__module__ = 'eso_downloader.tap'

__ne__(other)

Check equality and either forward a NotImplemented or return the result negated.

__replace__(**changes)

Method generated by attrs for class ScienceQueryResult.

__repr__()

Method generated by attrs for class ScienceQueryResult.

__setstate__(state)

Automatically created by attrs.

__slots__ = ('dp_id', 'datalink_url', 'ob_id', 'ob_name', '__weakref__')

__static_attributes__ = ()

__weakref__

datalink_url

dp_id

classmethod from_row(row)

ob_id

ob_name

eso_downloader.tap.get_science_datalink_urls(*, session, program_id)

Get an iterator which yields ScienceQueryResult for each result.

eso_downloader.utils

Various helper functions and classes.

exception eso_downloader.utils.DownloadError

Bases: Exception

Exception subclass for ‘user friendly’ errors that occur as part of the download process.

__cause__

__context__

__dict__ = mappingproxy({'__module__': 'eso_downloader.utils', '__firstlineno__': 28, '__doc__': "\nException subclass for 'user friendly' errors that occur as part of the\ndownload process.\n", '__static_attributes__': (), '__weakref__': <attribute '__weakref__' of 'DownloadError' objects>, '__annotations_cache__': {}})

__firstlineno__ = 28

__init__(*args, **kwargs)

__module__ = 'eso_downloader.utils'

classmethod __new__(*args, **kwargs)

__reduce__()

Helper for pickle.

__repr__()

Return repr(self).

__setstate__(state, /)

__static_attributes__ = ()

__str__()

Return str(self).

__suppress_context__

__traceback__

__weakref__

list of weak references to the object

add_note(note, /)

Add a note to the exception

args

with_traceback(tb, /)

Set self.__traceback__ to tb and return self.

eso_downloader.utils.url_to_votable(*, session, url, invalid='mask', **kwargs)

Download a URL and read in as a VOTable.

eso_downloader.writer

Common helper classes and functions for download files based on TAP queries.

class eso_downloader.writer.FileWriterBase(*, session)

Bases: object

__annotate_func__ = None

__annotations_cache__ = {}

__dict__ = mappingproxy({'__module__': 'eso_downloader.writer', '__firstlineno__': 10, '__init__': <function FileWriterBase.__init__>, 'save': <function FileWriterBase.save>, 'handle_tap_result': <function FileWriterBase.handle_tap_result>, 'filter_calibrations': <function FileWriterBase.filter_calibrations>, 'handle_calibrations': <function FileWriterBase.handle_calibrations>, 'finalise': <function FileWriterBase.finalise>, '__enter__': <function FileWriterBase.__enter__>, '__exit__': <function FileWriterBase.__exit__>, '__static_attributes__': ('session',), '__dict__': <attribute '__dict__' of 'FileWriterBase' objects>, '__weakref__': <attribute '__weakref__' of 'FileWriterBase' objects>, '__doc__': None, '__annotations_cache__': {}})

__enter__()

__exit__(exc_type, exc_val, exc_tb)

__firstlineno__ = 10

__init__(*, session)

__module__ = 'eso_downloader.writer'

__static_attributes__ = ('session',)

__weakref__

list of weak references to the object

filter_calibrations(initial_calibrations)

finalise()

handle_calibrations(*, cal_table, science_dp_id, science_ob_id, science_ob_name, dry_run=False)

handle_tap_result(*, result, use_processed_calibrations=False, dry_run=False)

save(*, dp_id, download_url, ob_name, ob_id, science_dp_id=None, dry_run=False)