pycarol.connectors

class pycarol.connectors.Connectors(carol)

This class handle all APIs related to Carol connectors

Args:

carol: class: pycarol.Carol

create(name, label=None, group_name='Others', overwrite=False)

Create a connector

Args:

name: ‘str’

Connector name

label: ‘str’

Connector label in UI

group_name: str default “Others”

Connector group name in UI

overwrite: bool default False

Overwrite if already exists.

delete_by_id(connector_id=None, mdm_id=None, force_deletion=True)

Delete Connector by ID

Args:

connector_id: str`

Connector ID

mdm_id: str`

Connector ID

force_deletion: bool default True

Force the deletion

Returns: None

delete_by_name(name, force_deletion=True)

Delete connector by name

Args:

name: str

Connector name

force_deletion: bool default True

Force the deletion

Returns: None

drop_etls(etl_list)

Drop ETLs from ETL list.

Args:

login: login: pycarol.Carol

Carol() instance.

etl_list: list

list of ETLs mdm_ids to delete.

Returns: None

drop_single_etl(staging_name, output_list, connector_name=None, connector_id=None)

Drop ETL based on the outputs of a given ETL.

Args:

login: login: pycarol.Carol

Carol() instance.

staging_name: str

staging to drop etls from

output_list: list

output list of the etl to drop. It will only drop the ETL if all the outputs are present.

connector_name: str

connector_name to drop etls from

connector_id: str

connector_id to drop etls from

Returns: None

find_by_staging(staging_name=None)

Find connector given a staging table

Args:

staging_name: str default None

Staging table name

Returns: dict

Connector information

get_all(offset=0, page_size=-1, sort_order='ASC', sort_by=None, include_connectors=False, include_mappings=False, include_consumption=False, print_status=True, save_results=False, filename='connectors.json')

Get all connectors.

Args:

offset: int, default 0

Offset for the response.

page_size: int, default 1000

Number of records in each call.

sort_by: str default None

Field to sort by

sort_order: str, default ASC

Sort Order. Possible values “ASC” and “DESC”

include_connectors: bool default False

Include connector information

include_mappings: bool default False

Include mapping information

include_consumption: bool default False

Include consumption information

print_status: bool default True

Print status of the request.

save_results: bool default False

If save json with the results.

filename: str default None

Filename to save

Returns: dict

Connector information

get_all_stagings(connector_name=None, connector_id=None)

Get all stagings from a connector.

Args:

connector_name: str, str, default None

Connector Name

connector_id: str, str, default None

Connector ID

Returns: list

List of staging tables.

get_by_id(id, errors='raise')

Get connector information using the connector name

Args:

id: ‘str’

Connector ID

errors: {‘ignore’, ‘raise’}, default ‘raise’

If ‘raise’, then invalid request will raise an exception If ‘ignore’, then invalid request will return the request response

Returns: dict

connector information.

get_by_name(name, errors='raise')

Get connector information using the connector name

Args:

name: ‘str’

Connector Name

errors: {‘ignore’, ‘raise’}, default ‘raise’

If ‘raise’, then invalid request will raise an exception If ‘ignore’, then invalid request will return the request response

Returns: dict

connector information.

get_dm_mappings(connector_id=None, connector_name=None, staging_name=None, dm_id=None, dm_name=None, reverse_mapping=False, offset=0, page_size=1000, sort_by=None, sort_order='ASC', all_connectors=False)

Get data models mappings information.

Args:

connector_id: str, default None

Connector Id

connector_name: str, default None

Connectot name

staging_name: str, default None

Staging name

dm_id: str, default None

Data model Id

dm_name: str, default None

Data model name

reverse_mapping: bool, default False

If to return the reverse mapping.

offset: int, default 0

Offset for the response.

page_size: int, default 1000

Number of records in each call.

sort_by: str default None

Field to sort by

sort_order: str, default ASC

Sort Order. Possible values “ASC” and “DESC”

all_connectors: bool, default False

It will return all the mapping for all connectors/stagings

Returns: dict

Mapping json definition

get_entity_mappings(connector_name=None, connector_id=None, reverse_mapping=False, staging_name=None, offset=0, page_size=1000, sort_order='ASC', sort_by=None, print_status=False, errors='raise')

Get all Entity Mappings.

Args:

connector_name: str, str, default None

Connector Name

connector_id: str, str, default None

Connector ID

reverse_mapping: bool default False

When using with consumer.False if you don’t know what consumer is.

staging_name: str` default None

Name of the staging to find the mapping. Returns 404 if there is not mapping..

offset: int, default 0

Offset for pagination. Only used when scrollable=False

page_size: int, default 100

Number of records downloaded in each pagination. The maximum value is 1000

sort_order: str, default ‘ASC’

Sort ascending (‘ASC’) vs. descending (‘DESC’).

sort_by: str, default None

Name to sort by.

print_status: bool, default False

Print the number of records in each interaction.

errors: {‘ignore’, ‘raise’}, default ‘raise’

If ‘raise’, then invalid request will raise an exception If ‘ignore’, then invalid request will return the request response.

Returns: list of dict

List of dict with mappings,

get_etl_information(connector_id=None, connector_name=None)

Get ETL Configurations for a connector

Args:

connector_name: str, str, default None

Connector Name

connector_id: str, str, default None

Connector ID

Returns: list of dict

Etl information

pause_etl(staging_name=None, connector_name=None, connector_id=None)

Pause ETL.

Args:

staging_name:

Staging name to stop the ETLs

connector_name: str, str, default None

Connector Name

connector_id: str, str, default None

Connector ID

Returns: dict

Dictionary with the response Carol’s response

pause_mapping(entity_mapping_id=None, staging_name=None, connector_name=None, connector_id=None, reverse_mapping=False)

Pause mapping.

Args:

entity_mapping_id: str or list of strings

Mapping ids to be stopped.

staging_name:

Staging name to stop the mappings

connector_name: str, str, default None

Connector Name

connector_id: str, str, default None

Connector ID

reverse_mapping: bool default False

When using with consumer.False if you don’t know what consumer is.

Returns: dict

Dictionary with the response of all mappings played.

pause_single_staging_etl(staging_name, output_list, connector_name=None, connector_id=None)

Pause a single staging ETL based on its output.

Args:

staging_name (str): Staging name output_list (list): List of output staging names connector_name (str, optional): Connector name. Defaults to None. connector_id (str, optional): Connector ID. Defaults to None.

Returns:

list: Carol’s response

play_etl(staging_name=None, connector_name=None, connector_id=None)

Start ETL processes.

Args:

staging_name:

Staging name for starting the ETLs

connector_name: str, str, default None

Connector Name

connector_id: str, str, default None

Connector ID

Returns: dict

Dictionary with the response of all mappings played.

play_mapping(entity_mapping_id=None, staging_name=None, connector_name=None, connector_id=None, reverse_mapping=False, process_cds=True)

Start mapping.

Args:

entity_mapping_id: str or list of strings

Mapping ids to be resumed.

staging_name:

Staging name for starting the mapping

connector_name: str, str, default None

Connector Name

connector_id: str, str, default None

Connector ID

reverse_mapping: bool default False

When using with consumer.False if you don’t know what consumer is.

process_cds: bool default True

Process pending records after play.

Returns: dict

Dictionary with the response of all mappings played.

staging_to_connectors_map()

Create a dictionary where the mapping of connectors and stagings.

Returns: dict

Dict

stats(connector_id=None, connector_name=None, all_connectors=False)

Get connector stats

Args:

connector_id: str, default None

Connector Id

connector_name: str, default None

Connectot name

all_connectors: bool default False

Get from all connectors.

Returns: dict

Dict with the status of the connectors.