 | REST API
REST API |
The tap-rest-api-msdk Meltano extractor pulls data from REST API that can then be sent to a destination using a loader.
Other Available Variants
- widen (default)
Getting Started
Prerequisites
If you haven't already, follow the initial steps of the Getting Started guide:
Installation and configuration
-
Add the tap-rest-api-msdk extractor to your project
using
:meltano add -
Configure the tap-rest-api-msdk settings using
:meltano config -
Test that extractor settings are valid using
:meltano test
meltano add extractor tap-rest-api-msdkmeltano config tap-rest-api-msdk set --interactivemeltano test tap-rest-api-msdkNext steps
Follow the remaining steps of the Getting Started guide:
If you run into any issues, learn how to get help.
Capabilities
The current capabilities fortap-rest-api-msdkThis plugin has the following capabilities:
- state
- catalog
- discover
- about
- stream-maps
You can
override these capabilities or specify additional ones
in your meltano.yml by adding the capabilities key.
Settings
The
tap-rest-api-msdk settings that are known to Meltano are documented below. To quickly
find the setting you're looking for, click on any setting name from the list:
You can
override these settings or specify additional ones
in your meltano.yml by adding the settings key.
Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this plugin.
Name (name)
-
Environment variable:
TAP_REST_API_MSDK_NAME
(required) name of the stream.
API Base URL (api_url)
-
Environment variable:
TAP_REST_API_MSDK_API_URL
(required) the base url/endpoint for the desired api.
Path (path)
-
Environment variable:
TAP_REST_API_MSDK_PATH
(optional) the path appeneded to the api_url.
Params (params)
-
Environment variable:
TAP_REST_API_MSDK_PARAMS
(optional) an object of objects that provide the params in a requests.get method.
Headers (headers)
-
Environment variable:
TAP_REST_API_MSDK_HEADERS
(optional) an object of headers to pass into the api calls.
Records Path (records_path)
-
Environment variable:
TAP_REST_API_MSDK_RECORDS_PATH
(optional) a jsonpath string representing the path in the requests response that contains the records to process. Defaults to $[*].
Primary Keys (primary_keys)
-
Environment variable:
TAP_REST_API_MSDK_PRIMARY_KEYS
(required) a list of the json keys of the primary key for the stream.
Replication Key (replication_key)
-
Environment variable:
TAP_REST_API_MSDK_REPLICATION_KEY
(optional) the json key of the replication key. Note that this should be an incrementing integer or datetime object.
Except Keys (except_keys)
-
Environment variable:
TAP_REST_API_MSDK_EXCEPT_KEYS
This tap automatically flattens the entire json structure and builds keys based on the corresponding paths. Keys, whether composite or otherwise, listed in this dictionary will not be recursively flattened, but instead their values will be turned into a json string and processed in that format. This is also automatically done for any lists within the records; therefore, records are not duplicated for each item in lists.
Number Inference Records (num_inference_records)
-
Environment variable:
TAP_REST_API_MSDK_NUM_INFERENCE_RECORDS
(optional) number of records used to infer the stream's schema. Defaults to 50.
Something missing?
This page is generated from a YAML file that you can contribute changes to.
Edit it on GitHub!Looking for help?
#plugins-generalInstall
meltano add extractor tap-rest-api-msdk
Maintenance Status
-brightgreen){kind=icon}
Maintainer
Meltano Stats
-334-c0c0c4){kind=icon}
-25-c0c0c4){kind=icon}
Keywords
