tap-postgres - Meltano Hub

PostgreSQL

tap-postgres from transferwise

PostgreSQL database extractor

The tap-postgres Meltano extractor pulls data from PostgreSQL that can then be sent to a destination using a loader.

Other Available Variants

singer-io
transferwise (default)

Getting Started

Prerequisites

If you haven't already, follow the initial steps of the Getting Started guide:

Installation and configuration

Add the tap-postgres extractor to your project using
```
meltano add
```
:

meltano add extractor tap-postgres

Configure the tap-postgres settings using
```
meltano config
```
:

meltano config tap-postgres set --interactive

Test that extractor settings are valid using

meltano test

meltano test tap-postgres

Next steps

Follow the remaining steps of the Getting Started guide:

If you run into any issues, learn how to get help.

Troubleshooting

Error: `pg_config executable not found` or `libpq-fe.h: No such file or directory`

This error message indicates that the libpq dependency is missing.

To resolve this, refer to the "Dependencies" section above.

Capabilities

The current capabilities for

tap-postgres

may have been automatically set when originally added to the Hub. Please review the capabilities when using this extractor. If you find they are out of date, please consider updating them by making a pull request to the YAML file that defines the capabilities for this extractor.

This plugin has the following capabilities:

catalog
discover
state

You can override these capabilities or specify additional ones in your meltano.yml by adding the capabilities key.

Settings

The tap-postgres settings that are known to Meltano are documented below. To quickly find the setting you're looking for, click on any setting name from the list:

host
port
user
password
dbname
ssl
filter_schemas
default_replication_method
max_run_seconds
logical_poll_total_seconds
break_at_end_lsn

You can override these settings or specify additional ones in your meltano.yml by adding the settings key.

Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this plugin.

Host (host)

Environment variable: TAP_POSTGRES_HOST
Default Value: localhost

PostgreSQL host

Port (port)

Environment variable: TAP_POSTGRES_PORT
Default Value: 5432

PostgreSQL port

User (user)

Environment variable: TAP_POSTGRES_USER

PostgreSQL user

Password (password)

Environment variable: TAP_POSTGRES_PASSWORD

PostgreSQL password

Database Name (dbname)

Environment variable: TAP_POSTGRES_DBNAME

PostgreSQL database name

SSL (ssl)

Environment variable: TAP_POSTGRES_SSL
Default Value: false

Using SSL via postgres sslmode='require' option. If the server does not accept SSL connections or the client certificate is not recognized the connection will fail

Filter Schemas (filter_schemas)

Environment variable: TAP_POSTGRES_FILTER_SCHEMAS

Scan only the specified comma-separated schemas to improve the performance of data extraction

Default Replication Method (default_replication_method)

Environment variable: TAP_POSTGRES_DEFAULT_REPLICATION_METHOD

[No description provided.]

Max Run Seconds (max_run_seconds)

Environment variable: TAP_POSTGRES_MAX_RUN_SECONDS
Default Value: 43200

Stop running the tap after certain number of seconds

Logical Poll Total Seconds (logical_poll_total_seconds)

Environment variable: TAP_POSTGRES_LOGICAL_POLL_TOTAL_SECONDS
Default Value: 10800

Stop running the tap when no data received from wal after certain number of seconds

Break At End LSN (break_at_end_lsn)

Environment variable: TAP_POSTGRES_BREAK_AT_END_LSN
Default Value: true

Stop running the tap if the newly received lsn is after the max lsn that was detected when the tap started

Something missing?

This page is generated from a YAML file that you can contribute changes to.

Edit it on GitHub!

Looking for help?

If you're having trouble getting the tap-postgres extractor to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the