API v1 - API Extract Profile

Dark
Light

API v1 - API Extract Profile

Dark
Light

Article Summary

Share feedback

Thanks for sharing your feedback!

Overview

The API Extract Profile endpoint allows API to extract the Profiles to be managed externally, similarly to how they are managed within Matillion ETL. To begin, click Project → Manage API Profiles → Manage Extract Profiles and the RSD files contained within the m—can then be exported and imported via the API Profile endpoint.

An Extract Profile is an API Profile that will invoke an external API and return unflattened or nested data in the form of a single-column JSON structure.

Important Information

This document is part of a series target="blank">API Extract Profiles and the Matillion ETL v1 API.
This process requires the Matillion ETL instance URL; the user's username and password, with appropriate permissions; and the name of the API Extract Profile and/or RSD file to be exported or imported.

API Extract Profile Endpoints

API Base URL

http(s)://<InstanceAddress>/rest/v1/<apiextractprofile>

API Endpoints and Function

The API Extract Profile is available on standard REST-based APIs that use a HTTP or HTTPS request to GET and POST data. The API Profile API service is accessed through the Uniform Resource Identifier (URI). All following references in this document will assume the API Base URL has been specified. The available API endpoints are listed below:

Method	Path	URL	Function
GET	/apiextractprofile	https://<InstanceAddress>/rest/v1/apiextractprofile	Get a list of API Extract profile connectors.
GET	/export	https://<InstanceAddress>/rest/v1/apiextractprofile/export	Export defined API extract connectors and details.
POST	/import	https://<InstanceAddress>/rest/v1/apiextractprofile/import	Import one or more API connector extracts.
GET	/name/<connectorName>/export	https://<InstanceAddress>/rest/v1/apiextractprofile/name/<ConnectorName>/export	Export the selected connector API extract.

Graphical Representation

To illustrate the API extract profile endpoints further, below is the graphical flow of the /apiprofile endpoint showing possible methods GET and POST .

Please Note

The username and password of a user with access to the API extract profiles within a Matillion ETL instance will need to be used to authenticate the connection. These may need to be specified before the command when using a command-line tool—for example: -u user:password.
When exporting or importing API extract profiles or RSD files, the source file or content type and target file will also need to be specified.
There is no API method for specifically exporting or importing all RSD files in an API extract profile, as this can be done when exporting or importing the API extract profile itself.
Matillion ETL can currently only use API extract profiles that are formatted in JSON.

API Parameters and Description

Below is the list of endpoint parameters and their brief description:

Parameter	Description
<InstanceAddress>	The server IP address or domain name of the Matillion ETL instance.
<name>	This allows an API extract profile or RSD file to be specified by the name used in the Matillion ETL instance.
<ConnectorName>	The name of the connector associated with the API extract profile
<export>	This allows the API to export the configuration to another Matillion ETL instance.
<import>	This allows the API to import the configuration from another Matillion ETL instance.

Endpoints and Server Response

These APIs provide a REST-based web service, offering ease of use and a flexible choice of programming language, and can be used to access and analyse data and service provider content. Below are a number of example endpoints along with the corresponding summary, parameters, and server responses:

The Extract Profile API family comprises of following PATHs and endpoints:

GET/apiextractprofile
GET/apiextractprofile/export
POST/apiextractprofile/import
GET/name/<connectorName>/export

The detailed description of each endpoint and associated methods of each PATH is discussed in the next section of this guide.

GET/apiextractprofile

This endpoint permits the API to list all the API extract profiles from the Matillion ETL instance.

Base URL

https://<InstanceAddress>/rest/v1/apiextractprofiles

Server Response
```
[
"API Extract",
]
```

GET/apiextractprofile/export

This endpoint permits the API to export one or more API Profiles, and/or their RSD files from the Matillion ETL instance.

Base URL

https://<InstanceAddress>/rest/v1/apiextractprofile/export

https://<InstanceAddress>/rest/v1/apiextractprofile/name/<apiextractprofile>/export

Server Response

{
"objects": [
{
"connectorName": "example",
"endpoints": [
{
"endpointName": "abc",
"connectorName": "Example",
"description": "",
"endpointUri": "https://api.example.com/v1/bpi/currentprice.json",
"httpRequestType": "GET",
"pagingEnabled": false,
"pagingType": null,
"endpointSchema": "{\\"schema\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"chartName\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"bpi\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"EUR\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"symbol\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate_float\\":{\\"precision\\":8,\\"scale\\":3,\\"type\\":\\"DOUBLE PRECISION\\"},\\"code\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"description\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}},\\"GBP\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"symbol\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate_float\\":{\\"precision\\":9,\\"scale\\":4,\\"type\\":\\"DOUBLE PRECISION\\"},\\"code\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"description\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}},\\"USD\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"symbol\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate_float\\":{\\"precision\\":9,\\"scale\\":4,\\"type\\":\\"DOUBLE PRECISION\\"},\\"code\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"description\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}}}},\\"time\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"updateduk\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"updatedISO\\":{\\"type\\":\\"TIMESTAMP\\"},\\"updated\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}},\\"disclaimer\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}}}",
"schemaType": "UNFLATTENED_NO_REPEAT_ELEMENT",
"authEnabled": false,
"authType": "",
"authScheme": "",
"pagingSizeParameter": null,
"pagingSize": null,
"nextAbsoluteUriDataKey": null,
"pagingOffsetParameter": null,
"pagingTotalItemsParameter": null,
"params": []
}
]
},
],
"version": "1.48.7",
"environment": "redshift"
}

POST/import

This endpoint permits the API to import one or more API Profiles, and/or their RSD files into the Matillion ETL instance. This will be a "POST" method API call, as it will have to attach the project in the body as a JSON file to import into the Matillion ETL instance.

Base URL

https://<InstanceAddress>/rest/v1/apiextractprofile/import

POST Body (JSON)

{
"objects": [
{
"connectorName": "example",
"endpoints": [ ]
}
],
"version": "1.48.7",
"environment": "redshift"
}

Server Response

{
"name": "API Extracts",
"statusList": [
{
"success": true,
"name": "example"
}
],
"success": true
}

GET/apiextractprofile/name/<connectorName>/export

This endpoint permits the API to export one or more API Profiles, and/or their RSD files from the Matillion ETL instance.

Base URL

https://<InstanceAddress>/rest/v1/apiextractprofile/name/<connectorName>/export

Server Response

{
"objects": [
{
"connectorName": "example",
"endpoints": [
{
"endpointName": "abc",
"connectorName": "example",
"description": "",
"endpointUri": "https://api.example.com/v1/bpi/currentprice.json",
"httpRequestType": "GET",
"pagingEnabled": false,
"pagingType": null,
"endpointSchema": "{\\"schema\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"chartName\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"bpi\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"EUR\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"symbol\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate_float\\":{\\"precision\\":8,\\"scale\\":3,\\"type\\":\\"DOUBLE PRECISION\\"},\\"code\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"description\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}},\\"GBP\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"symbol\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate_float\\":{\\"precision\\":9,\\"scale\\":4,\\"type\\":\\"DOUBLE PRECISION\\"},\\"code\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"description\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}},\\"USD\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"symbol\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate_float\\":{\\"precision\\":9,\\"scale\\":4,\\"type\\":\\"DOUBLE PRECISION\\"},\\"code\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"rate\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"description\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}}}},\\"time\\":{\\"type\\":\\"struct\\",\\"objectcontents\\":{\\"updateduk\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"},\\"updatedISO\\":{\\"type\\":\\"TIMESTAMP\\"},\\"updated\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}},\\"disclaimer\\":{\\"size\\":2000,\\"type\\":\\"VARCHAR\\"}}}}",
"schemaType": "UNFLATTENED_NO_REPEAT_ELEMENT",
"authEnabled": false,
"authType": "",
"authScheme": "",
"pagingSizeParameter": null,
"pagingSize": null,
"nextAbsoluteUriDataKey": null,
"pagingOffsetParameter": null,
"pagingTotalItemsParameter": null,
"params": []
}
]
}
],
"version": "1.48.7",
"environment": "redshift"
}

What's Next

API v1 - API Profile

Table of contents

Overview
API Extract Profile Endpoints
API Parameters and Description
Endpoints and Server Response