Contents x
    API v1 - Queue
      • Dark
        Light
      Contents

      API v1 - Queue

        • Dark
          Light
        Article Summary

        Overview

        The queue endpoint allows Queues to be managed externally, similarly to how they are managed within the Matillion ETL client via Project → Manage SQS Configuration or Manage Pub/Sub Configuration. Queue configuration can then be exported, imported and tested using the API v1

        Important Information

        • This document is part of a series target="blank">API Profiles and the Matillion ETL v1 API
        • This process requires the Matillion ETL instance URL, and the username and password of a user with appropraite permissions.

        Queue API Endpoints

        Queue API is available on standard REST-based APIs that uses HTTP or HTTPS request to GET and POST data. The Queue API service is accessed through the Uniform Resource Identifier (URI). All following references in this document will assume the API Base URL has been specified. The available API endpoints are listed below:

        FunctionMethodPathURL
        Export Queue configurationsGET/exporthttps://<InstanceURI>/rest/v1/queue/export
        Import Queue configurationsPOST/importhttps://<InstanceURI>/rest/v1/queue/import
        Test Queue configurationsGET/testhttps://<InstanceURI>/rest/v1/queue/test

        Please Note

        • The username and password of a user with access to the Queue within Matillion ETL instance will need to be used to authenticate the connection. These may need to be specified before the command when using a command line tool—for example: -u user:password.
        • When exporting or importing Queue configurations, the source file or content type and target file will also need to be specified.
        • Matillion ETL can currently only use Queue configurations formatted in JSON.

        Error

        While manually editing an imported Queue configuration file in JSON is possible, it is not recommended.

        API Parameters and Description

        Below is the list of endpoint parameters and their brief description:

        ParameterDescription
        <InstanceURI>The server IP address or domain name of the Matillion ETL instance
        <export>This allows the API to export the Queue configuration to another Matillion ETL instance
        <import>This allows the API to import the Queue configuration from another Matillion ETL instance
        <test>This allows the API to test the validity of the Queue configuration settings

        Endpoints and Server Response

        These APIs provide a REST-based web service—offering ease of use and a flexible choice of programming language—and can be used to access and analyse data and service provider content. Below are a number of example endpoints along with the corresponding summary, parameters and server responses:

        GET/export

        This endpoint permits the API to export the Queue configurations from the Matillion ETL instance.

        • Base URL
          https://<InstanceURI>/rest/v1/queue/export
        • Server Response
          {
"credentialsName": "Instance Credentials",
"successEnabled": true,
"compressSuccess": false,
"failureEnabled": true,
"compressFailure": false,
"region": "eu-west-1",
"sqsEnabled": true,
"listenQueue": "LISTEN_QUEUE",
"successQueue": "SUCCESS_QUEUE",
"failureQueue": "FAIL_QUEUE"
}

        POST/import

        This endpoint permits the API to import Queue configurations into the Matillion ETL instance. This will be a "POST" method API call as it will have to attach the project, in the body as a JSON file to import into the Matillion ETL instance.

        • Base URL
          https://<InstanceURI>/rest/v1/queue/import
        • POST Body (JSON)
          {
"credentialsName": "Instance Credentials",
"successEnabled": true,
"compressSuccess": false,
"failureEnabled": true,
"compressFailure": false,
"region": "eu-west-1",
"sqsEnabled": true,
"listenQueue": "LISTEN_QUEUE",
"successQueue": "SUCCESS_QUEUE",
"failureQueue": "FAIL_QUEUE"
}

          Below is the description of the fields included in the POST body:

          Field nameData typeDescription
          credentialsNameStringThe credentials used to talk to the queue.
          successEnabledBooleanA message will be placed on a queue when an orchestration job has completed if true.
          compressSuccessBooleanThe body of the "completed" message on the queue will be compressed if true.
          failureEnabledBooleanA message will be placed on a queue when an orchestration job has failed if true.
          compressFailureBooleanThe body of the "failed" message on the queue will be compressed if true.
          regionStringThe region in which the Queue exists.
          sqsEnabledBooleanThe sub-system that listens to queues is turned on if true.
          listenQueueStringThe name of the queue to listen for messages.
          successQueueStringThe name of the Queue on which "completed" messages will be placed.
          failQueueStringThe name of the Queue on which "failed" messages will be placed.
        • Server Response
          {
"success": true,
"msg": "Successfully imported queue.",
"id": 0
}

        GET/test

        This endpoint permits the API to test the Queue configurations on the Matillion ETL instance.

        • Base URL
          https://<InstanceURI>/rest/v1/queue/test
        • Server Response
          {
"success": true,
"msg": "Queue test was successful.",
"id": 0
}
        What's Next