Integrating Big Data Platforms with Zaloni: REST API

February 1, 2017 Adam Diaz

In conversations about the Zaloni Data Lake Management Platform, the most common question we hear is “Can it integrate with product X?” which is almost immediately followed by “Can I use it without the UI?”. The answer to both is “Yes.” And it’s all made possible by the robust REST API that is built into our Platform.

Screen Shot 2017-02-01 at 12.17.29 PM.png

This image displays an example of the interactive API documentation that is built on Swagger. Although classic documentation formats are available via PDF, you’ll want to check this out. The interactive documentation expedites the learning curve for making basic calls and crafting the correct input syntax for the API you are trying to use.

Understanding API Calls Through the Command Line

For example, you may want to understand variables in the Platform Namespace. Much like environmental variables in a Linux shell, Zaloni has a Namespace used to pass variables to different parts of the product. The command line below is a great example of what this will look like.

[bedrock@hostname]$ curl -D token.json -H "Content-Type: application/json" \
-X POST -d '{"username":"YYYYY","password":"XXXXX"}' \
 
[bedrock@hostname]$ JSESSIONID=$(cat /path/to/token/token.json | grep
JSESSIONID: | cut -d' ' -f 2)
 
[bedrock@hostname]$ curl -H "JSESSIONID: $JSESSIONID" -X POST --header
'Content-Type: application/json' --header 'Accept: application/json' -d '{}'
'http://localhost:8080/bedrock-app/services/rest/workflows/search?projectIds=9&
api_key=saveEntityInstance'
 
{
 "responseMessage": "success",
 "restUri": null,
 "result": {
   "currentPage": 1,
   "totalRecords": 2,
   "workFlowDetails": [
     {
 
          "CSVValues":
"59,Automated_Ingest_DQ_TM_noWM,unassigned,admin,11/18/2016
12:36:37,admin,11/22/2016 17:19:51",
       "category": "unassigned",
       "createdBy": "admin",
       "createdDate": "11/18/2016 12:36:37",
       "modifiedBy": "admin",
       "modifiedDate": "11/22/2016 17:19:51",
       "wfId": 59,
       "wfName": "Automated_Ingest_DQ_TM_noWM"
     },
     {
       "CSVValues": "62,Automated_Ingest_DQ_TM_TT,unassigned,admin,11/18/2016
16:22:33,admin,11/21/2016 21:46:07",
       "category": "unassigned",
       "createdBy": "admin",
       "createdDate": "11/18/2016 16:22:33",
       "modifiedBy": "admin",
       "modifiedDate": "11/21/2016 21:46:07",
       "wfId": 62,
       "wfName": "Automated_Ingest_DQ_TM_TT"
     }
   ]
 },

"page": null
}

For project ID 9 (aka Trouble Tickets in this system) there are two known workflows with IDs 59 and 62.

Practicing API Calls with the Swagger UI

Most users need some assistance when using a new API. It’s really helpful to have a way to practice iterating over the creation of a correct call before putting it into your process. In the interactive Swagger documentation, that same call would look something like the diagram below:

Screen Shot 2017-02-01 at 12.26.11 PM.png
Under the workflow page we navigate to the search API and then fill in the correct values. In this case passing just a project ID is enough. This corresponds to the concept of projects in the Zaloni Platform. These can be listed out via the “Administration” menu under “projects” or via another API call to obtain all project IDs.

curl -H "JSESSIONID: $JSESSIONID" -X GET --header 'Accept: application/json'
 

RESPONSE BODY TRUNCATED:

{
 "responseMessage": "SUCCESS",
 "restUri": "/projects",
 "result": [
   {
     "projectId": 10,
     "projectName": "Taxi Data",
     "createdBy": "admin",
     "createdDateTime": 1479150403000,
     "canManageUserRoles": true,
     "isDefaultProject": false,
     "roleNames": [
       "Project Administrator"
     ]
   },
   {
     "projectId": 9,
     "projectName": "Trouble Tickets",
     "createdBy": "admin",
     "createdDateTime": 1478618016000, …

 

Screen Shot 2017-02-01 at 12.28.26 PM.png

The final screen will show a response body along with the response header and response code (not shown here). This is a quick and easy way to interrogate and learn the REST API. There are actions for controlling the entire system. In fact, this is how the web interface for our Platform operates.

If you have applications you wish to integrate with the Zaloni Data Lake Management Platform via REST API know that this is completely possible. This can be done by our staff here at Zaloni or by your staff directly. We also offer training on all aspects of the platform use including developer training. Please contact us for more information.

About the Author

Adam Diaz

Big data & Hadoop thought-leader

More Content by Adam Diaz
Previous Article
Key Challenges of Building Next-Generation Data Architectures
Key Challenges of Building Next-Generation Data Architectures

Today’s technology and software advances allow us to process and analyze huge amounts of data. While it’s c...

Next Article
Governed Data Lakes Deliver Exceptional Value and Accessibility
Governed Data Lakes Deliver Exceptional Value and Accessibility

When enterprises hit a wall with their data lake, typically it's because they didn't build their architectu...

Want an agile, governed data lake?

Contact Us