february 10, 2023
The Interaction with "Apache Kylin" via RestAPI
Glossary
| Source | Description |
|---|---|
|
API (Application Programming Interface) |
A group of methods that implement client-side and server-side interaction |
|
Apache Kylin |
Multi-component open-source toolkit for business intelligence and BigData |
|
GET, POST, PUT |
Types of API requests |
|
JSON |
Data format type; used commonly when working with APIs |
|
Base64
|
Coding, used in the Basic authorisation method*** ***An unsafe authorisation method. In fact, the username and password are transmitted in clear text. If possible, other options should be developed |
Overview
Using Apache Kylin as a component of complex business intelligence solution requires the ability to quickly get information about the current status of the OLAP cube, regulatory tasks, as well as to implement routine updates by ETL-tool. To solve these tasks the most convenient tool is the use of API tools.
At present, Apache NiFi with the ability to run PowerShell scripts through ExecuteProcess / ExecuteStreamCommand tasks seems promising for the automation task. The development of a block of universal scripts is beyond the scope of this article; the examples used are parametrized manually, in the script code.
This article looks at RestAPI Apache Kylin using two methods as examples — getting information about the status of a job and running BUILD on a cube.
Stack used
Apache Kylin on remote server with «ABC» cube deployed
-
Windows PowerShell + Visual Studio Code (debugging)
-
Postman
-
Apache NiFi (ETL)
Authentication
The API method used for authentication:
POST http://localhost:7070/kylin/api/user/authentication
The data for user authentication in the API is passed in the header part of the request in clear text (base64 encoding) and is a concatenate string code of the form:
Authorization: Basic base64(login:password),
where base64() is a base64-encoded string.
Sample header part of request for user authentication:
Authorization:Basic YWRtXXXXXXXXXXXXXXXZA==
Content-Type:application/json
The need for a separate authentication request has not been established at this time.
Method call from the Postman environment
PowerShell script:
Process (BUILD) the cube
OLAP cube processing is a mandatory routine task and must be performed after each update of the repository with new data.
API method used for BUILD:
PUT http://localhost:7070/kylin/api/cubes/названиеКуба’cubename’/build
The response from the server is in JSON format.
The header part of the request must contain parameters:
-
Authorisation (in Postman, filled in automatically from the relevant tab)
-
Content-type: application/json
The other header parameters listed are optional.
The body part of the request must contain parameters in JSON format:
-
startTime (in YYYYYMMDDHHMMSS format) - mark the start date/time for processing
-
endTime (in the format YYYYYMMDDHMMSS) - mark the end date-time for processing
-
buildType (supported - 'BUILD', 'MERGE', 'REFRESH') - type of cube processing
Setting up Postman to call a method
In the event of successful method call, the response will return information about GUID of job launch, date-time parameters («segment» of cube), commands for server part, status of job (usually PENDING at the time of request execution). You can also track the success of the operation in the Kylin web interface — the «Monitoring» tab will display a record of the operation performed on the cube.
PowerShell script:
Obtaining data on the status of a job
When a cube processing is executed, the API returns a JSON containing information about the current status of the job. Knowing the GUID of a particular job, it is possible to track its current status at the time of the request (in particular, the progress of execution and the execution/error mark).
API method to get the current status of a job:
GET http://localhost:7070/kylin/api/jobs/{jobId}
{jobID} as identificator of job «e50b3a7c-18b4-4660-ac5d-c21213e55d78» (Where {jobID} is the Job ID of the form «e50b3a7c-18b4-4660-ac5d-c21213e55d78»)
The response from the server in JSON format.
Setting up Postman to call a method
PowerShell script:
Sources and references
