info@lasmart.biz

may 18, 2023

Deploying Kylin cubes from the script

Recently, open-source software has been gaining in popularity and active use. Our company has followed the main trend and started to master the new Apache technology stack. After several successful projects we want to share our development experience and some features.

The metadata of analytics solutions are frequently similar. Apache Kylin provides the ability to move or edit the metadata of different objects using the command line. This speeds up the development process. 

Here below are the steps for deploying a cube from a script (Listing 1):

1) Make a backup of the metadata;

2) Create a folder for further metadata recovery (e.g., my_restore);

3) Create there folders project, cube, cube_desc.

4) Copy the file project/name_project.json into the directory my_restore/project.

5) In /my_restore/project/name_project.json, write the name of the new cube.

6) Copy the files of the cube you want to change/expand into cube and cube_desc directory with the new name of the cube in the file name.

7) In cube/name_new_cube.json set a new unique name (corresponding to the file name) and UUID.vi

8) In cube_desc/name_new_cube.json set a new unique name (corresponding to the file name) and UUID, and at the end of the file signature (signature), which should always end with ==

9) Retrieve metadata from my_restore. This will add the metadata to the existing metadata.

Listing 1 — script for automating the processing of Kylin cubes sections

$KYLIN_HOME/bin/metastore.sh backup mkdir /my_restore mkdir /my_restore/cube mkdir /my_restore/project mkdir /my_restore/cube_desc cp meta_backups/meta_2023_01_10_20_24_50/project/name_project.json /my_restore/project vi /my_restore/project/name_project.json cp meta_backups/meta_2023_01_10_20_24_50/cube/name_cube.json /my_restore/cube/name_new_cube.json cp meta_backups/meta_2023_01_10_20_24_50/cube_desc/name_cube.json /my_restore/cube_desc/name_new_cube.json vi /my_restore/cube/name_new_cube.json vi /my_restore/cube_desc/name_new_cube.json $KYLIN_HOME/bin/metastore.sh restore /my_restore $KYLIN_HOME/bin/metastore.sh list /cube


 

10) Then update the metadata in the System tab of the Kylin web interface.

11) The cube will display with an error, to fix it you need to click next at each step in the cube editor to assign the correct signature.

Source List:

https://kylin.apache.org/docs31/howto/howto_backup_metadata.html

For more information, please get in touch here:

Contact Form main