february 13, 2025
Installing Arenadata components on a cluster of nodes running Ubuntu 22.04
Our first experience
Our first experience with Arenadata was very positive. The installation of the components went smoothly without significant issues, thanks to well-documented processes and the user-friendly ADCM interface. Despite initial concerns about setting up host providers, SSH connections worked on the first attempt after correctly configuring the keys. We were particularly impressed by the ease of managing ADB and ADQM clusters, as well as the flexibility in service configuration.
The only issues we encountered were with setting up access to NiFi, but these were quickly resolved using Nginx. As a result, we obtained a stable and scalable platform for working with big data, which significantly accelerated our internal processes.
Configuring ADCM
1. Installing Docker on the ADCM management machine. All commands will be executed as the root user.
2. Installing ADCM — offline method via installation script. Upload the installation script (.sh) to the server and execute it.
3. After executing the script, a container with a web interface will be created, accessible at:
http://your_ip:8000/login
Default login credentials:
- Username: admin
- Password: admin
4. Changing the administrator password.
After the first login, go to /profile and change the administrator password to enhance security.
5. Configuring the ADCM panel.
In the settings, specify the ADCM panel URL to ensure the correct operation of all components.
For the current installation, the ADCM’s URL key should be set to http://your_ip:8000, then click the Save button.
Configuring host providers
SSH will be used to manage cluster nodes.
1. Uploading the SSH Bundle.
Go to the /bundles section in the ADCM interface and upload the appropriate SSH bundle.
2. Creating a host provider.
After uploading the bundle:
- Go to the /hostproviders section.
- Click Create hostprovider.
3. Configuring the host provider.
4. Go to the created provider, navigate to its configuration, and fill in the SSH private key field. This key will be used to connect to the cluster nodes.
For example, the key can be taken from the adcm container that was previously launched in Docker. To view it, run the following command on the main adcm node:
Alternatively, you can use any other previously generated key. Then, click Save to save the configuration.
Preparing and adding worker nodes
1. Configuring hostnames.
Ensure that the hostnames resolve to IP addresses using DNS or the /etc/hosts file, as this is critical for some components.
Example configuration for resolving via the file:
2. Configuring SSH access.
Add the public key of your host provider to the /root/.ssh/authorized_keys file on each node:
3. Adding nodes to ADCM.
Go to the /hosts section and click Create host to add each worker node.
4. Checking node availability.
After adding the nodes, verify their availability through the Actions menu in the ADCM interface.
Creating an ADB cluster
We will deploy the ADB cluster in a minimal configuration (1 master and 2 workers).
1. Uploading the ADB bundle.
Go to the /bundles section and upload the ADB bundle.
2. Creating the cluster.
Based on the uploaded bundle, create a new cluster:
- Go to /clusters.
- Click Create cluster.
3. Adding services to the cluster.
Go to the Services section and add the required services to the cluster (the minimum configuration includes one master and two workers).
4. Mapping services to nodes.
In the Mapping section, assign services to the corresponding nodes.
5. Installing services.
Go to the Services section, open the Actions menu, and select Install. The installation status will be displayed in the interface.
6. Managing the cluster.
The cluster can be managed via the Actions menu in ADCM or using the gpadmin user on the ADB master node.
7. Configuring password authentication.
Log in to the master node as the gpadmin user and connect to the database:
- Creating a user for authentication:
- Allowing Connections from Any IP.
Add the following entry to pg_hba.conf:
- Restarting the database:
Installing ADQM
The installation process for ADQM is similar to ADB:
- Upload the ADQM bundle.
- Create a cluster.
- Add the required service.
- Configure node mapping.
- Install the services.
Configuring access to ADQM
By default, ADQM is only accessible from localhost. To allow connections from any location:
1. Edit the file /etc/clickhouse-server/users.xml and add the following:
2. Specify the hashed password for the default user in the password_sha256_hex section in /etc/clickhouse-server/users.d/credentials.xml.
3. Restart the server:
Installing ADS
The installation process for ADS is similar to ADB and ADQM:
- Upload the ADS bundle.
- Create a cluster.
- Add the required services.
- Configure node mapping.
- Install the services.
Notes on NiFi Installation
By default, NiFi in ADCM assumes authentication via AD/LDAP. If these components are not available, access to NiFi will remain open.
One way to restrict access is to use HTTP authentication via Nginx.
1. Restricting access to NiFi via local interface.
Ensure that the node name corresponds to the local interface in /etc/hosts:
Restart NiFi via ADCM to apply the settings.
2. Installing and configuring Nginx:
3. Create a configuration file for Nginx:
4. Create a .htpasswd file to protect access:
5. Restart Nginx:
Now, access to NiFi will be protected by a login and password at:
http://your_ip:8081
