Skip to content

Commit 37d4b8f

Browse files
rrelledgerrelledge
authored
DOC-6083 RS: Add user-defined modules during bootstrapping (#2526)
1 parent ea873d3 commit 37d4b8f

File tree

5 files changed

+294
-3
lines changed

5 files changed

+294
-3
lines changed

‎content/operate/oss_and_stack/stack-with-enterprise/install/_index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ linkTitle: Install and upgrade modules
1111
weight: 4
1212
---
1313

14-
Several modules that provide additional Redis capabilities, such as search and query, JSON, time series, and probabilistic data structures, come packaged with [Redis Enterprise Software]({{< relref "/operate/rs" >}}). As of version 8.0, Redis Enterprise Software includes four feature sets, compatible with different Redis database versions.
14+
Several modules that provide additional Redis capabilities, such as search and query, JSON, time series, and probabilistic data structures, come packaged with [Redis Enterprise Software]({{< relref "/operate/rs" >}}). As of version 8.0, Redis Enterprise Software includes multiple feature sets, compatible with different Redis database versions.
1515

1616
However, if you want to use additional modules or upgrade a module to a more recent version, you need to:
1717

‎content/operate/oss_and_stack/stack-with-enterprise/install/add-module-to-cluster.md‎

Lines changed: 246 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,14 +10,28 @@ linkTitle: Install on a cluster
1010
weight: 10
1111
---
1212

13-
[Redis Enterprise Software]({{< relref "/operate/rs" >}}) comes packaged with several modules that provide additional Redis capabilities such as [search and query]({{<relref "/operate/oss_and_stack/stack-with-enterprise/search">}}), [JSON]({{<relref "/operate/oss_and_stack/stack-with-enterprise/json">}}), [time series]({{<relref "/operate/oss_and_stack/stack-with-enterprise/timeseries">}}), and [probabilistic data structures]({{<relref "/operate/oss_and_stack/stack-with-enterprise/bloom">}}). As of version 8.0, Redis Enterprise Software includes four feature sets, compatible with different Redis database versions. You can view the installed modules, their versions, and their minimum compatible Redis database versions from **Cluster > Modules** in the Cluster Manager UI.
13+
[Redis Enterprise Software]({{< relref "/operate/rs" >}}) comes packaged with several modules that provide additional Redis capabilities such as [search and query]({{<relref "/operate/oss_and_stack/stack-with-enterprise/search">}}), [JSON]({{<relref "/operate/oss_and_stack/stack-with-enterprise/json">}}), [time series]({{<relref "/operate/oss_and_stack/stack-with-enterprise/timeseries">}}), and [probabilistic data structures]({{<relref "/operate/oss_and_stack/stack-with-enterprise/bloom">}}). As of version 8.0, Redis Enterprise Software includes multiple feature sets, compatible with different Redis database versions. You can view the installed modules, their versions, and their minimum compatible Redis database versions from **Cluster > Modules** in the Cluster Manager UI.
1414

1515
To use other modules or upgrade an existing module to a more recent version, you need to install the new module package on your cluster.
1616

1717
{{<warning>}}
1818
Some module versions are not supported or recommended for use with Redis Enterprise Software.
1919
{{</warning>}}
2020

21+
## Module package requirements
22+
23+
The module must be packaged as a `.zip` file containing:
24+
25+
- **module.json**: A metadata file with module information including:
26+
- `module_name`: The actual module name
27+
- `version`: Numeric version
28+
- `semantic_version`: Semantic version string (for example, "1.0.0")
29+
- `min_redis_version`: Minimum compatible Redis version
30+
- `commands`: List of commands the module provides
31+
- `capabilities`: List of module capabilities
32+
33+
- **Module binary**: The compiled `.so` file for the target platform
34+
2135
## Get packaged modules
2236

2337
To install or upgrade a module on a [Redis Enterprise Software]({{< relref "/operate/rs" >}}) cluster, you need a module package.
@@ -26,6 +40,237 @@ To install or upgrade a module on a [Redis Enterprise Software]({{< relref "/ope
2640

2741
- For custom-packaged modules, download a [custom-packaged module](https://redislabs.com/community/redis-modules-hub/) from the developer.
2842

43+
- User-defined modules are downloaded automatically if you [add them during bootstrapping](#bootstrap-user-defined-module).
44+
45+
## Add user-defined modules during bootstrapping (Redis Software v8.0.6 and later) {#bootstrap-user-defined-module}
46+
47+
As of Redis Enterprise Software version 8.0.6, you can include `user_defined_modules` in REST API requests to [initiate boostrap operations]({{<relref "/operate/rs/references/rest-api/requests/bootstrap#post-bootstrap">}}) such as `create_cluster`, `join_cluster`, or `recover_cluster`. Each node in the cluster independently downloads and installs the specified modules during its bootstrap process.
48+
49+
`user_defined_modules` has the following JSON schema:
50+
51+
```json
52+
{
53+
"user_defined_modules": [
54+
{
55+
"name": "string (required)",
56+
"location": {
57+
"location_type": "http | https (required)",
58+
"url": "string (required)",
59+
"credentials": {
60+
"username": "string (optional)",
61+
"password": "string (optional)"
62+
}
63+
}
64+
}
65+
]
66+
}
67+
```
68+
69+
### Best practices
70+
71+
- Use `https` instead of `http` for secure module downloads.
72+
73+
- Include version numbers in module URLs.
74+
75+
- Use the same `user_defined_modules` configuration for all nodes in a cluster.
76+
77+
- If using authenticated downloads, ensure credentials are properly secured.
78+
79+
- Ensure modules are compatible with the Redis database version running on your cluster.
80+
81+
- Verify modules work correctly before deploying to production environments.
82+
83+
### Example requests
84+
85+
{{< multitabs id="bootstrap-modules"
86+
tab1="Create cluster"
87+
tab2="Join cluster"
88+
tab3="Recover cluster" >}}
89+
90+
The following example creates a cluster with multiple modules:
91+
92+
93+
```sh
94+
POST /v1/bootstrap/create_cluster
95+
{
96+
"action": "create_cluster",
97+
"credentials": {
98+
"username": "[email protected]",
99+
"password": "your-secure-password"
100+
},
101+
"cluster": {
102+
"name": "my-cluster.example.com"
103+
},
104+
"user_defined_modules": [
105+
{
106+
"name": "ModuleA",
107+
"location": {
108+
"location_type": "https",
109+
"url": "https://private-repo.example.com/enterprise-module-2.0.0.zip",
110+
"credentials": {
111+
"username": "download-user",
112+
"password": "download-password"
113+
}
114+
}
115+
},
116+
{
117+
"name": "ModuleB",
118+
"location": {
119+
"location_type": "https",
120+
"url": "https://modules.example.com/module-b-2.5.0.zip"
121+
}
122+
},
123+
{
124+
"name": "ModuleC",
125+
"location": {
126+
"location_type": "http",
127+
"url": "http://internal-server.local/module-c-1.2.0.zip"
128+
}
129+
}
130+
]
131+
}
132+
```
133+
134+
-tab-sep-
135+
136+
The following example joins a node to a cluster with multiple modules:
137+
138+
```sh
139+
POST /v1/bootstrap/join_cluster
140+
{
141+
"action": "join_cluster",
142+
"credentials": {
143+
"username": "[email protected]",
144+
"password": "your-secure-password"
145+
},
146+
"cluster": {
147+
"name": "my-cluster.example.com",
148+
"nodes": ["192.168.1.10", "192.168.1.11"]
149+
},
150+
"user_defined_modules": [
151+
{
152+
"name": "ModuleA",
153+
"location": {
154+
"location_type": "https",
155+
"url": "https://private-repo.example.com/enterprise-module-2.0.0.zip",
156+
"credentials": {
157+
"username": "download-user",
158+
"password": "download-password"
159+
}
160+
}
161+
},
162+
{
163+
"name": "ModuleB",
164+
"location": {
165+
"location_type": "https",
166+
"url": "https://modules.example.com/module-b-2.5.0.zip"
167+
}
168+
},
169+
{
170+
"name": "ModuleC",
171+
"location": {
172+
"location_type": "http",
173+
"url": "http://internal-server.local/module-c-1.2.0.zip"
174+
}
175+
}
176+
]
177+
}
178+
```
179+
180+
-tab-sep-
181+
182+
The following example recovers a cluster with multiple modules:
183+
184+
```sh
185+
POST /v1/bootstrap/recover_cluster
186+
{
187+
"action": "recover_cluster",
188+
"recovery_filename": "/path/to/backup.rdb",
189+
"credentials": {
190+
"username": "[email protected]",
191+
"password": "your-secure-password"
192+
},
193+
"user_defined_modules": [
194+
{
195+
"name": "ModuleA",
196+
"location": {
197+
"location_type": "https",
198+
"url": "https://private-repo.example.com/enterprise-module-2.0.0.zip",
199+
"credentials": {
200+
"username": "download-user",
201+
"password": "download-password"
202+
}
203+
}
204+
},
205+
{
206+
"name": "ModuleB",
207+
"location": {
208+
"location_type": "https",
209+
"url": "https://modules.example.com/module-b-2.5.0.zip"
210+
}
211+
},
212+
{
213+
"name": "ModuleC",
214+
"location": {
215+
"location_type": "http",
216+
"url": "http://internal-server.local/module-c-1.2.0.zip"
217+
}
218+
}
219+
]
220+
}
221+
```
222+
223+
{{< /multitabs >}}
224+
225+
### Troubleshooting
226+
227+
#### Error handling
228+
229+
Download failures do not fail the bootstrap process. If a module fails to download or install, a warning is logged and the bootstrap process continues with the remaining modules.
230+
231+
Warnings are recorded in the bootstrap status with:
232+
- `warning_type`: `"module_download_failed"`
233+
- `message`: Error description
234+
- `details`: `{"module_name": "<name>"}`
235+
236+
#### Module download failed
237+
238+
Check the bootstrap logs for detailed error messages:
239+
240+
```
241+
Failed to download and install custom module '<name>': <error details>
242+
```
243+
244+
Common causes:
245+
- Invalid URL
246+
- Network connectivity issues
247+
- Authentication failures
248+
- Module package format issues
249+
250+
#### Module compatibility errors
251+
252+
After processing user-defined modules, the system validates that all custom modules are compatible with existing databases in the cluster. This validation:
253+
254+
1. Checks which custom modules are used by existing databases.
255+
256+
1. Verifies that compatible module versions are available on the node.
257+
258+
1. Fails the bootstrap process if incompatible modules are detected.
259+
260+
If the bootstrap process fails with an `incompatible_modules` error:
261+
262+
1. Verify the module version is compatible with existing databases.
263+
264+
1. Ensure the module binary exists and is accessible.
265+
266+
#### Missing module.json
267+
268+
If you see `"module.json missing"` errors:
269+
270+
1. Verify the zip file contains a valid `module.json` at the root level.
271+
272+
1. Verify the JSON is properly formatted.
273+
29274
## Add a user-defined module to a cluster (Redis Software v8.0.x and later) {#add-user-defined-module-to-cluster}
30275

31276
To add a custom module to a cluster running Redis Enterprise Software version 8.0.x or later, use the following REST API requests:

‎content/operate/rs/references/rest-api/objects/bootstrap/_index.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,5 +39,7 @@ A bootstrap configuration object.
3939
| recovery_filename | string | Name of backup file to recover from |
4040
| required_version | string | This node can only join the cluster if all nodes in the cluster have a version greater than the required_version (deprecated as of Redis Enterprise Software v7.8.6) |
4141
| retry_time | integer | Max waiting time between retries (in seconds) |
42+
| user_defined_modules | array of [user_defined_module]({{< relref "/operate/rs/references/rest-api/objects/bootstrap/user_defined_module" >}}) objects | List of custom modules to download and install during bootstrap. Each node downloads and installs the modules independently. |
43+
| witness_disk | object | An API object that represents the Witness Disk bootstrap configuration |
4244

4345

‎content/operate/rs/references/rest-api/objects/bootstrap/user_defined_module.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
Title: user_defined_module object
3+
alwaysopen: false
4+
categories:
5+
- docs
6+
- operate
7+
- rs
8+
description: An object for user-defined module configuration during bootstrap
9+
hideListLinks: true
10+
linkTitle: user_defined_module
11+
weight: $weight
12+
---
13+
14+
A user-defined module configuration object for bootstrap operations.
15+
16+
| Name | Type/Value | Description |
17+
|------|------------|-------------|
18+
| name | string | Module name for presentation and logging purposes (required) |
19+
| location | object | Information on where to download the module from (required)<br />{{<code>}}{
20+
"location_type": "http | https",
21+
"url": "string",
22+
"credentials": {
23+
"username": "string",
24+
"password": "string"
25+
}
26+
}{{</code>}}<br />**location_type**: The type of location, either `http` or `https` (required)<br />**url**: The URL to download the module zip file from (required)<br />**credentials**: Optional credentials for downloads that require basic authentication |
27+
28+
## Module package requirements
29+
30+
The module must be packaged as a `.zip` file containing:
31+
32+
- **module.json**: A metadata file with module information including:
33+
- `module_name`: The actual module name
34+
- `version`: Numeric version
35+
- `semantic_version`: Semantic version string (for example, "1.0.0")
36+
- `min_redis_version`: Minimum compatible Redis version
37+
- `commands`: List of commands the module provides
38+
- `capabilities`: List of module capabilities
39+
40+
- **Module binary**: The compiled `.so` file for the target platform

‎content/operate/rs/release-notes/rs-8-0-releases/rs-8-0-6-tba.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ categories:
66
- operate
77
- rs
88
compatibleOSSVersion: Redis 8.2.1, 8.0.2, 7.4.3, 7.2.7, 6.2.13
9-
description: Single sign-on for the Cluster Manager UI. Slot migration API. Error reports for Replica Of migration status.
9+
description: Single sign-on for the Cluster Manager UI. Slot migration API. Error reports for Replica Of migration status. Automatically download and install user-defined modules during bootstrapping.
1010
linkTitle: 8.0.6-tba (December 2025)
1111
weight: 88
1212
---
@@ -23,6 +23,8 @@ This version offers:
2323

2424
- Error reports for Replica Of migration status
2525

26+
- Automatically download and install user-defined modules during bootstrapping
27+
2628
## New in this release
2729

2830
### New features
@@ -51,6 +53,8 @@ New database actions allow you to migrate and cancel slot migrations between Red
5153

5254
- Added error report to Replica Of [migration status]({{<relref "/operate/rs/references/rest-api/requests/migrations">}}) REST API responses.
5355

56+
- Added support for automatically downloading and installing user-defined modules during bootstrap operations. You can now specify `user_defined_modules` in [bootstrap requests]({{<relref "/operate/rs/references/rest-api/requests/bootstrap#post-bootstrap">}}) for `create_cluster`, `join_cluster`, and `recover_cluster` actions. See [Add user-defined modules during bootstrapping]({{<relref "/operate/oss_and_stack/stack-with-enterprise/install/add-module-to-cluster#bootstrap-user-defined-module">}}) for details.
57+
5458
### Redis database versions
5559

5660
Redis Enterprise Software version 8.0.6 includes five Redis database versions: 8.2.1, 8.0.2, 7.4.3, 7.2.7, and 6.2.13.

0 commit comments

Comments
 (0)