-
-
Notifications
You must be signed in to change notification settings - Fork 6
feat: Replace ubi9 with ubi10 #1353
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
4c6276f
feat(ubi-rust-builder): replace ubi9 with ubi10
NickLarsenNZ 5de8a0b
feat: Use ubi10 in base images
NickLarsenNZ 538b3ba
feat(airflow/wip): use a newer uv version, and install the correct py…
NickLarsenNZ b3a8f7d
chore: fix typo in upload script
NickLarsenNZ 01d1d59
chore(wip): Upload script for nvm
NickLarsenNZ fa263be
chore: Move lint config into the right place
NickLarsenNZ ccc9669
chore(airflow): replace python/pip package installations with uv for …
NickLarsenNZ a8262eb
Merge remote-tracking branch 'origin/main' into feat/ubi10
NickLarsenNZ 3ec8524
chore(airflow): Replace source with .
NickLarsenNZ 37c8469
chore(airflow): Remove unneeded comments
NickLarsenNZ 1e71c9a
uvx migrate-to-uv
razvan 1ccdaf8
chore(superset/opa-authorizer): Add description and format the file
NickLarsenNZ 36fcf26
chore(superset): Upgrade uv version
NickLarsenNZ fde2223
chore(superset): Use uv instead of pip
NickLarsenNZ b2f3b46
fix(superset): Add patch to pin the sax package to 1.2.4
Techassi b20b6c5
chore(mirror): Allow uv image to be mirrored
NickLarsenNZ d67b340
chore: Use mirrored uv image
NickLarsenNZ 6115244
chore(airflow): Add hint about updating git-sync
NickLarsenNZ 96ce2d9
chore(airflow):Remove unused args
NickLarsenNZ 12aa476
chore(superset):Remove unused args
NickLarsenNZ 10e1b12
Merge remote-tracking branch 'origin/main' into feat/ubi10
NickLarsenNZ 0d0e4df
chore: Update changelog
NickLarsenNZ 46994bd
chore: Remove unused upload script
NickLarsenNZ 2e0e711
Apply suggestions from code review
NickLarsenNZ File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,33 +1,32 @@ | ||
| # syntax=docker/dockerfile:1.16.0@sha256:e2dd261f92e4b763d789984f6eab84be66ab4f5f08052316d8eb8f173593acf7 | ||
| # check=error=true;skip=InvalidDefaultArgInFrom,SecretsUsedInArgOrEnv | ||
| # Disabled error checks: | ||
| # - SecretsUsedInArgOrEnv : OPA_AUTH_MANAGER is a false positive and breaks the build. | ||
| # check=error=true;skip=InvalidDefaultArgInFrom,SecretsUsedInArgOrEnv | ||
|
|
||
| ARG GIT_SYNC_VERSION | ||
| ARG UV_VERSION | ||
|
|
||
| # For updated versions check https://github.com/kubernetes/git-sync/releases | ||
| # which should contain a image location (e.g. registry.k8s.io/git-sync/git-sync:v3.6.8) | ||
| FROM oci.stackable.tech/sdp/git-sync/git-sync:${GIT_SYNC_VERSION} AS gitsync-image | ||
|
|
||
| FROM oci.stackable.tech/sdp/astral-sh/uv:${UV_VERSION} AS uv-image | ||
|
|
||
| FROM local-image/shared/statsd-exporter AS statsd_exporter-builder | ||
|
|
||
| FROM local-image/vector AS opa-auth-manager-builder | ||
|
|
||
| ARG OPA_AUTH_MANAGER | ||
| ARG PYTHON_VERSION | ||
| ARG UV_VERSION | ||
| ARG STACKABLE_USER_UID | ||
|
|
||
| COPY airflow/opa-auth-manager/${OPA_AUTH_MANAGER} /tmp/opa-auth-manager | ||
|
|
||
| WORKDIR /tmp/opa-auth-manager | ||
|
|
||
| RUN <<EOF | ||
| microdnf update | ||
| microdnf install python${PYTHON_VERSION}-pip | ||
| microdnf clean all | ||
|
|
||
| pip${PYTHON_VERSION} install --no-cache-dir uv==${UV_VERSION} | ||
| COPY --from=uv-image --chown=${STACKABLE_USER_UID}:0 /uv /uvx /bin/ | ||
|
|
||
| RUN <<EOF | ||
| # This folder is required by the tests to set up an sqlite database | ||
| mkdir /root/airflow | ||
|
|
||
|
|
@@ -43,10 +42,8 @@ ARG PRODUCT_VERSION | |
| ARG PYTHON_VERSION | ||
| ARG TARGETARCH | ||
| ARG STACKABLE_USER_UID | ||
| ARG NODEJS_VERSION | ||
| ARG S3FS_VERSION | ||
| ARG CYCLONEDX_BOM_VERSION | ||
| ARG UV_VERSION | ||
|
|
||
| # Airflow "extras" packages are listed here: https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html | ||
| # They evolve over time and thus belong to the version-specific arguments. | ||
|
|
@@ -62,8 +59,9 @@ ARG AIRFLOW_EXTRAS_EXTERNAL_SERVICES="" | |
| ARG AIRFLOW_EXTRAS_LOCALLY_INSTALLED_SOFTWARE="" | ||
| ARG AIRFLOW_EXTRAS_OTHER="" | ||
|
|
||
| RUN microdnf module enable -y nodejs:${NODEJS_VERSION} && \ | ||
| microdnf update && \ | ||
| COPY --from=uv-image --chown=${STACKABLE_USER_UID}:0 /uv /uvx /bin/ | ||
|
|
||
| RUN microdnf update && \ | ||
| microdnf install \ | ||
| cyrus-sasl-devel \ | ||
| # Needed for kerberos | ||
|
|
@@ -78,10 +76,6 @@ RUN microdnf module enable -y nodejs:${NODEJS_VERSION} && \ | |
| libpq-devel \ | ||
| openldap-devel \ | ||
| openssl-devel \ | ||
| python${PYTHON_VERSION} \ | ||
| python${PYTHON_VERSION}-devel \ | ||
| python${PYTHON_VERSION}-pip \ | ||
| python${PYTHON_VERSION}-wheel \ | ||
| # The airflow odbc provider can compile without the development files (headers and libraries) (see https://github.com/stackabletech/docker-images/pull/683) | ||
| unixODBC \ | ||
| # Needed for Airflow UI assets | ||
|
|
@@ -111,14 +105,10 @@ AIRFLOW_EXTRAS="$AIRFLOW_EXTRAS_CORE,$AIRFLOW_EXTRAS_META,$AIRFLOW_EXTRAS_PROVID | |
| # Removing duplicates | ||
| AIRFLOW_EXTRAS=$(echo "$AIRFLOW_EXTRAS" | tr ',' '\n' | awk 'NF > 0 {if (!seen[$0]++) print $0}' | tr '\n' ',' | sed 's/,$//') | ||
|
|
||
| python${PYTHON_VERSION} -m venv --system-site-packages /stackable/app | ||
|
|
||
| source /stackable/app/bin/activate | ||
| uv venv --python "${PYTHON_VERSION}" --system-site-packages /stackable/app | ||
| . /stackable/app/bin/activate | ||
|
|
||
| # Upgrade pip to the latest version | ||
| # Also install uv to get support for build constraints | ||
| pip install --no-cache-dir --upgrade pip | ||
| pip install --no-cache-dir uv==${UV_VERSION} | ||
| # Another project manager | ||
| uv tool install hatch | ||
|
|
||
| cd "$(/stackable/patchable --images-repo-root=src checkout airflow ${PRODUCT_VERSION})" | ||
|
|
@@ -225,14 +215,15 @@ COPY --from=gitsync-image --chown=${STACKABLE_USER_UID}:0 /git-sync ${HOME}/git- | |
|
|
||
| COPY airflow/licenses /licenses | ||
|
|
||
| COPY --from=uv-image --chown=${STACKABLE_USER_UID}:0 /uv /uvx /bin/ | ||
|
|
||
| # Update image and install needed packages | ||
| RUN <<EOF | ||
| microdnf update | ||
|
|
||
| # git: Needed for the gitsync functionality | ||
| # openldap: Needed for authentication of clients against LDAP servers | ||
| # openssh-clients: We need the openssh libs for the gitsync functionality (the clone target could be e.g. [email protected]:org/repo.git) | ||
| # python: Airflow needs Python | ||
| microdnf install \ | ||
| ca-certificates \ | ||
| cyrus-sasl \ | ||
|
|
@@ -242,8 +233,6 @@ microdnf install \ | |
| openldap-clients \ | ||
| openssh-clients \ | ||
| openssl-libs \ | ||
| openssl-pkcs11 \ | ||
| python${PYTHON_VERSION} \ | ||
| socat \ | ||
| unixODBC | ||
| microdnf clean all | ||
|
|
@@ -253,6 +242,13 @@ rm -rf /var/cache/yum | |
| # It is used as an init alternative in the entrypoint | ||
| curl -o /usr/bin/tini "https://repo.stackable.tech/repository/packages/tini/tini-${TINI_VERSION}-${TARGETARCH}" | ||
|
|
||
| # There is currently no way to tell `uv` where to write to, and it assumes `$HOME` for the XDG paths. | ||
| export XDG_DATA_HOME=/usr/local/share | ||
| export XDG_CACHE_HOME=/var/cache | ||
| uv venv --python "${PYTHON_VERSION}" --system-site-packages /stackable/app | ||
| chown -R "${STACKABLE_USER_UID}:0" /stackable/app/bin | ||
| . /stackable/app/bin/activate | ||
|
|
||
| # fix missing permissions | ||
| chmod a+x /entrypoint.sh | ||
| chmod a+x /run-airflow.sh | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NickLarsenNZ marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.