Konflux CI/CD:
Installation & Reference Guide for OpenShift 4

A complete reference guide covering installation, application onboarding, build pipelines, integration testing, and release management on Red Hat OpenShift 4. Includes all YAML manifests, commands, and configuration needed to get started.

The Complete System Architecture Map

Konflux is not a monolith. It's a collection of independent Kubernetes controllers that talk to each other exclusively through CRDs โ€” never direct API calls. Every action you take creates or modifies a CRD; every background action updates a CRD status. This makes everything observable.

External โ€” Developer Inputs GitHub / GitLab Repo Developer / kubectl quay.io Registry Konflux UI (app.redhat.com) openshift-pipelines namespace (Tekton operator-managed) Pipelines as Code (pac-controller) Tekton Pipelines (pipeline controller) Tekton Chains (signing controller) Tekton Results (archive โ†’ Postgres) cert-manager (ns: cert-manager) + Kyverno (ns: kyverno) Konflux Core Controllers (each a separate Kubernetes reconciler) HAS Application+Component Build Service manages pipeline SA Integration Service Snapshot + tests Release Service gated releases Enterprise Contract (Conforma) Rego policy on OCI attestations Tenant Namespace (e.g. default-tenant) โ€” your workspace Application CRD Component CRD PipelineRun (build + test) Snapshot CRD IntegrationTest Scenario CRD ReleasePlan CRD Release CRD OpenShift 4 Infrastructure OLM (Operators) Routes (Ingress) etcd (all state) SCC / RBAC Kyverno Policies Default StorageClass (PVCs) ๐Ÿ”‘ Controllers only communicate via CRDs
All Konflux CRDs at a Glance
CRD KindAPI GroupOwning ControllerPurpose
Applicationappstudio.redhat.com/v1alpha1HASTop-level grouping of related microservices
Componentappstudio.redhat.com/v1alpha1HAS + Build ServiceOne microservice โ€” source repo, destination image, build config
Repositorypipelinesascode.tekton.dev/v1alpha1Build Service โ†’ PaCMaps a GitHub repo to a tenant namespace for PaC webhook routing (auto-created)
Snapshotappstudio.redhat.com/v1alpha1Integration ServicePoint-in-time map of every Component's image digest for an Application
IntegrationTestScenarioappstudio.redhat.com/v1beta2Integration ServiceDeclares a test pipeline to run against every new Snapshot
ReleasePlanappstudio.redhat.com/v1alpha1Release ServiceTenant-side: where and how to release; which ReleasePlanAdmission to pair with
ReleasePlanAdmissionappstudio.redhat.com/v1alpha1Release ServiceManaged-side: approves the ReleasePlan, defines the release pipeline and EC policy
Releaseappstudio.redhat.com/v1alpha1Release ServiceA triggered release โ€” binds Snapshot to ReleasePlan; runs the release pipeline
EnterpriseContractPolicyenterprisecontract.dev/v1alpha1ConformaRego rules that OCI attestations must satisfy before any release proceeds
PipelineRun / TaskRuntekton.dev/v1Tekton PipelinesOne triggered run; contains task graph, results, and log references
Konfluxkonflux-ci.dev/v1alpha1Konflux OperatorCluster-wide CR controlling which Konflux components are deployed

Key Takeaways

  • Konflux = 5+ independent Kubernetes controllers, each owning specific CRDs
  • Controllers communicate ONLY by reading/writing Kubernetes resources
  • All Tekton + PaC + Chains live in openshift-pipelines on OCP
  • Kyverno auto-creates RBAC when a namespace gets the tenant label
  • All state is in etcd โ€” controllers are stateless and idempotent
  • kubectl describe <crd> โ†’ Conditions section is your #1 debug tool

From Code Push to Running PipelineRun

Eight steps happen automatically between git push and a Tekton PipelineRun appearing in your tenant namespace. Here's every one of them.

git push your commit GitHub webhook POST PaC Controller validates HMAC Fetch .tekton/ from commit SHA Substitute {{revision}} etc. PipelineRun in tenant namespace Pods t = 0s t โ‰ˆ 1-2s t โ‰ˆ 5-10s

๐Ÿš€ Hold on โ€” What if .tekton/ Doesn't Exist Yet? (First-Time Onboarding)

The flow above assumes .tekton/ already exists in your repo. But when you first create a Component CRD pointing at a fresh repository, there are no pipeline files there yet. Konflux handles this entirely automatically โ€” the Build Service raises a PR on your behalf to add those files. You only need to review and merge it.

๐Ÿ“‹ Phase 0: First-Time Onboarding โ€” What Build Service Does Automatically
You create Component CRD kubectl apply -f component.yaml โ‘  You do this Build Service reconciles creates Repository CR registers webhook via PaC โ‘ก Automatic GitHub PR opened Build Service calls GitHub API .tekton/component-push.yaml .tekton/component-pr.yaml โ‘ข Automatic You review & merge PR inspect the generated pipeline YAML files โ‘ฃ You do this โœ“ Webhooks Active future pushes trigger PipelineRuns automatically โ‘ค Automatic PR contains two pipeline files: โ€ข push.yaml โ†’ triggers on git push (full build + sign + scan) โ€ข pull-request.yaml โ†’ on PRs (build only, image expires 5d) โŒ No .tekton/ yet โœ“ .tekton/ now in repo

๐Ÿ”Ž Why does Build Service open a PR instead of committing directly?

Build Service does NOT have write access to push directly to your repo's main branch. Instead, it creates a branch on your fork and opens a PR, so you retain full control. The PR title is typically "Add Konflux CI pipelines". You should review the generated YAML before merging โ€” this is your pipeline definition.

โœ… After Merging: The Normal Push Flow Begins

Once the .tekton/ PR is merged, every subsequent push to the target branch (main by default) automatically triggers the push pipeline. Every PR opened against the repo triggers the pull-request pipeline. You never have to do this setup again for this Component.
๐Ÿ” After .tekton/ Is Merged โ€” Step-by-Step: What Happens on Every Push
1

git push โ†’ GitHub receives the commit

GitHub processes your push. It looks up all webhook subscriptions on the repository and finds one registered by your GitHub App, pointing at the PaC controller Route in your cluster.

2

GitHub POSTs to the PaC webhook Route

GitHub sends the push event payload to https://pipelines-as-code-controller-openshift-pipelines.apps.<cluster>. The X-Hub-Signature-256 HMAC header is calculated from the webhook.secret in pipelines-as-code-secret.

3

PaC validates HMAC + routes to tenant namespace

The pac-controller pod in openshift-pipelines validates the HMAC signature, identifies the event type, then finds the matching Repository CR in the cluster by comparing the GitHub repo URL. The Repository CR says which namespace to create PipelineRuns in.

4

PaC fetches .tekton/ pipeline files from the exact commit SHA

Using the GitHub App private key, PaC makes an authenticated GitHub API call to read .tekton/ from the pushed commit. It filters for files with on-event: "[push]". It also fetches the pipeline YAML from the OCI bundle reference (pipelineRef.resolver: bundles).

5

PaC substitutes template variables and creates the PipelineRun

Variables like {{revision}}, {{repo_url}}, {{target_branch}} are replaced with real values from the webhook payload. PaC calls the Kubernetes API to create a tekton.dev/v1 PipelineRun in your tenant namespace, with the build-pipeline-<component> ServiceAccount.

6

Tekton Pipelines controller picks up the PipelineRun

The Tekton Pipelines controller (in openshift-pipelines) watches all PipelineRuns cluster-wide. It sees the new one and creates child TaskRun objects in topological order, each backed by a Pod in your tenant namespace. Task definitions are fetched from their OCI bundles on quay.io.

7

PaC posts a "CI started" GitHub Check Run

Simultaneously, PaC creates a GitHub Check Run on the commit (yellow dot in GitHub). It updates the check on each task status change. On success/failure, the check shows the final result with a link to logs.

8

Integration Service watches for PipelineRun completion

When the build PipelineRun completes with Succeeded, the Integration Service reads the IMAGE_URL and IMAGE_DIGEST Tekton results and creates a Snapshot CRD. (Covered in Section 5.)

๐Ÿ”‘ The Repository CR is the bridge

The pipelinesascode.tekton.dev/v1alpha1 Repository CR (auto-created by Build Service) maps a GitHub repo URL to a tenant namespace. PaC reads this to know where to create PipelineRuns. Without it, webhooks are silently ignored.

โš ๏ธ Why the secret must be in 3 namespaces

openshift-pipelines: PaC reads it to validate webhook HMAC + call GitHub API.
build-service: Build Service reads it to open the initial .tekton/ PR.
integration-service: Integration Service reads it to post test status to GitHub.

Key Takeaways

  • PaC is a webhook server โ€” it receives GitHub events and creates PipelineRuns
  • The Repository CR maps a GitHub repo to a tenant namespace
  • PaC fetches .tekton/ YAML from the exact commit SHA that was pushed
  • Template variables ({{revision}}, {{repo_url}}) substituted at webhook time
  • Pipeline structure is defined inline as pipelineSpec in your .tekton/ files; individual Task implementations are fetched from digest-pinned OCI bundles on quay.io
  • All PipelineRuns run in your tenant namespace under a dedicated ServiceAccount

Build Pipeline Internals โ€” Every Task, Every Pod, Every Result

Once the PipelineRun exists, Tekton orchestrates a series of tasks. Individual Task implementations are fetched from digest-pinned OCI bundles on quay.io at runtime โ€” they are never stored as CRDs in the cluster. Modern Konflux builds use OCI Trusted Artifacts (tasks suffixed -oci-ta) to pass source code and build outputs between tasks as OCI artifacts in the registry rather than a shared PVC.

PHASES 1 & 2 โ€” Sequential (each waits for the previous to finish) PHASE 3 โ€” Parallel (all 6 run concurrently) init validate params clone-repository git clone โ†’ /workspace/source prefetch- dependencies Cachi2 (skip if not hermetic) build-container Buildah โ†’ quay.io push โ†’ IMAGE_URL + IMAGE_DIGEST build-image- index multi-arch OCI index ๐Ÿ“ฆ OCI Trusted Artifacts (-oci-ta tasks): source passed as OCI artifacts in registry, not a shared PVC deprecated-base-image-check checks for deprecated base images โ†’ TEST_OUTPUT clamav-scan antivirus scan on container image โ†’ TEST_OUTPUT sast-shell-check shell script SAST analysis โ†’ TEST_OUTPUT sast-unicode-check unicode/bidi attack detection โ†’ TEST_OUTPUT rpms-signature-scan RPM package signature check โ†’ TEST_OUTPUT tpa-scan Trusted Profile Analyzer scan โ†’ TEST_OUTPUT summary โ€” finalize results โ†’ Tekton Chains fires ๐Ÿ”— After pipeline: Tekton Chains passively signs image + generates SLSA (Section 4 โ†“)
Critical Tekton Results โ€” The Handoff Mechanism
Result NameSet ByRead ByContains
IMAGE_URLbuild-containerIntegration Service, Tekton ChainsFull image ref without digest: quay.io/org/repo:git-sha
IMAGE_DIGESTbuild-containerIntegration Service, Tekton Chainssha256 digest of the pushed OCI manifest
IMAGE_REFbuild-containerIntegration ServiceCombined: quay.io/org/repo@sha256:abc...
CHAINS-GIT_URLclone-repositoryTekton ChainsSource repo URL โ€” embedded in SLSA provenance
CHAINS-GIT_COMMITclone-repositoryTekton ChainsGit commit SHA โ€” embedded in SLSA provenance
SBOM_BLOB_URLsbom-syft-generate (post-pipeline)Tekton ChainsURL of the SBOM OCI artifact in quay.io
TEST_OUTPUTdeprecated-base-image-check, clamav-scan, sast-shell-check, sast-unicode-check, rpms-signature-scan, tpa-scanIntegration Service (gating)JSON: {"result":"SUCCESS","timestamp":"..."}

โš  Why Buildah Needs Elevated Privileges on OpenShift

Buildah runs as root inside the build pod on OCP (using anyuid SCC pre-configured on the build-pipeline-<name> ServiceAccount by Build Service). It uses the overlay storage driver โ€” confirmed by the STORAGE_DRIVER: overlay param visible in every build's SLSA attestation. Running as root with anyuid means kernel-level overlay support is available directly, so VFS is not needed and not used.

๐Ÿ“ฆ OCI Bundles โ€” Only for Tasks, Not the Pipeline Definition

Konflux does not store Task YAML in the cluster. Each Task implementation is packaged as a digest-pinned OCI artifact on quay.io and fetched at runtime via taskRef.resolver: bundles. However, the Pipeline definition itself is embedded inline as pipelineSpec directly inside your .tekton/ PipelineRun files โ€” it is version-controlled in your own repository, not fetched from a remote bundle. This gives you full ownership of the pipeline structure while the task implementations remain immutable and supply-chain-verified.

Key Takeaways

  • Task implementations are fetched from digest-pinned OCI bundles on quay.io at runtime
  • The pipeline structure (pipelineSpec) lives inline in your .tekton/ files โ€” not in a remote OCI bundle
  • OCI Trusted Artifacts (-oci-ta tasks) pass source code as OCI artifacts in the registry, not via a shared PVC
  • Tekton Results (IMAGE_DIGEST, IMAGE_URL) are the handoff mechanism to downstream services
  • Buildah uses the overlay storage driver (confirmed in SLSA attestation params) inside an anyuid privileged container on OCP
  • After build-image-index, 6 scans run in parallel: deprecated-base-image-check, clamav-scan, sast-shell-check, sast-unicode-check, rpms-signature-scan, tpa-scan
  • clamav-scan scans the built container image layers โ€” not source files
  • Tekton Chains is NOT a pipeline task โ€” it's a separate controller that fires after TaskRun completion

Tekton Chains โ€” Image Signing and SLSA Provenance

Tekton Chains is a completely passive Kubernetes controller. It never participates in the pipeline โ€” it only watches TaskRun completions and fires afterwards to produce the supply chain security record.

tekton-chains-controller (openshift-pipelines) โ€” passive observer, fires AFTER the pipeline TaskRun Succeeds has IMAGE_URL + IMAGE_DIGEST results Generate SLSA Attestation in-toto format: subject= image digest, materials= git commit + repo Sign with Cosign (cluster key) signs: image digest signs: attestation JSON Push to quay.io as OCI artifacts :sha256-abcโ€ฆsig :sha256-abcโ€ฆatt Annotate TaskRun chains.tekton.dev/ signed: "true" quay.io/yourorg/demo-server โ€” after Chains runs :git-abc1234 (image) :sha256-abcโ€ฆ.sig (signature) :sha256-abcโ€ฆ.att (SLSA) :sha256-abcโ€ฆ.sbom (SBOM) Containerfile

๐Ÿ“‹ What the SLSA Attestation Contains

predicateType: https://slsa.dev/provenance/v0.2 (SLSA v0.2)
subject: image digest that was built
builder.id: https://tekton.dev/chains/v2
buildType: tekton.dev/v1/TaskRun (or PipelineRun)
materials: git commit SHA + repo URL (from CHAINS-GIT_* results)
invocation.parameters: all build task params
metadata.buildFinishedOn: TaskRun completion timestamp

๐Ÿ” How to verify what Chains produced

On self-hosted Konflux, Chains uses the cluster's internal keypair (not public Rekor). Add --insecure-ignore-tlog to skip the public Rekor lookup:

kubectl get secret signing-secrets -n openshift-pipelines -o jsonpath='{.data.cosign\.pub}' | base64 -d > /tmp/cosign.pub

cosign verify --key /tmp/cosign.pub --insecure-ignore-tlog IMAGE@DIGEST

cosign verify-attestation --key /tmp/cosign.pub --insecure-ignore-tlog --type slsaprovenance IMAGE@DIGEST | jq '.payload|@base64d|fromjson'

Key Takeaways

  • Tekton Chains is a passive observer โ€” fires AFTER the build, not during
  • Chains reads IMAGE_URL + IMAGE_DIGEST from the build-image-index TaskRun (the task that produces the final manifest digest, wrapping the per-arch images)
  • Generates a SLSA v0.2 in-toto attestation (predicateType: https://slsa.dev/provenance/v0.2) documenting all build inputs
  • Both image and attestation are signed with the cluster Cosign key
  • Signatures and attestations pushed as OCI artifacts alongside the image in quay.io
  • Enterprise Contract (Section 6) evaluates these attestations before any release

Snapshots and Integration Tests

A Snapshot is a versioned, point-in-time map of the exact image digest for every Component in an Application. Integration tests run against the Snapshot โ€” testing all components together โ€” not individual components in isolation.

demo-server build PipelineRun โ†’ sha256:abc demo-frontend build PipelineRun โ†’ sha256:def demo-worker build PipelineRun โ†’ sha256:ghi Integration Service watches PipelineRun Snapshot CRD demo-server: sha256:abc demo-frontend: sha256:def demo-worker: sha256:ghi auto-created by IS status: AppStudioTestSucceeded smoke-test pipeline (IntegrationTestScenario) e2e-test pipeline (IntegrationTestScenario) contract-test pipeline (IntegrationTestScenario) All Pass AppStudioTestSucceeded: True โ†’ Snapshot is release-eligible AppStudioTestSucceeded: False โ†’ Snapshot cannot be released If NO IntegrationTestScenario exists โ†’ immediately: True โœ“

๐Ÿ“ธ Why Snapshots Are the Right Abstraction

In a microservices app with 5 components, component A might build 3 times while components B-E each build once. Which combination should be tested? After each build completes, the Integration Service creates (or updates) a Snapshot capturing the latest successful digest for every component. Tests run against this coherent combination โ€” not individual components in isolation. This is how Konflux ensures you're always testing what you'll ship.

IntegrationTestScenario โ€” Wiring Your Tests
example-integration-test.yaml
apiVersion: appstudio.redhat.com/v1beta2
kind: IntegrationTestScenario
metadata:
  name: smoke-test
  namespace: default-tenant
  labels:
    appstudio.redhat.com/application: my-first-app
spec:
  application: my-first-app
  resolverRef:
    resolver: bundles
    params:
      - name: bundle
        value: quay.io/yourorg/test-pipelines:latest
      - name: name
        value: smoke-test-pipeline
  params:
    - name: SNAPSHOT
      value: "$(params.SNAPSHOT)"  # Integration Service injects the full Snapshot JSON here
      # Your test pipeline parses this to know which image digests to pull and test

Key Takeaways

  • A Snapshot is auto-created after every successful component build
  • It captures the exact digest of every component at that moment
  • Tests run against the Snapshot โ€” testing all components together
  • All IntegrationTestScenarios must pass before a Snapshot can be released
  • No IntegrationTestScenario โ†’ Snapshot is immediately release-eligible
  • AppStudioTestSucceeded condition is the gate the Release Service watches

Release Service and Enterprise Contract โ€” Gated, Policy-Enforced Releases

Releases require agreement between two parties: the tenant (your team) and the managed service (the ops team who controls where production artifacts go). This two-party model prevents accidental production releases and enforces supply chain policy.

Tenant Namespace (default-tenant) developer-controlled ReleasePlan target: release-service application: my-first-app autoRelease: false Release CR snapshot: my-first-app-8xkpq releasePlan: my-release-plan โ†’ triggers the release process Snapshot (passed tests) AppStudioTestSucceeded: True Release Service matches validates creates PR EC Policy check (Conforma) Rego on OCI attestation Managed Namespace (release-service) ops-team-controlled โ€” production credentials here ReleasePlanAdmission origin: default-tenant (approves this tenant) application: my-first-app pipeline: push-to-prod-pipeline policy: enterprise-contract-policy Release PipelineRun runs with ELEVATED PRIVILEGES has access to production registry creds promotes image โ†’ production quay.io copies SBOM + attestation too Release CR status: Released: True โœ“ ๐Ÿ”’ Production creds NEVER cross into tenant namespace
1

Release CR is created (manually or via autoRelease)

Either you run kubectl apply -f release.yaml, or โ€” if spec.autoRelease: true in the ReleasePlan โ€” the Integration Service creates a Release automatically when the Snapshot passes its tests.

2

Release Service finds the matching ReleasePlanAdmission

Reads the Release โ†’ reads the ReleasePlan โ†’ searches for a ReleasePlanAdmission in the managed namespace with spec.origin = default-tenant and spec.application = my-first-app. No match โ†’ Release fails immediately.

3

Enterprise Contract (Conforma) evaluates policy

Conforma pulls the SLSA attestation for each component image in the Snapshot from quay.io. It evaluates Rego policy rules: image must be signed, SLSA provenance present, base image from approved registry, no critical CVEs. If policy fails โ†’ Release is blocked and the failure reason is written to Release.status.conditions.

4

Release PipelineRun created in managed namespace

If policy passes, the Release Service creates a PipelineRun in release-service namespace with elevated permissions. Production registry credentials are accessible here โ€” but not from the tenant namespace. The release pipeline promotes the image and publishes supply chain artifacts.

5

Release CR status updated to Released: True

As the release PipelineRun succeeds, Release Service writes Released: True to the Release CR with a completion time and link to the pipeline run for audit trail.

Key Takeaways

  • Two-party model: ReleasePlan (tenant) โ†” ReleasePlanAdmission (managed service)
  • Enterprise Contract evaluates Rego policy on SLSA attestations BEFORE pipeline runs
  • Release pipelines run in managed namespace โ€” access to production credentials
  • Production credentials NEVER reach the tenant namespace
  • autoRelease: true โ†’ releases trigger automatically when tests pass
  • The Release CR is the full audit trail โ€” policy results, timing, pipeline link

Namespace and RBAC Architecture

Konflux uses namespace isolation as its primary security boundary. System controllers are in locked-down namespaces; your CRDs and PipelineRuns live in tenant namespaces; production credentials live in managed namespaces.

System Namespaces (cluster-admin only) openshift-pipelines PaC controller Tekton Pipelines Tekton Chains + Results konflux-operator Operator controller reconciles Konflux CR build-service Build Service ctrl + pac-secret copy integration-service Integration Service ctrl + pac-secret copy cert-manager TLS cert issuance for service-to-service kyverno auto-creates RBAC in tenant namespaces Tenant Namespace (default-tenant) โ€” your workspace Application Component CRDs (yours) PipelineRuns build + test (Tekton pods) Snapshot IntTestScenario ReleasePlan regcred secret build registry push quay.io creds build-pipeline-<component> ServiceAccount (auto-created by Build Service per Component) Kyverno auto-creates RBAC here Managed Namespace (release-service) ReleasePlanAdmission release pipeline ref EC policy ref Prod registry creds secret โ€” inaccessible from tenant namespace ๐Ÿšซ Kubernetes RBAC blocks cross-namespace Secret reads enterprise-contract-service Conforma controller + service โ€” evaluates Rego policy against OCI attestations from quay.io

๐Ÿค– What Kyverno Creates in Every Tenant Namespace (Automatically)

  • RoleBindings โ€” grants Integration Service, Build Service, and Release Service controllers read/write access to CRDs in the namespace
  • ClusterRoleBindings โ€” grants PaC controller permission to create PipelineRuns in the namespace
  • LimitRange โ€” sets default CPU/memory requests for build pods
  • ResourceQuota โ€” caps total resource consumption per tenant namespace

All of this fires automatically when you label a namespace with konflux-ci.dev/type=tenant. No manual RBAC setup needed.

Key Takeaways

  • System namespaces (build-service, integration-service) are locked โ€” cluster-admin only
  • Tenant namespaces hold your CRDs and PipelineRuns
  • Kyverno auto-creates all RBAC when namespace gets konflux-ci.dev/type=tenant label
  • Each Component gets its own SA (build-pipeline-<name>) for isolated registry auth
  • Managed namespaces hold production credentials โ€” no cross-namespace reads
  • RBAC is enforced by Kubernetes itself โ€” Release Service cannot be bypassed

How Controllers Work โ€” The Reconciliation Loop

Every Konflux service is a standard Kubernetes controller built with controller-runtime. Each runs an infinite reconciliation loop. Understanding this loop explains why Konflux is resilient, idempotent, and always observable via CRD status.

The Kubernetes Controller Reconciliation Loop (runs continuously in each controller pod) Watch Queue Kubernetes informer caches CRD changes Reconciler Called Reconcile(ns/name) Read Current State kubectl GET resource Compute Desired State business logic Apply Changes Create/Update/Delete Update Status .status.conditions โ† loop restarts on next watch event (or after requeue delay on error) โœ… Idempotent Running reconcile twice for same input produces the same result โœ… Crash-Safe Controller restart replays from current Kubernetes state (etcd) ๐Ÿ” Always Observable kubectl describe <crd> โ†’ Conditions section shows all actions
What Each Controller Watches and Creates
ControllerWatches (triggers on)Creates / Updates
HASApplication, Component CRDsValidates CRD structure; updates Application.status with component list; runs admission webhooks
Build ServiceComponent CRDsRepository CR (for PaC); build-pipeline-<name> ServiceAccount; opens .tekton/ PR on GitHub
Pipelines as CodeRepository CRDs + GitHub webhook events (HTTP)PipelineRun (in tenant namespace); GitHub Check Run (via GitHub API)
Tekton PipelinesPipelineRun, TaskRun CRDsTaskRun per task; Pod per TaskRun; PVC from volumeClaimTemplate
Tekton ChainsTaskRun CRDs that produce IMAGE_URL + IMAGE_DIGEST results (e.g. build-image-index)Cosign signature (.sig) in quay.io; SLSA v0.2 attestation (.att) in quay.io; annotates TaskRun with chains.tekton.dev/signed: "true"
Integration ServicePipelineRun CRDs (with component label) + Snapshot CRDsSnapshot CR; integration test PipelineRun; updates Snapshot.status.conditions
Release ServiceRelease CRDs + Snapshot.status (AppStudioTestSucceeded)Release PipelineRun in managed namespace; updates Release.status
Konflux OperatorKonflux CRAll Konflux service deployments; namespace creation; RBAC setup via Kyverno
Debug Conditions on Any CRD
Reading status conditions โ€” your #1 debugging tool
NS="default-tenant"

# Component โ€” is it accepted and pipeline set up?
kubectl get component demo-server -n $NS \
  -o jsonpath='{.status.conditions}' | jq '.[] | {type, status, message}'

# Snapshot โ€” did integration tests pass?
kubectl get snapshot  -n $NS \
  -o jsonpath='{.status.conditions}' | jq '.[] | {type, status, reason, message}'
# Key: AppStudioTestSucceeded: True/False/Unknown

# Release โ€” what happened during release?
kubectl get release  -n $NS \
  -o jsonpath='{.status.conditions}' | jq '.[] | {type, status, reason, message}'
# Key conditions: Released, ReleaseValidated (EC pass/fail), ReleasePlanValid

# Konflux operator โ€” are all system components healthy?
kubectl describe konflux konflux | grep -A 60 "Conditions:"

Key Takeaways

  • Every controller runs: watch โ†’ reconcile โ†’ create/update โ†’ status
  • Controllers are stateless โ€” all state is in Kubernetes etcd
  • Status conditions are the communication channel between controllers
  • Controllers are idempotent โ€” safe to restart, requeue, or retry
  • A controller crash is not catastrophic โ€” it replays from current etcd state
  • kubectl describe <crd> โ†’ Conditions section is always your first debug step

OCI Registry Internals and the Complete End-to-End Timeline

Konflux uses quay.io not just for container images but as a universal artifact store. OCI has evolved beyond container images โ€” any blob can be stored alongside an image using OCI reference types. Every build produces 5 linked artifacts.

quay.io/yourorg/demo-server โ€” all artifacts for one build (sha256:abc1234...) OCI Image :git-abc1234 Container layers + image config โ† this is what docker pull downloads Pushed by: Buildah in build-container task Cosign Signature :sha256-abcโ€ฆ.sig Signed image digest + Cosign envelope JSON Pushed by: Tekton Chains SLSA Attestation :sha256-abcโ€ฆ.att Signed in-toto JSON: build inputs, git commit, builder identity SBOM Artifact :sha256-abcโ€ฆ.sbom CycloneDX / SPDX JSON: all software components in image Containerfile OCI artifact (.source) Your Containerfile stored as OCI blob alongside image Pushed by: push-dockerfile task
The Complete Timeline: git push โ†’ Production
git push t = 0s PipelineRun created t โ‰ˆ 5s Build completes t โ‰ˆ 5-15min Image signed + SLSA t +30s (Chains) Snapshot created t +10s (IS) Tests pass varies EC check โ†’ Release +5min Developer PaC Tekton Chains Integration Svc Integration Svc Release Svc
Quick Debugging Cheat Sheet
When things go wrong โ€” where to look first
NS="default-tenant"

# "My pipeline isn't triggering"
# 1. Repository CR present?
kubectl get repository -n $NS
# 2. GitHub App secret in all 3 namespaces?
for ns in openshift-pipelines build-service integration-service; do
  echo -n "$ns: "; kubectl get secret pipelines-as-code-secret -n $ns -o name 2>&1
done
# 3. PaC controller logs
kubectl logs -n openshift-pipelines \
  -l app.kubernetes.io/component=controller,app.kubernetes.io/part-of=pipelines-as-code \
  --tail=50 | grep -i "error\|webhook"

# "Build failed"
tkn pipelinerun logs --last -n $NS
kubectl get pipelinerun -n $NS --sort-by=.metadata.creationTimestamp | tail -5

# "No Snapshot created after successful build"
kubectl logs -n integration-service -l control-plane=controller-manager --tail=80

# "Image signing not happening"
kubectl get taskrun -n $NS \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.annotations.chains\.tekton\.dev/signed}{"\n"}{end}'

# "Release is stuck"
kubectl describe release  -n $NS | grep -A 30 "Conditions:"
kubectl get releaseplanadmission -n release-service   # check matching RPA exists

# "Enterprise Contract failed"
kubectl get release  -n $NS \
  -o jsonpath='{.status.conditions[?(@.type=="ReleaseValidated")].message}'

Key Takeaways โ€” The Complete Picture

  • quay.io stores: image, signature (.sig), attestation (.att), SBOM (.sbom), Containerfile
  • All supply chain artifacts linked to image digest via OCI referrers
  • Tekton Results archives PipelineRun history in Postgres for audit
  • Full journey: pushโ†’PipelineRun (~5s)โ†’build (~10min)โ†’signโ†’Snapshotโ†’testsโ†’release
  • Every step is auditable via CRD status conditions
  • Every artifact is signed; every release is policy-enforced โ€” this is Konflux's core promise

End-to-End Production Flow โ€” Every Object, Every Step

From Application creation to a signed, released image โ€” exactly which Kubernetes object is created, who creates it, and when. The most important timing insight: the Snapshot is created the instant a build succeeds, before any integration test starts. Tests are triggered by the Snapshot โ€” not the other way around.

โ‘  SETUP done once before any code is pushed You Application CR logical grouping Component CR source repo + image reconciles โ†’ Build Service Repository CR auto-created, webhook reg. PR opened on repo adds .tekton/ pipeline files You merge PR .tekton/ in repo โœ“ You IntegrationTestScenario which test pipeline to run per Snapshot ReleasePlan tenant namespace Ops ReleasePlanAdmission managed namespace, ops team โœ“ Setup complete โ€” webhooks active Every push to the target branch now automatically triggers a build PipelineRun โ‘ก EVERY PUSH git push developer GitHub webhook POST to PaC Route HMAC verified PaC Controller reads .tekton/, substitutes {{revision}} and vars Build PipelineRun in tenant namespace initโ†’cloneโ†’buildโ†’scan tasks IMAGE_URL + IMAGE_DIGEST produced by build-container as Tekton results โ†’ PipelineRun reaches Succeeded โ‘ข โ˜… SNAPSHOT CREATED Integration Service watches build PipelineRuns reads IMAGE_DIGEST result SNAPSHOT CR immutable image digest map for every component โš  BEFORE any integration test has run The Snapshot creation IS the event that triggers integration tests. Tests run against the Snapshot โ€” not against the build pipeline. โ‘ฃ INTEGRATION TESTS per scenario Integration Service finds IntegrationTestScenario(s) for this Application creates one test run each Test PipelineRun runs against Snapshot pipeline fetched via git resolver at test time AppStudioTestSucceeded: True โœ“ โ†’ release-eligible AppStudioTestSucceeded: False โœ— โ†’ cannot be released โ‘ค RELEASE Release CR Release Service Enterprise Contract โœ“ Release PipelineRun โœ“ Image released to production

โ˜… The Most Commonly Misunderstood Timing

Many people assume integration tests run first, and if they pass, a Snapshot is created as the "test report". It works the opposite way:

  • Build succeeds โ†’ Integration Service immediately creates the Snapshot (image digest locked in)
  • Snapshot creation โ†’ Integration Service looks up all IntegrationTestScenarios and fires test PipelineRuns
  • Test PipelineRuns complete โ†’ Integration Service writes the result back onto the Snapshot's status conditions

The Snapshot is the trigger for tests, not the output of them. It always exists before the first test runs.

Phase-by-Phase Detail
โ‘ 

Setup โ€” done once per project

You create Application โ†’ Component. Build Service reconciles Component and creates a Repository CR (webhook registration) and opens a PR on your repo adding .tekton/ pipeline files. You merge that PR. Then you apply IntegrationTestScenario (what to test), ReleasePlan (how to release, tenant side), and an ops team applies ReleasePlanAdmission (managed side). Setup is complete โ€” you never repeat these steps for this component.

  • Application CR โ€” first object, logical grouping for all components
  • Component CR โ€” source repo + target image; triggers Build Service
  • Repository CR โ€” auto-created by Build Service; maps GitHub repo to tenant namespace for PaC routing
  • IntegrationTestScenario โ€” declares which test pipeline to run on every Snapshot
  • ReleasePlan / ReleasePlanAdmission โ€” two-sided release contract between tenant and ops teams
โ‘ก

Every push to the target branch โ€” Build PipelineRun created

GitHub delivers a webhook to the PaC controller Route in openshift-pipelines. PaC validates the HMAC signature, fetches .tekton/ files from the exact commit SHA, substitutes template variables ({{revision}}, {{source_url}}), and creates a Build PipelineRun in your tenant namespace. Tekton orchestrates the tasks: init โ†’ clone-repository โ†’ prefetch-dependencies โ†’ build-container โ†’ build-image-index then parallel scans. The critical outputs are the IMAGE_URL and IMAGE_DIGEST Tekton results produced by build-container.

โ‘ข

โ˜… Build succeeds โ†’ Snapshot created immediately (BEFORE tests)

The Integration Service watches all Build PipelineRuns cluster-wide. The instant a build PipelineRun transitions to Succeeded and has IMAGE_URL + IMAGE_DIGEST results, the Integration Service creates a Snapshot CR. This Snapshot contains the immutable image digest for every component in the Application. No integration test has run yet at this point. After this, Tekton Chains independently signs the image and generates the SLSA attestation.

โ‘ฃ

Snapshot created โ†’ integration tests triggered automatically

The Integration Service sees the new Snapshot and looks up all IntegrationTestScenario CRDs for the Application. For each one, it creates an integration test PipelineRun in the tenant namespace, fetching the test pipeline from the git resolver at runtime. When all test PipelineRuns complete, the Integration Service writes the verdict onto the Snapshot: AppStudioTestSucceeded: True (release-eligible) or False (blocked). If no IntegrationTestScenario exists, the Snapshot is marked True immediately.

โ‘ค

Release โ€” manual trigger or auto-release

A Release CR is created โ€” either by you pointing at a passing Snapshot, or automatically by the Integration Service if auto-release: "true" is set on the ReleasePlan. The Release Service finds the matching ReleasePlanAdmission, runs Enterprise Contract (Rego policy) validation against the SLSA attestation, and if EC passes, creates a Release PipelineRun in the managed namespace. The release pipeline does the actual deployment work โ€” pushing to a prod registry, updating a GitOps repo, etc. The Release CR status conditions record the outcome.

Complete Object Creation Timeline
ObjectCreated byWhen (precisely)
Application You First object created โ€” before any other Konflux CRD. Nothing works without this.
Component You Created after Application exists โ€” immediately triggers Build Service reconciliation.
Repository Build Service Auto-created within seconds of Component being reconciled โ€” registers the GitHub webhook via PaC.
IntegrationTestScenario You Applied once during setup after the Repository exists โ€” must be in place before the first push that you want tested.
ReleasePlan You Applied once in the tenant namespace during setup โ€” must be in place before you want to trigger releases.
ReleasePlanAdmission Ops team Applied once in the managed namespace by the ops/SRE team โ€” must exist and match the ReleasePlan before any Release can run.
Build PipelineRun PaC Controller Created on every push to the target branch (from the push pipeline) or on every PR open/update (from the PR pipeline). One PipelineRun per event.
Snapshot Integration Service Created immediately when a build PipelineRun reaches Succeeded โ€” before any integration test starts. The Snapshot creation is what triggers the tests.
Integration test PipelineRun Integration Service Created per IntegrationTestScenario immediately after the Snapshot is created. One PipelineRun per scenario, all triggered at the same moment.
Release You (or Integration Service) Created manually after confirming the Snapshot has AppStudioTestSucceeded: True, or auto-created by Integration Service if auto-release: true is set on the ReleasePlan.
Release PipelineRun Release Service Created in the managed namespace after the Release CR is accepted and Enterprise Contract validation passes. Never created if EC fails.

๐Ÿ“‹ PR Pipeline vs Push Pipeline โ€” Snapshot behaviour

Both pipelines produce a Snapshot when they succeed. The difference:

  • Push pipeline (merge to main) โ†’ image has no expiry โ†’ Snapshot is release-eligible after tests pass
  • PR pipeline (open/update a PR) โ†’ image has image-expires-after: 5d โ†’ Snapshot is created and tests run so reviewers can see results, but the image expires and the Snapshot is never used for a release

Key Takeaways

  • Setup objects (Application, Component, IntegTestScenario, ReleasePlan, RPA) are created once and stay forever
  • Every push creates a Build PipelineRun โ†’ on success, Integration Service creates a Snapshot immediately
  • Snapshot is created BEFORE integration tests โ€” it IS the trigger for them
  • Integration tests run against the Snapshot (not the build pipeline) via IntegrationTestScenario
  • Snapshot status condition AppStudioTestSucceeded is the gate for releases
  • Release flow: Release CR โ†’ EC validation โ†’ Release PipelineRun in managed namespace
  • Every transition is observable via kubectl describe <crd> Conditions section
  • All controllers communicate only via CRD status โ€” no direct API calls between them

When and What Gets Pushed to the OCI Registry

A complete Konflux pipeline cycle results in two distinct registry push events with fundamentally different purposes โ€” and a significant no-push verification phase between them. Understanding this lifecycle is essential for debugging, cost optimisation, and designing production registry strategies. The image content (layers) is written exactly once. Every subsequent operation works with that content-addressed artifact.

The Two Canonical Push Events
โ‘  BUILD PIPELINE PUSH Buildah writes image bytes to registry Trigger git push โ†’ main (push pipeline) PR open / update (PR pipeline) What gets written to registry โ€ข OCI image config (JSON blob) โ€ข Filesystem layer tarballs (the bytes) โ€ข OCI manifest (wires config + layers) Tool: Buildah inside build-container task Tag convention push: :git-sha123 โ† permanent build ID PR: :on-pr-sha123 โ† expires in 5d Critical property Digest (sha256 of manifest) is the immutable identity. Tag is a mutable pointer. Always reference by digest in production, never by floating tag. VERIFICATION PHASE no new image bytes โ€” registry is read-only Tekton Chains (passive, fires after TaskRun) Pushes supply-chain artifacts (not the image): :sha256-sha123....sig :sha256-sha123....att Integration Service โ†’ Snapshot CR reads IMAGE_DIGEST from TaskRun results (no push) Integration Test PipelineRuns pulls and runs image to assert behaviour (no push) Enterprise Contract (at release time) fetches .att from registry, verifies policy (no push) โœ“ Release CR approved manual approval or auto-release label this is the formal promotion decision โ‘ก RELEASE PIPELINE PUSH skopeo copy โ€” zero new bytes written Trigger Release CR created + EC passes Release Service runs release pipeline What gets written to registry โ€ข New manifest tag โ†’ existing digest โ€ข No layers transferred (content-addressed) โ€ข Registry deduplicates identical blobs Tool: skopeo copy in release pipeline task Tag convention :latest :stable :v1.2.0 :release-X different org/registry in production Critical property sha256 digest is IDENTICAL to Push 1. The release is the approval record โ€” the Release CR is the audit trail that links digest โ†’ approved for production.
What Actually Lives in Your OCI Registry After a Full Cycle

๐Ÿ—‚ OCI Content-Addressable Storage โ€” the key mental model

An OCI registry is not a file server. It stores blobs (addressed by sha256) and manifests (also addressed by sha256). A tag is just a named pointer to a manifest digest. When you run skopeo copy image:sha โ†’ image:latest, the registry simply moves the latest pointer โ€” no bytes are transferred. The base image layers shared with other images are stored once across all images that reference them.

quay.io / yourorg / your-app โ€” artifact tree after one complete build + release cycle Each artifact below is stored as content-addressed blobs (sha256). Tags are mutable pointers to immutable content. :git-abc123def456 โ† PUSH 1 (push pipeline) ยท Buildah ยท commit SHA tag Content: image config JSON + filesystem layers (the actual image bytes). sha256 of manifest = permanent digest. :on-pr-xyz789012 โ† PUSH 1 (PR pipeline) ยท Buildah ยท PR image, expires in 5d, NOT releasable Same structure as push image. May share base layers. Deleted by quay.io after expiry. :sha256-abc123def456....sig โ† Tekton Chains ยท cosign OCI signature artifact JWS (JSON Web Signature) over the image digest, signed with the cluster cosign key from openshift-pipelines/signing-secrets. :sha256-abc123def456....att โ† Tekton Chains ยท SLSA provenance in-toto attestation (signed) in-toto SLSA v0.2 statement: subject=image digest, materials=git commit+repo, builder=Tekton Chains. Verified by EC. :sha256-abc123def456....sbom โ† Tekton Chains / build pipeline ยท SBOM (if configured) SPDX or CycloneDX Software Bill of Materials. Attached via cosign attach sbom. Not always present in minimal setups. :latest โ†’ sha256:abc123def456.... โ† PUSH 2 (release pipeline) ยท skopeo copy ยท no bytes transferred Points to the same digest as :git-abc123def456. Manifest tag written atomically. Consumers pull this stable name. :stable โ†’ sha256:abc123def456.... โ† PUSH 2 (release pipeline) ยท skopeo copy ยท same digest again A more conservative stable pointer. Some teams use :latest for "newest release" and :stable for "last known good". Build pipeline Tekton Chains Release pipeline PR / ephemeral Optional artifact
The Production Registry Promotion Model
Build / CI Registry quay.io/yourorg-ci/app (example) Access control: โ€ข All engineers can push (CI robot accounts) โ€ข Images may carry expiry annotations โ€ข Not subject to production RBAC Contents: :git-sha :on-pr-sha (all builds) :sha256-sha....sig .att .sbom Characteristics: โ€ข High write frequency โ€ข Ephemeral images (PR builds expire) โ€ข Source of truth for all builds Release Pipeline skopeo copy EC validated ยท Release CR audited ๐Ÿ”’ only release pipeline can write to prod Production Registry quay.io/yourorg-prod/app (example) Access control: โ€ข Write: release pipeline SA only โ€ข Read: all deployment systems โ€ข Air-gapped in regulated environments Contents: :latest :stable :v1.2.0 (curated) :sha256-sha....sig .att (promoted) Characteristics: โ€ข Low write frequency (releases only) โ€ข Every write is auditable via Release CR โ€ข Deployment systems pull only from here
Push-by-Push Walkthrough
1

PR pipeline: ephemeral build push (optional, pre-merge validation)

When a pull request is opened against main, PaC triggers the PR pipeline (.tekton/app-pull-request.yaml). Buildah builds the image and pushes it tagged as :on-pr-<git-SHA> with image-expires-after: 5d. This image is used exclusively for pre-merge confidence โ€” the developer and reviewer can confirm the image builds and integration tests pass before the code lands in main. The expiry annotation tells the registry to garbage-collect this image automatically. It is never eligible for release.

2

Push pipeline: canonical build push (the permanent build artifact)

When a merge to main occurs, PaC triggers the push pipeline (.tekton/app-push.yaml). The build-container (Buildah) task constructs the OCI image and pushes it to the registry tagged as :<git-SHA>. This is the canonical artifact โ€” the result of the official CI build. The tag uses the full commit SHA, making it a permanent, human-readable reference to the exact source that produced it. The build-image-index task then creates an OCI image index (manifest list) that wraps per-architecture manifests โ€” this is the final digest referenced by all downstream systems.

3

Tekton Chains: supply-chain artifact push (automatic, post-TaskRun)

Tekton Chains watches all TaskRun completions cluster-wide. When the build-image-index TaskRun completes and exposes IMAGE_URL + IMAGE_DIGEST results, Chains fires. It generates an in-toto SLSA v0.2 attestation encoding every build input (git commit, task images, parameters) and signs both the image and the attestation with the cluster cosign key from openshift-pipelines/signing-secrets. These are stored in the same registry as OCI artifacts with tags following the cosign legacy format: :sha256-<digest>.sig and :sha256-<digest>.att. This entire process is transparent to the pipeline โ€” Chains is a passive observer, never a pipeline task.

  • .sig โ€” JWS signature proving the image digest was signed by the cluster key
  • .att โ€” SLSA in-toto provenance: who built it, from what source, with what tools
  • .sbom โ€” Software Bill of Materials (if attached via cosign attach sbom)
4

No push: Snapshot, integration tests, EC validation

The Integration Service reads IMAGE_DIGEST from the PipelineRun results and creates a Snapshot CR โ€” no registry write. Integration test PipelineRuns pull the image and test its behaviour โ€” no registry write. Enterprise Contract fetches the .att artifact from the registry, evaluates Rego policy rules against the attestation content, and marks the Snapshot โ€” no registry write. The entire verification phase is read-only against the registry. This is by design: verification must not alter what it is verifying.

5

Release pipeline: tag promotion push (approval materialised as a registry write)

When a Release CR is created (manually or via auto-release), the Release Service creates a release PipelineRun in the managed namespace. The push-to-staging-registry task runs skopeo copy docker://source@sha256:<digest> docker://dest:latest. This creates a new manifest tag pointing to the exact same content-addressed blobs already in the registry. No image bytes are re-transferred if source and destination share a registry โ€” the OCI spec guarantees blob deduplication. If source and destination are different registries (the production pattern), only blobs absent from the destination are transferred. The resulting :latest and :stable tags are the production-consumable references. The digest they point to is provably identical to the signed, attested build artifact from Push 1.

๐Ÿ”‘ Digest is truth. Tag is a human-readable alias.

A tag like :latest is a mutable pointer โ€” any push to :latest silently updates it. Two deployments of :latest one hour apart may run completely different code with no visible difference in the manifest name. A digest like @sha256:abc123... is immutable โ€” it is the sha256 of the manifest content. If the content changes, the digest changes. Always specify image references by digest in production Kubernetes manifests, Helm values, and GitOps repos. Use tags for human readability in CLIs only.

โšก Why skopeo copy is nearly instantaneous for same-registry promotion

OCI registries are content-addressed blob stores. When you push :abc123 (Push 1), all layer blobs and the image config are written once. When the release pipeline runs skopeo copy :abc123 โ†’ :latest (Push 2) against the same registry, the registry checks each blob's sha256 โ€” all already exist. Only the manifest (a few hundred bytes of JSON) is written. The promotion is atomic and near-instant regardless of image size. For cross-registry promotions (CI โ†’ prod), only blobs absent on the destination travel over the network.
Complete Push Lifecycle โ€” At a Glance
EventWho writes to registryTag formatNew bytes?Releasable?
PR pipeline build Buildah (build-container task in PR pipeline) :on-pr-<git-SHA> Yes โ€” full image layers written No โ€” expires in 5d, test use only
Push pipeline build Buildah (build-container task in push pipeline) :<git-SHA> Yes โ€” full image layers written Yes โ€” after tests + EC pass
Image signing Tekton Chains controller (passive, post-TaskRun) :sha256-<digest>.sig Yes โ€” JWS signature blob N/A โ€” supply-chain artifact
SLSA attestation Tekton Chains controller (passive, post-TaskRun) :sha256-<digest>.att Yes โ€” signed provenance JSON N/A โ€” supply-chain artifact
Snapshot creation Integration Service (CRD write, not registry) โ€” No N/A โ€” Kubernetes object
Integration tests Test pipeline (reads image only) โ€” No N/A
EC validation EC task (reads .att from registry only) โ€” No N/A
Release tag promotion skopeo (release pipeline, push-to-staging task) :latest :stable :v1.x No (same registry) or layer delta (cross-registry) This IS the release

๐Ÿ”„ What happens when a CVE is found in a base image

This is where understanding the two-push lifecycle pays off in production:

  • Trigger a rebuild โ€” update the base image reference in the Dockerfile/Containerfile and push. Push 1 runs: a new image with the patched base is written to the CI registry with a new commit SHA tag and a new sha256 digest.
  • The release flow runs again โ€” integration tests, EC validation, Release CR. Push 2 runs: :latest and :stable now point to the patched digest.
  • The old image remains in the registry โ€” content-addressed storage is immutable. The old :git-sha-of-vuln-build tag still exists and can be inspected. It is simply no longer referenced by :latest or deployed by any system that follows the stable tags.
  • GitOps repositories update automatically โ€” if the release pipeline commits the new digest to the GitOps repo, ArgoCD/Flux reconciles the change and redeploys. Zero manual intervention.

Key Takeaways

  • Image bytes are written exactly once โ€” by Buildah in the build pipeline. All subsequent operations use content-addressed references.
  • Tekton Chains pushes .sig + .att after the build โ€” these are supply-chain artifacts, not the image itself.
  • Snapshot creation, integration tests, and EC validation are read-only against the registry by design.
  • Release pipeline push is a manifest tag write only โ€” no new bytes for same-registry promotion. The approval is the act of writing the stable tag.
  • PR images (:on-pr-sha) are never releasable โ€” they expire in 5d and exist only for pre-merge confidence.
  • Always reference images by digest in production (@sha256:abc), never by floating tag. Tags are mutable; digests are not.
  • In production: build registry and production registry are different with separate access controls. The release pipeline is the only writer to production.
  • The Release CR is the audit trail โ€” it links a specific digest to a specific approval decision at a specific time.

Installing & Deploying Konflux on OpenShift 4 โ€” Official SRE Onboarding Playbook

What You'll Learn

Official prerequisites: exact OCP version, tool versions, and cluster-admin scope
Every component deployed by deploy-konflux-on-ocp.sh โ€” what it is and why it's there
How the 6-step installation script works internally (no black boxes)
Creating a GitHub App and deploying its credentials into the 3 required namespaces
Configuring Quay.io registry credentials for build and release pipelines
Onboarding your first application via both Kubernetes manifests and the Konflux UI

Prerequisites

OpenShift v4.20+ with cluster-admin access oc (OpenShift CLI) v1.31.4+ git v2.46+, make, Go v1.26+, openssl v3.0.13+ GitHub account โ€” permission to create a GitHub App quay.io account or any OCI-compatible registry Cluster internet-reachable from GitHub (for webhooks) โ€” or Smee proxy

โ˜ธ Do NOT pre-install OpenShift Pipelines separately

The deploy-konflux-on-ocp.sh script installs OpenShift Pipelines via OLM as a dependency automatically. Manually pre-installing it can cause version mismatches. Just meet the tool version requirements above and run the script.

Steps

  1. Check prerequisites โ€” run the version commands in the Commands section to confirm OCP 4.20+, cluster-admin access, and correct tool versions. Also verify a default StorageClass exists (oc get storageclass) so PVC binding works during installation.
  2. Clone the repository and run the installation script โ€” git clone https://github.com/konflux-ci/konflux-ci.git && cd konflux-ci && ./deploy-konflux-on-ocp.sh. The script installs OpenShift Pipelines automatically as part of its six printed phases โ€” no separate step is needed.
  3. Wait for Konflux to become ready โ€” run oc wait --for=condition=Ready=True konflux konflux --timeout=600s and confirm all component pods are running with oc get pods -n konflux-ci.
  4. Create a GitHub App โ€” go to GitHub Developer Settings, create a new GitHub App, generate a webhook secret with openssl rand -hex 20, and download the private key .pem file. Keep these three values โ€” App ID, webhook secret, and private key โ€” ready for the next step.
  5. Deploy the GitHub App credentials โ€” use the for ns in openshift-pipelines build-service integration-service loop in the Commands section to create the PaC secret in all three namespaces. Note that on OCP, PaC lives in openshift-pipelines, not pipelines-as-code.
  6. Configure Quay.io registry access โ€” generate an encrypted password from the Quay UI, create the regcred secret in your tenant namespace, and link it to the build-pipeline service account so the build task can push images.
  7. Fork testrepo and onboard as a Component โ€” fork konflux-ci/testrepo on GitHub, apply the Application and Component manifests from the YAML Files section, and copy the .tekton/ pipeline files into your fork.
  8. Open a pull request and follow the build โ€” create a PR against your fork's main branch and follow the PipelineRun logs with tkn pipelinerun logs --last -f -n default-tenant until the build completes and a Snapshot is created automatically by the Integration Service.

YAML Files & Commands

Phase 1 โ€” Clone and run the official installation script
# โ”€โ”€ Prerequisites check โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
oc version            # must be 4.20+
oc version --client  # must be v1.31.4+
git --version         # must be 2.46+
go version            # must be 1.26+
openssl version       # must be 3.0.13+
make --version

# Verify cluster-admin
oc auth can-i '*' '*' --all-namespaces   # must print: yes

# Verify default StorageClass exists (required for PVC binding)
oc get storageclass
# Look for a row with (default) โ€” e.g. gp3-csi (default)

# โ”€โ”€ Clone the official Konflux repository โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
git clone https://github.com/konflux-ci/konflux-ci.git
cd konflux-ci

# โ”€โ”€ NOTE: Pre-export optional variables before running the script โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
#
# The script can automatically configure the GitHub App secret and the Smee
# client if the corresponding environment variables are exported beforehand.
#
# If you already have a GitHub App created and want the script to configure
# everything in one shot, export these before running ./deploy-konflux-on-ocp.sh:
#
#   export GITHUB_APP_ID="123456"                       # numeric App ID from GitHub
#   export WEBHOOK_SECRET="your-webhook-secret"         # secret you set in the GitHub App
#   export GITHUB_PRIVATE_KEY_PATH="/path/to/app.pem"  # private key .pem downloaded from GitHub
#
# Additionally, if your cluster is NOT reachable from the internet (e.g. behind
# a VPN or firewall) and GitHub cannot deliver webhooks directly, also export:
#
#   export SMEE_CHANNEL="https://smee.io/your-channel-id"
#
# When SMEE_CHANNEL is set, the script deploys a Smee proxy client inside the
# cluster that forwards GitHub webhook events to the PaC controller.
#
# If you don't have a GitHub App yet or prefer to configure these manually
# after installation, simply skip the exports โ€” the platform installs fine
# without them. You can run scripts/deploy-secrets.sh standalone afterwards.
# See Phase 3 below for the manual post-installation steps.
#
# โ”€โ”€ Run the deployment script โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# This single script does EVERYTHING:
#   - Installs OpenShift Pipelines Operator via OLM (DO NOT pre-install separately)
#   - Installs Red Hat cert-manager Operator via OLM
#   - Deploys Kyverno for namespace RBAC policy
#   - Sets up Tekton Chains RBAC for image signing
#   - Installs Prometheus CRDs for observability
#   - Installs Konflux CRDs (make install in operator/)
#   - Deploys the Konflux Operator to konflux-operator namespace
#   - Applies the default Konflux CR
#   - Waits for the CR to reach Ready
./deploy-konflux-on-ocp.sh

# To use a specific operator image (optional override):
# OPERATOR_IMAGE=quay.io/konflux-ci/konflux-operator:v0.1.0 ./deploy-konflux-on-ocp.sh
Phase 2 โ€” Verify the Konflux CR and all components are Ready
# Wait for the Konflux CR to be Ready (the script already does this,
# but run manually if you need to re-check after a restart)
oc wait --for=condition=Ready=True konflux konflux --timeout=600s

# Check the Konflux CR status and component conditions
oc describe konflux konflux

# Verify the Konflux Operator is running
oc get pods -n konflux-operator

# โ”€โ”€ Namespace inventory for deploy-konflux-on-ocp.sh on OCP โ”€โ”€
#
# SOURCE A โ€” OpenShift Pipelines OLM Operator (deploy-deps.sh, USE_OPENSHIFT_PIPELINES=true)
#   openshift-pipelines    โ†’ TektonConfig targetNamespace on OCP; all Tekton + PaC + Chains
#                            components co-located here (PaC does NOT get its own ns on OCP)
#
# SOURCE B โ€” Red Hat cert-manager OLM Operator (deploy-deps.sh, USE_OPENSHIFT_CERTMANAGER=true)
#   cert-manager-operator  โ†’ OLM Subscription + OperatorGroup namespace
#                            (dependencies/cert-manager-subscription/namespace.yaml)
#   cert-manager           โ†’ Created by the cert-manager operator itself upon reconciliation
#
# SOURCE C โ€” Kyverno (deploy-deps.sh โ†’ dependencies/kyverno/kustomization.yaml)
#   kyverno                โ†’ Kyverno policy engine; kyverno/releases/v1.18.1/install.yaml
#
# SOURCE D โ€” Konflux Operator CRDs + deploy (make install + make deploy, Steps 2-3/6)
#   konflux-operator       โ†’ operator/config/default (namespace for the operator controller)
#
# SOURCE E โ€” Konflux CR reconciliation (oc apply -f, Step 5/6 โ€” ALWAYS enabled by default)
#   build-service          โ†’ operator/pkg/manifests/build-service/manifests.yaml (KonfluxBuildService)
#   integration-service    โ†’ operator/pkg/manifests/integration/manifests.yaml (KonfluxIntegrationService)
#   release-service        โ†’ operator/pkg/manifests/release/manifests.yaml (KonfluxReleaseService)
#   namespace-lister       โ†’ operator/pkg/manifests/namespace-lister/manifests.yaml
#   konflux-ui             โ†’ operator/pkg/manifests/ui/manifests.yaml (KonfluxUI)
#   enterprise-contract-service โ†’ operator/pkg/manifests/enterprise-contract/manifests.yaml
#   konflux-info           โ†’ operator/pkg/manifests/info/manifests.yaml (KonfluxInfo)
#   konflux-cli            โ†’ operator/pkg/manifests/cli/manifests.yaml (KonfluxCLI)
#   default-tenant         โ†’ operator/pkg/manifests/default-tenant/manifests.yaml
#                            (conditional on spec.defaultTenant.enabled: true โ€” DEFAULT true)
#
# SOURCE F โ€” Konflux CR reconciliation (CONDITIONAL โ€” NOT enabled by default on OCP)
#   image-controller       โ†’ spec.imageController.enabled: true  (off by default)
#   segment-bridge         โ†’ spec.telemetry.enabled: true        (off by default)
#   kind-registry          โ†’ spec.internalRegistry.enabled: true (SKIP_INTERNAL_REGISTRY=true on OCP)
#
# NOT present on OCP (pipeline-operator specific behaviour):
#   pipelines-as-code      โ†’ upstream Tekton/Kind only; on OCP PaC runs in openshift-pipelines
#   tekton-pipelines       โ†’ upstream Tekton only; OCP uses openshift-pipelines
#   dex                    โ†’ SKIP_DEX=true; OCP has its own OAuth
#   application-service    โ†’ HAS installs only CRDs (no Namespace in operator/pkg/manifests/application-api/)

echo "=== Always-created namespaces ==="
for ns in \
  openshift-pipelines \
  cert-manager-operator \
  cert-manager \
  kyverno \
  konflux-operator \
  build-service \
  integration-service \
  release-service \
  namespace-lister \
  konflux-ui \
  enterprise-contract-service \
  konflux-info \
  konflux-cli \
  default-tenant; do
  echo ""
  echo "--- $ns ---"
  oc get pods -n "$ns" 2>/dev/null || echo "  (namespace not found or no pods yet)"
done

echo ""
echo "=== Conditionally-created namespaces ==="

# image-controller: only when spec.imageController.enabled: true in the Konflux CR
oc get namespace image-controller &>/dev/null && {
  echo ""
  echo "--- image-controller (imageController.enabled: true) ---"
  oc get pods -n image-controller
} || echo "  image-controller: not present (imageController not enabled in Konflux CR)"

# segment-bridge: only when spec.telemetry.enabled: true in the Konflux CR
oc get namespace segment-bridge &>/dev/null && {
  echo ""
  echo "--- segment-bridge (telemetry.enabled: true) ---"
  oc get pods -n segment-bridge
} || echo "  segment-bridge: not present (telemetry not enabled in Konflux CR)"

# smee-client: only when SMEE_CHANNEL was set during deploy-konflux-on-ocp.sh
oc get namespace smee-client &>/dev/null && {
  echo ""
  echo "--- smee-client (SMEE_CHANNEL was set during install) ---"
  oc get pods -n smee-client
} || echo "  smee-client: not present (cluster is internet-reachable โ€” no Smee proxy needed)"

# Verify all Konflux CRDs are installed
oc api-resources | grep -E 'appstudio|enterprisecontract|konflux'

# Access the Konflux UI
# The Konflux CR creates a Route in its namespace
oc get route -A | grep konflux
# Open the URL in your browser
Troubleshooting โ€” most common issues from the official docs
# โ”€โ”€ Issue: Pipelines not triggering on PRs โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Check PaC can reach GitHub (must have valid GitHub App secret in all 3 namespaces)
# On OCP: openshift-pipelines (NOT pipelines-as-code โ€” that is the upstream/Kind namespace)
oc get secret pipelines-as-code-secret -n openshift-pipelines
oc get secret pipelines-as-code-secret -n build-service
oc get secret pipelines-as-code-secret -n integration-service  # all 3 must exist!

# Check PaC controller logs for webhook delivery errors
# On OCP, PaC controller runs in openshift-pipelines (not pipelines-as-code)
oc logs -n openshift-pipelines \
  -l app.kubernetes.io/component=controller,app.kubernetes.io/part-of=pipelines-as-code --tail=50

# Check the Repository CRD status
oc describe repository testrepo -n default-tenant

# If cluster is not internet-reachable, set up Smee proxy:
# 1. Generate a channel: head -c 30 /dev/random | base64 | tr -dc 'a-zA-Z0-9'
# 2. Use https://smee.io/<channel-id> as webhook URL in GitHub App
# 3. The smee client must be deployed in the cluster as a Deployment

# โ”€โ”€ Issue: Build PVC stuck in Pending โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Verify default StorageClass supports ReadWriteOnce
oc get storageclass
oc describe pvc -n default-tenant | grep -A 10 Events

# โ”€โ”€ Issue: Running out of resources โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
oc get nodes -o custom-columns=NAME:.metadata.name,CPU:.status.allocatable.cpu,MEMORY:.status.allocatable.memory
oc top nodes  # requires metrics-server

# โ”€โ”€ Issue: Unable to create Application via UI (404 error) โ”€โ”€โ”€โ”€
# Ensure image-controller is enabled and the quaytoken secret is in image-controller ns
oc get secret quaytoken -n image-controller
oc get pods -n image-controller

# โ”€โ”€ Issue: Conflict CR status โ€” check Konflux CR conditions โ”€โ”€โ”€
oc describe konflux konflux | grep -A 30 "Conditions:"

# โ”€โ”€ Uninstall Konflux โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Remove the CR (this removes all managed components)
oc delete konflux konflux
# Remove the operator and CRDs
cd operator
make undeploy
make uninstall

What Each Deployed Component Does

Component / NamespaceInstalled ViaRole in the Platform
OpenShift PipelinesOLM SubscriptionExecutes all Tekton PipelineRuns; provides TektonConfig, Chains, PaC, and Dashboard in the openshift-pipelines namespace
Pipelines as Code (PaC)Part of OpenShift PipelinesReceives GitHub webhooks, resolves .tekton/ files from the repo, and creates PipelineRuns in the tenant namespace on push or PR events
Tekton ChainsPart of OpenShift PipelinesPassively watches completed TaskRuns; extracts IMAGE_URL + IMAGE_DIGEST results; generates SLSA in-toto attestations; signs both image and attestation using the cluster signing key stored in openshift-pipelines
Red Hat cert-managerOLM SubscriptionIssues and rotates TLS certificates for internal Konflux service-to-service communication
Kyvernodeploy-deps.shEnforces namespace policies; automatically creates RoleBindings and ServiceAccounts when a new tenant namespace is created, so users have the right permissions without manual RBAC setup
Konflux Operatormake deploy (operator/)Reconciles the Konflux CR; deploys and manages all Konflux core services as sub-resources; handles upgrades
Hybrid Application ServiceKonflux CRRuns validation webhooks for Application and Component CRDs; ensures referential integrity (e.g. Component must belong to an existing Application)
Build ServiceKonflux CRWatches Component CRDs; generates the build PipelineRun definition; manages the build-pipeline-<name> ServiceAccount in tenant namespaces
Integration ServiceKonflux CRWatches build PipelineRuns; on success, creates or updates a Snapshot; triggers test PipelineRuns for each IntegrationTestScenario; marks Snapshots as passed or failed; optionally auto-creates Releases
Release ServiceKonflux CRWatches Release CRDs; matches ReleasePlan to ReleasePlanAdmission; creates release PipelineRuns in managed namespaces; enforces Enterprise Contract (Conforma) policy before any release pipeline runs
Enterprise Contract (Conforma)Konflux CREvaluates Rego policy rules against SLSA attestations stored in the OCI registry; called by the Release Service before every release; blocks non-compliant images from being released
Image ControllerKonflux CR (opt-in)Automatically creates Quay.io repositories when Components are onboarded via the UI; requires a Quay OAuth token and imageController.enabled: true in the Konflux CR

Common Installation Issues

๐Ÿ”ง Official Troubleshooting Reference

  • Pipelines not triggering on PRs: The GitHub App secret must exist in all three namespaces (openshift-pipelines, build-service, integration-service). On OCP use openshift-pipelines โ€” not pipelines-as-code, which is the upstream Tekton/Kind namespace. Missing even one will cause silent failures for that service's operations.
  • Cluster not internet-reachable: GitHub cannot deliver webhooks to a private cluster. Set up a Smee channel: generate an ID with head -c 30 /dev/random | base64 | tr -dc 'a-zA-Z0-9', use https://smee.io/<id> as the webhook URL in the GitHub App, and set SMEE_CHANNEL when running the script.
  • PVC cannot bind (Pending): No default StorageClass with ReadWriteOnce support. Check oc get sc โ€” on AWS use gp3-csi, on GCP use standard-rwo.
  • Unable to create Application via UI (404): The image-controller is not enabled or the quaytoken secret is missing in the image-controller namespace.
  • Conflict or NotReady in Konflux CR: Run oc describe konflux konflux and read the Conditions section. Each condition maps to a specific component โ€” the message tells you which deployment failed and why.

Key Takeaways

  • Use konflux-ci/konflux-ci repo โ€” NOT infra-deployments (deprecated for this flow)
  • One script: ./deploy-konflux-on-ocp.sh โ€” OpenShift Pipelines installed automatically
  • GitHub App secret goes into 3 namespaces: openshift-pipelines, build-service, integration-service (OCP: PaC lives in openshift-pipelines, not pipelines-as-code)
  • Registry secret uses kubernetes.io/dockerconfigjson type; patch it onto the SA
  • Onboard apps by forking testrepo, copying pipelines to .tekton/, and opening a PR
  • Check oc describe konflux konflux when troubleshooting โ€” conditions are the source of truth

GitHub App & Registry Configuration

Configuration Steps

Phase 3 โ€” Create a GitHub App (browser steps + CLI secret deployment)  ยท  ๐Ÿ“„ Official Docs
# โ”€โ”€ STEP A: Generate a webhook secret (save this value) โ”€โ”€โ”€โ”€โ”€โ”€โ”€
WEBHOOK_SECRET=$(head -c 30 /dev/random | base64)
echo "Your webhook secret: $WEBHOOK_SECRET"

# โ”€โ”€ STEP A2: Get the PaC controller Route hostname BEFORE creating the GitHub App โ”€โ”€
# On OCP, the OpenShift Pipelines operator creates a Route for the PaC controller
# in the openshift-pipelines namespace. This is the URL GitHub will POST events to.
#
# Wait for the Route to exist (the OpenShift Pipelines operator creates it after
# TektonConfig reaches Ready โ€” may take 2-3 minutes after install completes)
echo "Waiting for PaC controller Route to be created..."
until oc get route pipelines-as-code-controller -n openshift-pipelines &>/dev/null; do
  echo "  Route not ready yet, retrying in 10s..."
  sleep 10
done

# Extract the full HTTPS webhook URL โ€” use this exactly in the GitHub App "Webhook URL" field
WEBHOOK_URL=$(oc get route pipelines-as-code-controller \
  -n openshift-pipelines \
  -o jsonpath='https://{.spec.host}')
echo ""
echo "================================================"
echo "  PaC Webhook URL (copy this into GitHub App):"
echo "  $WEBHOOK_URL"
echo "================================================"

# Verify the URL is reachable from your machine (optional sanity check)
# -k / --insecure is needed when the Route uses a self-signed or internal CA certificate,
# which is common on OpenShift clusters that haven't configured a custom ingress cert.
curl -sk -o /dev/null -w "HTTP status: %{http_code}\n" "$WEBHOOK_URL" || \
  echo "Note: curl failed โ€” confirm the cluster is internet-reachable from GitHub"

# โ”€โ”€ NOTE: If the pipelines-as-code-controller Route is NOT reachable from the internet โ”€โ”€
#
# GitHub needs to POST webhook events to the PaC controller URL ($WEBHOOK_URL above).
# If your cluster sits behind a VPN, firewall, or private network, GitHub cannot reach
# that Route directly. In that case you must use a Smee proxy channel as the webhook URL.
#
# 1. Go to https://smee.io/ and click "Start a new channel" โ€” you will get a unique URL
#    like: https://smee.io/abc123XYZ
#
# 2. Use that Smee URL as the Webhook URL when creating the GitHub App in STEP B below
#    (instead of the $WEBHOOK_URL printed above).
#
# 3. Export the channel URL before running deploy-konflux-on-ocp.sh so the script
#    deploys the Smee client inside the cluster automatically:
#      export SMEE_CHANNEL="https://smee.io/abc123XYZ"
#    Or deploy it manually after installation โ€” see the Smee deployment commands
#    in the Phase 1 NOTE above.

# โ”€โ”€ STEP B: Create the GitHub App in your browser โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# 1. Go to: https://github.com/settings/apps/new
#    (or https://github.com/organizations/YOUR-ORG/settings/apps/new for an org)
#
# 2. Fill in the form:
#    GitHub App name:  "My Konflux CI"  (must be globally unique on GitHub)
#    Homepage URL:     https://localhost:9443  (value does not matter)
#    Webhook:          Active = YES (toggle on)
#    Webhook URL:      Paste the value of $WEBHOOK_URL printed by STEP A2 above.
#                      It looks like:
#                      https://pipelines-as-code-controller-openshift-pipelines.apps.<cluster-domain>
#                      โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
#                      If the cluster is NOT internet-reachable from GitHub (e.g. behind a firewall
#                      or a private VPN), use a smee.io channel URL instead:
#                        SMEE_CHANNEL=$(head -c 30 /dev/random | base64 | tr -dc 'a-zA-Z0-9')
#                        echo "https://smee.io/$SMEE_CHANNEL"
#                      Then re-run deploy-konflux-on-ocp.sh with:
#                        SMEE_CHANNEL="https://smee.io/$SMEE_CHANNEL" ./deploy-konflux-on-ocp.sh
#    Webhook secret:   Paste the value of $WEBHOOK_SECRET from STEP A above.
#
# 3. Repository permissions โ€” set these exactly:
#    Checks:           Read & Write
#    Contents:         Read & Write
#    Issues:           Read & Write
#    Metadata:         Read-only (required, auto-selected)
#    Pull requests:    Read & Write
#    Commit statuses:  Read & Write
#
# 4. Subscribe to events โ€” check all of these:
#    โœ… Check run
#    โœ… Commit comment
#    โœ… Issue comment
#    โœ… Pull request
#    โœ… Push
#
# 5. Where can this GitHub App be installed? โ†’ Any account
#
# 6. Click "Create GitHub App"
#    โ†’ Note the numeric App ID shown on the next page (e.g. 123456)
#
# 7. Scroll down โ†’ "Private keys" โ†’ "Generate a private key"
#    โ†’ A .pem file downloads automatically โ€” note its path
#
# 8. Install the App on your repositories:
#    โ†’ Left sidebar: "Install App" โ†’ your org/user โ†’ All repositories
#    (or select specific repos)

# โ”€โ”€ STEP C: Deploy the secret to ALL 3 required namespaces โ”€โ”€โ”€โ”€
# Source: https://raw.githubusercontent.com/konflux-ci/konflux-ci/refs/heads/main/scripts/deploy-secrets.sh
#
# This block mirrors scripts/deploy-secrets.sh create_github_integration_secrets()
# exactly โ€” same flag syntax, same two-path conditional, same secret key names.
#
# On OCP (USE_OPENSHIFT_PIPELINES=true):
#   pac_ns is set to "openshift-pipelines", not "pipelines-as-code".
#   The loop targets: openshift-pipelines  build-service  integration-service

GITHUB_APP_ID="123456"                          # replace with your App ID from Step B.6
GITHUB_PRIVATE_KEY_PATH="/path/to/github-app.pem"  # path to the .pem from Step B.7
# WEBHOOK_SECRET is already set from STEP A above

# The official script has two code paths depending on how the private key is provided:
#   Path A โ€” key is a file on disk (GITHUB_PRIVATE_KEY_PATH set + file exists)
#   Path B โ€” key is passed as a literal string (e.g. from a CI env var: GITHUB_PRIVATE_KEY)
# Match this exactly so the secret data format is identical to what the script produces.

for ns in openshift-pipelines build-service integration-service; do
  echo "Creating secret in ${ns}..."

  if [ -n "${GITHUB_PRIVATE_KEY_PATH:-}" ] && [ -f "${GITHUB_PRIVATE_KEY_PATH}" ]; then
    # Path A: read private key from a .pem file (most common for interactive installs)
    oc -n "$ns" create secret generic pipelines-as-code-secret \
      --from-file=github-private-key="$GITHUB_PRIVATE_KEY_PATH" \
      --from-literal github-application-id="$GITHUB_APP_ID" \
      --from-literal webhook.secret="$WEBHOOK_SECRET" \
      --dry-run=client -o yaml | oc apply -f -
  else
    # Path B: key content is already in the GITHUB_PRIVATE_KEY env var (CI pipelines)
    oc -n "$ns" create secret generic pipelines-as-code-secret \
      --from-literal github-private-key="$GITHUB_PRIVATE_KEY" \
      --from-literal github-application-id="$GITHUB_APP_ID" \
      --from-literal webhook.secret="$WEBHOOK_SECRET" \
      --dry-run=client -o yaml | oc apply -f -
  fi
done

# Verify the secret exists in all 3 namespaces
echo ""
echo "Secret verification:"
for ns in openshift-pipelines build-service integration-service; do
  echo -n "  $ns โ€” App ID: "
  oc get secret pipelines-as-code-secret -n $ns \
    -o jsonpath='{.data.github-application-id}' 2>/dev/null | base64 -d \
    || echo '(missing!)'
  echo ""
done

# โ”€โ”€ STEP D: image-controller Quay secret (optional) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Required ONLY when spec.imageController.enabled: true in the Konflux CR.
# image-controller auto-creates Quay.io repos when you onboard Components
# via the Konflux UI. Skip entirely if you create Components with oc.
#
# โš  This step is NOT covered here โ€” refer to the official documentation:
#
#   quay.io (cloud):
#   https://konflux-ci.dev/konflux-ci/docs/guides/registry-configuration/#quayio-auto-provisioning-image-controller
#
#   Self-hosted Quay registry:
#   https://konflux-ci.dev/konflux-ci/docs/guides/registry-configuration/#self-hosted-quay-registry
Phase 4 โ€” Configure Quay.io registry credentials & tenant namespace
# Ref: https://konflux-ci.dev/docs/installation/registry-configuration/
#
# SEQUENCING:
#   Part A (NOW โ€” before Phase 5): verify tenant namespace + create regcred secret
#   Part B (AFTER Phase 5):        patch regcred onto the Component ServiceAccount
#                                  (the SA doesn't exist until Build Service reconciles
#                                   the Component CR, so Part B must come after)

# โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
# PART A โ€” Run before Phase 5
# โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•

# โ”€โ”€ Step 1: Verify the tenant namespace โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
NS="default-tenant"   # or your custom namespace name

oc get namespace "$NS" --show-labels
# Required label: konflux-ci.dev/type=tenant
# If it's missing, label it:
#   oc label namespace "$NS" konflux-ci.dev/type=tenant

# To create a brand-new custom tenant namespace:
# oc create namespace my-team-tenant
# oc label namespace my-team-tenant \
#   konflux-ci.dev/type=tenant \
#   pod-security.kubernetes.io/audit=baseline \
#   pod-security.kubernetes.io/audit-version=latest \
#   pod-security.kubernetes.io/warn=baseline \
#   pod-security.kubernetes.io/warn-version=latest

# โ”€โ”€ Step 2: Create the regcred push secret โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# For how to obtain registry credentials for quay.io, Docker Hub, or any
# other OCI-compatible registry, refer to the official documentation:
#
#   https://konflux-ci.dev/konflux-ci/docs/guides/registry-configuration/#obtaining-registry-credentials
#
# Once you have your credentials as a Docker config JSON file, create the secret:

REGISTRY_AUTH_JSON="/path/to/your/auth.json"   # path to your Docker config JSON file

# Delete first to avoid "field is immutable" error on re-runs
oc delete secret regcred -n "$NS" --ignore-not-found

oc create secret generic regcred \
  --from-file=.dockerconfigjson="${REGISTRY_AUTH_JSON}" \
  --type=kubernetes.io/dockerconfigjson \
  -n "$NS"

# โ”€โ”€ Label the secret so Build Service links it to all Component SAs automatically โ”€โ”€
# The label below marks regcred as a "common secret" for the tenant namespace.
# Build Service watches for this label and automatically mounts the secret onto
# every build-pipeline-<component-name> ServiceAccount it creates, so you do
# NOT need to manually patch each SA after onboarding a new Component (Phase 5 Part B).
oc label secret regcred \
  -n "$NS" \
  build.appstudio.openshift.io/common-secret=true

# โ”€โ”€ Step 3: Confirm the secret was created correctly โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
oc get secret regcred -n "$NS"
oc get secret regcred -n "$NS" \
  -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d | python3 -m json.tool

# Confirm the label is present
oc get secret regcred -n "$NS" --show-labels

Onboarding Your First Application

Steps

Phase 5 โ€” Onboard a sample application (manifest approach)
## Step 1: Fork https://github.com/konflux-ci/testrepo (click Fork in GitHub)
##
## โš  IMPORTANT โ€” GitHub App installation on the forked repo:
##   In Phase 3 Step B.8 you installed the GitHub App. If you chose
##   "Only select repositories" (rather than "All repositories") the App
##   is NOT installed on your new fork and PaC webhooks will be silently ignored.
##
##   To add the fork now:
##     GitHub โ†’ Settings โ†’ Applications โ†’ <Your App Name> โ†’ Configure
##     โ†’ Repository access โ†’ "Only select repositories"
##     โ†’ Add your fork โ†’ Save
##
##   Alternatively, switch the App to "All repositories" to avoid this for
##   every future fork or new repository you onboard to Konflux.
##
## Step 2: Confirm the GitHub App is installed on the fork (see note above)
## Step 3: Apply the Application and Component resources below

---
# The Application is a logical grouping of related Components
apiVersion: appstudio.redhat.com/v1alpha1
kind: Application
metadata:
  name: my-first-app
  namespace: default-tenant    # your tenant namespace
spec:
  displayName: My First Konflux Application

---
# The Component points to the source repository and target image
apiVersion: appstudio.redhat.com/v1alpha1
kind: Component
metadata:
  name: testrepo
  namespace: default-tenant
spec:
  application: my-first-app
  componentName: testrepo
  source:
    git:
      url: https://github.com/YOUR-USERNAME/testrepo.git  # your fork
      revision: main
  containerImage: quay.io/YOUR-ORG/testrepo    # where to push the image


## NOTE: Do NOT create a Repository CR manually.
## When the Build Service reconciles the Component CR above, it automatically:
##   1. Creates the pipelinesascode.tekton.dev/v1alpha1 Repository object
##   2. Opens a PR on your fork adding the .tekton/ pipeline definitions
## You will see this PR appear in your GitHub fork shortly after applying
## the Component. Merge it (or follow Phase 6 steps) to activate builds.
Phase 6 โ€” Merge the auto-generated PR and observe the first build
NS="default-tenant"

# โ”€โ”€ Step 1: Verify the Component was accepted by the Build Service โ”€โ”€
# The Build Service records its reconciliation result in the annotation
# build.appstudio.openshift.io/status (not in status.conditions).
# A successful reconciliation shows: {"pac":{"state":"enabled",...}}
oc get component testrepo -n "$NS" \
  -o jsonpath='{.metadata.annotations.build\.appstudio\.openshift\.io/status}' \
  | python3 -m json.tool

# โ”€โ”€ Step 2: Verify the Repository CR was auto-created โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# The Build Service creates the pipelinesascode.tekton.dev/v1alpha1 Repository
# object automatically โ€” you do NOT need to create it manually.
oc get repository -n "$NS"

# The NAME column from the command above is what you pass to describe.
# It typically matches the Component name but use the exact value shown.
REPO_NAME=$(oc get repository -n "$NS" -o jsonpath='{.items[0].metadata.name}')
echo "Repository name: $REPO_NAME"
oc describe repository "$REPO_NAME" -n "$NS"

# โ”€โ”€ Step 3: Find the auto-generated PR in your GitHub fork โ”€โ”€โ”€โ”€โ”€
# Shortly after the Component CR was applied, the Build Service (via PaC)
# opened a PR on your fork adding a .tekton/ directory with two pipeline files:
#   .tekton/testrepo-pull-request.yaml  โ†’ runs on every PR opened/updated
#   .tekton/testrepo-push.yaml          โ†’ runs on every merge to main
#
# Go to your fork on GitHub and look for this open PR.
# It will be titled something like: "Add Konflux CI pipelines"
# URL: https://github.com/YOUR-USERNAME/testrepo/pulls
#
# IMPORTANT: Make sure the PR base is set to YOUR fork's main branch,
# not the upstream konflux-ci/testrepo. GitHub sometimes defaults to the
# upstream โ€” change it before merging.

# โ”€โ”€ Step 4: Wait for the pull-request PipelineRun to complete โ”€โ”€
# Opening the auto-generated PR is itself a pull_request event. PaC
# immediately triggers the pull-request pipeline (.tekton/testrepo-pull-request.yaml)
# against the PR branch. This is a lighter build (no image push, no signing)
# that verifies the component builds successfully before the PR is merged.
#
# Watch for the pull-request PipelineRun to appear (may take 10-30 seconds
# after the PR is opened on GitHub):
oc get pipelinerun -n "$NS" -w
# Look for a PipelineRun whose name contains "pull-request" or "on-pull-request"

# Follow its logs:
tkn pipelinerun logs --last -f -n "$NS"

# Wait until the PipelineRun shows Succeeded before proceeding:
oc get pipelinerun -n "$NS" \
  --sort-by=.metadata.creationTimestamp \
  -o custom-columns="NAME:.metadata.name,STATUS:.status.conditions[0].reason,STARTED:.metadata.creationTimestamp"
# STATUS must be "Succeeded" โ€” do NOT merge the PR until all tasks pass

# The GitHub PR will also show a green check from PaC once the pipeline passes.
# Only proceed to Step 5 when you see the check mark on the PR.

# โ”€โ”€ Step 5: Approve and merge the PR โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Once the pull-request PipelineRun has Succeeded and the GitHub check is green,
# review the generated .tekton/ pipeline files, then merge the PR.
#
# Go to: https://github.com/YOUR-USERNAME/testrepo/pulls
#   โ†’ Open the auto-generated PR
#   โ†’ Confirm all checks pass (green โœ“ from PaC)
#   โ†’ Click "Merge pull request" โ†’ "Confirm merge"
#
# DO NOT merge if the pull-request PipelineRun is still running or has failed โ€”
# the pipeline files may have an issue that needs to be resolved first.

# โ”€โ”€ Step 6: After merging the PR โ€” the push pipeline triggers โ”€โ”€
# Merging the PR is a push event to main. PaC delivers the webhook to the
# cluster and creates a NEW PipelineRun for the push pipeline automatically.
# This is the FULL build pipeline (clone โ†’ build โ†’ scan โ†’ sign โ†’ push image).
# Watch for it to appear (may take 10-30 seconds after the merge):
oc get pipelinerun -n "$NS" -w
# Look for a second PipelineRun whose name contains "push" or "on-push"

# โ”€โ”€ Step 7: Follow the push build logs โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
tkn pipelinerun logs --last -f -n "$NS"

# โ”€โ”€ Step 8: Verify the build results โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# After the push pipeline succeeds the Integration Service creates a Snapshot
oc get snapshot -n "$NS"

# Extract the built image URL and digest from the PipelineRun results
PR_NAME=$(oc get pipelinerun -n "$NS" \
  --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{.items[-1].metadata.name}')

IMAGE_URL=$(oc get pipelinerun "$PR_NAME" -n "$NS" \
  -o jsonpath='{.status.results[?(@.name=="IMAGE_URL")].value}')
IMAGE_DIGEST=$(oc get pipelinerun "$PR_NAME" -n "$NS" \
  -o jsonpath='{.status.results[?(@.name=="IMAGE_DIGEST")].value}')

echo "Built image: ${IMAGE_URL}@${IMAGE_DIGEST}"

Konflux Build Pipeline Deep Dive โ€” Every Task, Every Result, Every Secret

What You'll Learn

Every task in the default Konflux build pipeline and its purpose
How the build pipeline is fetched as an OCI bundle at runtime
How SBOM generation with Syft integrates into the pipeline
How ClamAV, SAST, and Clair vulnerability scanning tasks work
How Tekton Chains automatically signs the final image
Debugging a failed build step-by-step

Build Pipeline Task Map

Default Konflux Build Pipeline โ€” Task Execution Order (as seen on OCP Console)

Sequential โ€” each task runs after the previous completes
init
validate params & secrets
โ†’
clone-repository
git clone โ†’ workspace PVC
โ†’
prefetch-dependencies
hermetic dep cache (gomod/pip/npm)
โ†’
build-container
Buildah โ†’ IMAGE_DIGEST result
โ†’
build-image-index
create multi-arch manifest index
Parallel โ€” all six run simultaneously after build-image-index
deprecated-base-image-check
flags EOL base images
clamav-scan
malware / virus signature scan
sast-shell-check
shell script static analysis (ShellCheck)
sast-unicode-check
detects Unicode trojan-source attacks
rpms-signature-scan
verifies RPM package signatures
tpa-scan
Trustification / SBOM analysis
After PipelineRun completes โ€” Tekton Chains (not a pipeline task)
Tekton Chains
signs image + generates SLSA in-toto attestation
passive observer โ€” triggered by IMAGE_DIGEST result

Steps

  1. Trigger a build โ€” push any change to your testrepo fork's main branch, then open the OpenShift console and navigate to Pipelines โ†’ PipelineRuns to see the task DAG begin executing.
  2. Follow each task's logs in sequence โ€” observe init, clone-repository, prefetch-dependencies, build-container, and build-image-index running one after the other, then watch clamav-scan, sast-shell-check, sast-unicode-check, deprecated-base-image-check, rpms-signature-scan, and tpa-scan run concurrently in the post-build fan-out.
  3. Extract the IMAGE_DIGEST result โ€” once the PipelineRun completes, run the oc get pipelinerun -o jsonpath command from the Commands section to capture the exact image digest that Tekton Chains will sign.
  4. Download and inspect the SBOM โ€” run cosign download sbom against the built image to retrieve the Syft-generated SBOM and confirm it lists your image's packages.
  5. Verify the image signature โ€” run cosign verify with the cluster's public key to confirm Tekton Chains signed the image after the PipelineRun completed.
  6. Practice the debugging workflow โ€” introduce a deliberate syntax error in the Containerfile, push, and trace the failure through the build-container task logs to understand how broken builds surface in Konflux.
  7. Recover the build โ€” revert the syntax error, push again, and confirm the pipeline returns to a passing state and creates a new Snapshot.

Commands

terminal โ€” inspecting build results
NAMESPACE="your-username-tenant"

# โ”€โ”€ Watch build pipeline tasks in real time โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
tkn pipelinerun logs --last -f -n $NAMESPACE

# โ”€โ”€ Describe the PipelineRun โ€” see all task results โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
tkn pipelinerun describe --last -n $NAMESPACE

# โ”€โ”€ Extract the IMAGE_DIGEST result from the PipelineRun โ”€โ”€โ”€โ”€โ”€โ”€โ”€
PR_NAME=$(oc get pipelinerun -n $NAMESPACE \
  --sort-by=.metadata.creationTimestamp -o jsonpath='{.items[-1].metadata.name}')

IMAGE_URL=$(oc get pipelinerun $PR_NAME -n $NAMESPACE \
  -o jsonpath='{.status.results[?(@.name=="IMAGE_URL")].value}')

IMAGE_DIGEST=$(oc get pipelinerun $PR_NAME -n $NAMESPACE \
  -o jsonpath='{.status.results[?(@.name=="IMAGE_DIGEST")].value}')

echo "IMAGE_URL:    $IMAGE_URL"
echo "IMAGE_DIGEST: $IMAGE_DIGEST"

# โ”€โ”€ Inspect the SBOM attached to the image โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Install cosign if needed
brew install cosign   # macOS
# go install github.com/sigstore/cosign/v2/cmd/cosign@latest

# NOTE: cosign uses ~/.docker/config.json to authenticate to the registry when
# pulling the image manifest and attached artifacts (SBOM, attestations).
# Ensure this file exists with valid credentials for the registry where the image
# was pushed. If missing or stale, cosign will fail with an auth error.
#   docker login quay.io          # for quay.io
#   podman login quay.io          # alternative; writes to the same config path
# The file must contain an entry for the registry host used in $IMAGE_URL.

# โ”€โ”€ Download the SBOM generated by the Konflux build pipeline โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
#
# Konflux always uploads the SBOM via `cosign attach sbom` (the upload-sbom
# pipeline step). This is the deprecated API in cosign 2.x but is still fully
# functional โ€” use Option A below, which works with all Konflux installations.
#
# Additionally, when keyless signing (Fulcio + Rekor) is configured on the
# cluster, the pipeline also stores the SBOM as a proper cosign attestation via
# `cosign attest --type spdxjson`. Use Option B to retrieve it in that case.
# You can tell keyless signing is active if you see Rekor/Fulcio URLs in the
# cluster-config ConfigMap: oc get cm cluster-config -n konflux-info -o yaml
#
# Option A โ€” cosign attach sbom path (works on all Konflux clusters):
cosign download sbom "${IMAGE_URL}@${IMAGE_DIGEST}" | python3 -m json.tool | head -60

# Option B โ€” cosign attest path (only when keyless signing is enabled):
# predicateType will be https://spdx.dev/Document (spdxjson) or a CycloneDX URL.
cosign download attestation "${IMAGE_URL}@${IMAGE_DIGEST}" \
  | jq -r 'select(.payload != null) | .payload | @base64d | fromjson
            | select(
                .predicateType == "https://spdx.dev/Document" or
                (.predicateType | startswith("https://cyclonedx.org") or startswith("https://spdx.dev"))
              ) | .predicate' \
  | head -60

# โ”€โ”€ Verify the image signature from Tekton Chains โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# You need the public key from the Chains signing secret
oc get secret signing-secrets -n openshift-pipelines \
  -o jsonpath='{.data.cosign\.pub}' | base64 -d > /tmp/cosign.pub

# WHY --insecure-ignore-tlog IS REQUIRED HERE:
#
# Tekton Chains signs images using the cluster's own keypair stored in
# signing-secrets. On a self-hosted Konflux installation the signatures are
# recorded in the cluster's internal Rekor instance (or no transparency log
# at all), NOT in the public Sigstore Rekor at rekor.sigstore.dev.
#
# By default cosign verify tries to look up the signature in rekor.sigstore.dev
# to confirm the transparency log entry โ€” this will always fail for cluster-
# signed images because no entry was ever written there.
#
# --insecure-ignore-tlog tells cosign to verify only the cryptographic
# signature against the public key and skip the Rekor lookup entirely.
# This is the correct approach for self-hosted Konflux; the "insecure" label
# is a cosign convention meaning "no transparency log required", not that
# the signature itself is weaker.
#
# Additional note: if you are behind a corporate or ISP proxy (e.g. Airtel)
# that performs TLS inspection, the Rekor request would also fail with an
# x509 certificate error even if the endpoint were correct. The flag bypasses
# that network issue as well.
cosign verify \
  --key /tmp/cosign.pub \
  --insecure-ignore-tlog \
  "${IMAGE_URL}@${IMAGE_DIGEST}"

# โ”€โ”€ Verify the SLSA provenance attestation โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Same reasoning applies: the attestation was recorded by Tekton Chains
# against the cluster's internal Rekor, not the public one. --insecure-ignore-tlog
# skips the transparency log lookup and verifies the attestation signature only.
cosign verify-attestation \
  --key /tmp/cosign.pub \
  --insecure-ignore-tlog \
  --type slsaprovenance \
  "${IMAGE_URL}@${IMAGE_DIGEST}" | jq '.payload | @base64d | fromjson'

# โ”€โ”€ Debugging a failed task โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Find which TaskRun failed
oc get taskruns -n $NAMESPACE \
  --selector=tekton.dev/pipelineRun=$PR_NAME \
  '--sort-by=.status.conditions[-1].lastTransitionTime'

# Get logs of the specific failed TaskRun
FAILED_TR=$(oc get taskruns -n $NAMESPACE \
  --selector=tekton.dev/pipelineRun=$PR_NAME \
  -o jsonpath='{.items[?(@.status.conditions[0].reason=="Failed")].metadata.name}')

tkn taskrun logs $FAILED_TR -n $NAMESPACE

# If the TaskRun pod is stuck in Pending:
oc describe pod -n $NAMESPACE \
  --selector=tekton.dev/taskRun=$FAILED_TR

# Check Tekton Chains controller logs (if signing failed)
# On OCP, Tekton Chains runs inside openshift-pipelines (not a separate tekton-chains namespace)
oc logs -n openshift-pipelines \
  -l app=tekton-chains-controller --tail=50

All available Konflux pipeline definitions

The Konflux team publishes every supported pipeline variant โ€” docker-build, fbc-builder, multi-platform, java-builder, and more โ€” in the canonical build-definitions repository. Browsing these files is the fastest way to understand what tasks each pipeline includes, what params it accepts, and which OCI bundle digest to pin in your pipelineRef.

github.com/konflux-ci/build-definitions/tree/main/pipelines

Key Takeaways

  • IMAGE_DIGEST result from build-container is the lynchpin
  • Scans (Clair, SAST, ClamAV) run in parallel after the build
  • SBOM is generated by Syft and attached as OCI artifact
  • Tekton Chains signs after PipelineRun โ€” not a pipeline task
  • Use tkn pipelinerun describe first when debugging
  • Pipeline definition is fetched from quay.io OCI bundle at runtime

Bundle-Based Build Pipelines โ€” Replacing Inline pipelineSpec with a Pinned OCI Bundle

What You'll Learn

What a Tekton pipeline bundle is and how it differs from an inline pipelineSpec
Using pipelineRef with the bundles resolver to reference a pipeline by OCI digest
Why pinning by digest (not :latest) matters for reproducibility and supply-chain security
Pros and cons of bundle-based vs inline pipelineSpec โ€” when to use each
The skip-checks parameter โ€” what it skips and when it is needed (arm64 / CRC environments)
Key differences between the push pipeline and the pull-request pipeline

Steps

  1. Understand what the bundle contains โ€” the pipeline-docker-build-oci-ta bundle is a complete Pipeline resource packaged as an OCI image. It includes init, clone, prefetch, buildah, build-image-index, SAST, Clair, ClamAV, deprecated-image-check, rpms-signature-scan, apply-tags, and push-dockerfile tasks. No checks are dropped compared to the auto-generated inline pipeline.
  2. Copy the bundle-based pipeline files โ€” copy testrepo-push.yaml and testrepo-pull-request.yaml from the YAML Files section into your testrepo fork's .tekton/ directory, replacing the auto-generated files from onboarding.
  3. Replace the placeholders โ€” update YOUR-USERNAME with your GitHub username and YOUR-ORG with your quay.io org or username in both files.
  4. Set skip-checks: "true" if running on CRC arm64 โ€” the post-build scan task images are built amd64-only and cannot be pulled on a Silicon Mac CRC node. Remove this param when moving to a full amd64 cluster or the hosted Konflux service.
  5. Commit both files and push to main โ€” PaC picks up the updated .tekton/ files and resolves the pipeline from the OCI bundle on the next push event.
  6. Confirm the PipelineRun results โ€” verify that IMAGE_URL and IMAGE_DIGEST results are populated and that the same stages (build โ†’ scan โ†’ index) complete as with the previous inline pipeline.

Bundle vs Inline pipelineSpec โ€” Pros and Cons

Aspect Bundle (pipelineRef) Inline (pipelineSpec)
File size ~50 lines ~230 lines
Task visibility Hidden inside bundle โ€” you must inspect the OCI image to see tasks Fully visible in the file โ€” every task and param is readable
Custom tasks Not possible โ€” Tekton does not allow appending tasks to a pipelineRef Fully supported โ€” add any inline taskSpec or taskRef
Reproducibility Pinned by digest โ€” exact same pipeline runs every time Each task bundle reference must be pinned individually
Upgrades Update one digest line to get a new pipeline version Renovate/Mintmaker must update each task bundle digest separately
Per-task overrides taskRunSpecs can override resources and pod template per task Full control โ€” modify any task param, image, or when condition
Best for Standard builds with no custom tasks; teams that want minimal YAML Custom steps, learning, or skipping specific individual tasks

YAML Files

About skip-checks

The skip-checks param is built into the Konflux pipeline bundle. When set to "true", all post-build scan and certification tasks (Clair, ClamAV, SAST shell-check, SAST unicode-check, deprecated-image-check, rpms-signature-scan, ecosystem-cert-preflight-checks) are skipped via their internal when conditions. The build, clone, prefetch, and index tasks are not affected.

When to use it: on CRC (OpenShift Local) running on a Silicon Mac (arm64), the task images for these checks are built amd64-only. Pulling them on an arm64 node produces no image found in manifest list for architecture "arm64", causing the task pod to hang then fail. Setting skip-checks: "true" is the designed escape hatch for this situation. Remove it when deploying on a full cluster or the hosted Konflux service.

.tekton/testrepo-push.yaml (bundle-based push pipeline โ€” replaces the auto-generated inline pipelineSpec)
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  annotations:
    # Replace YOUR-USERNAME with your GitHub username (owner of your testrepo fork)
    build.appstudio.openshift.io/repo: https://github.com/YOUR-USERNAME/testrepo?rev={{revision}}
    build.appstudio.redhat.com/commit_sha: '{{revision}}'
    build.appstudio.redhat.com/target_branch: '{{target_branch}}'
    # cancel-in-progress: false โ€” a new push to main does NOT cancel an in-flight push run.
    # Push builds produce production images; letting them finish ensures the Snapshot
    # and integration tests always correspond to a completed, signed image.
    pipelinesascode.tekton.dev/cancel-in-progress: "false"
    pipelinesascode.tekton.dev/max-keep-runs: "3"
    pipelinesascode.tekton.dev/on-cel-expression: event == "push" && target_branch == "main"
  labels:
    appstudio.openshift.io/application: my-first-app
    appstudio.openshift.io/component: testrepo
    pipelines.appstudio.openshift.io/type: build
  name: testrepo-on-push
  namespace: default-tenant
spec:
  params:
  - name: git-url
    value: '{{source_url}}'
  - name: revision
    value: '{{revision}}'
  # Replace YOUR-ORG with your quay.io username or org.
  # Must match the containerImage field in your Component CR.
  - name: output-image
    value: quay.io/YOUR-ORG/testrepo:{{revision}}
  - name: dockerfile
    value: Dockerfile
  # โ”€โ”€ skip-checks โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  # Set to "true" on CRC arm64 (Silicon Mac) โ€” the post-build scan task images
  # (Clair, ClamAV, SAST, ecosystem-cert-preflight-checks) are built amd64-only
  # and cannot be pulled on an arm64 node. This skips ALL post-build checks via
  # the when conditions built into the bundle pipeline.
  # Remove this param (or set to "false") on a full amd64 cluster or hosted Konflux.
  - name: skip-checks
    value: "true"

  # โ”€โ”€ Pipeline bundle โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  # References the full pipeline-docker-build-oci-ta pipeline as an OCI bundle,
  # pinned by digest for reproducibility. The bundle includes all standard tasks:
  # init, clone, prefetch, buildah, build-image-index, SAST, Clair, ClamAV,
  # deprecated-image-check, rpms-signature-scan, apply-tags, push-dockerfile.
  #
  # To add custom tasks (e.g. print-build-summary), switch to the inline
  # pipelineSpec approach shown in item 17 โ€” pipelineRef does not support
  # appending tasks (Tekton constraint).
  pipelineRef:
    resolver: bundles
    params:
    - name: bundle
      value: quay.io/konflux-ci/tekton-catalog/pipeline-docker-build-oci-ta@sha256:49d8470b97d44f0e557b73d50e7f96c21238bcf66b603e7450ed4927842036ce
    - name: name
      value: docker-build-oci-ta
    - name: kind
      value: pipeline

  taskRunTemplate:
    serviceAccountName: build-pipeline-testrepo
  workspaces:
  - name: git-auth
    secret:
      secretName: '{{ git_auth_secret }}'
status: {}
.tekton/testrepo-pull-request.yaml (bundle-based PR pipeline โ€” cancel-in-progress, 5d expiry, skip-checks for arm64 CRC)
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  annotations:
    # Replace YOUR-USERNAME with your GitHub username (owner of your testrepo fork)
    build.appstudio.openshift.io/repo: https://github.com/YOUR-USERNAME/testrepo?rev={{revision}}
    build.appstudio.redhat.com/commit_sha: '{{revision}}'
    build.appstudio.redhat.com/pull_request_number: '{{pull_request_number}}'
    build.appstudio.redhat.com/target_branch: '{{target_branch}}'
    # cancel-in-progress: true โ€” a new push to the same PR cancels the prior run.
    # PR builds are disposable feedback loops; there is no point running an old
    # commit once a newer one has been pushed to the same branch.
    pipelinesascode.tekton.dev/cancel-in-progress: "true"
    pipelinesascode.tekton.dev/max-keep-runs: "3"
    pipelinesascode.tekton.dev/on-cel-expression: event == "pull_request" && target_branch == "main"
  labels:
    appstudio.openshift.io/application: my-first-app
    appstudio.openshift.io/component: testrepo
    pipelines.appstudio.openshift.io/type: build
  name: testrepo-on-pull-request
  namespace: default-tenant
spec:
  params:
  - name: git-url
    value: '{{source_url}}'
  - name: revision
    value: '{{revision}}'
  # Replace YOUR-ORG with your quay.io username or org.
  # The on-pr- prefix and short expiry mark this as a disposable PR image โ€”
  # it is NOT promoted by the release pipeline.
  - name: output-image
    value: quay.io/YOUR-ORG/testrepo:on-pr-{{revision}}
  # PR images expire after 5 days โ€” they exist only for local testing and
  # integration-test feedback. Push images have no expiry by default.
  - name: image-expires-after
    value: 5d
  - name: dockerfile
    value: Dockerfile
  # โ”€โ”€ skip-checks โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  # Same reasoning as the push pipeline: required on CRC arm64 (Silicon Mac)
  # because the scan task images are amd64-only. Remove on a full cluster.
  - name: skip-checks
    value: "true"

  # โ”€โ”€ Pipeline bundle โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  # Same pinned bundle as the push pipeline. All standard tasks are included.
  # The bundle handles skip-checks internally via when conditions on each task โ€”
  # no manual task list editing required.
  #
  # To add the pr-build-summary custom task shown in item 17, switch to the
  # inline pipelineSpec approach โ€” pipelineRef does not support appending tasks.
  pipelineRef:
    resolver: bundles
    params:
    - name: bundle
      value: quay.io/konflux-ci/tekton-catalog/pipeline-docker-build-oci-ta@sha256:49d8470b97d44f0e557b73d50e7f96c21238bcf66b603e7450ed4927842036ce
    - name: name
      value: docker-build-oci-ta
    - name: kind
      value: pipeline

  taskRunTemplate:
    serviceAccountName: build-pipeline-testrepo
  workspaces:
  - name: git-auth
    secret:
      secretName: '{{ git_auth_secret }}'
status: {}

Commands

terminal
NAMESPACE="default-tenant"

# In your local testrepo fork clone, replace the auto-generated .tekton/ files
# with the bundle-based versions above (update YOUR-USERNAME and YOUR-ORG first):
cp /path/to/testrepo-push.yaml         .tekton/testrepo-push.yaml
cp /path/to/testrepo-pull-request.yaml .tekton/testrepo-pull-request.yaml

git add .tekton/testrepo-push.yaml .tekton/testrepo-pull-request.yaml
git commit -m "chore: switch to bundle-based pipeline (pipeline-docker-build-oci-ta)"
git push origin main

# Watch the PipelineRun triggered by the push
oc get pipelinerun -n $NAMESPACE -w

# Confirm the bundle resolver fetched the pipeline correctly
PR_NAME=$(oc get pipelinerun -n $NAMESPACE \
  --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{.items[-1].metadata.name}')
oc describe pipelinerun "$PR_NAME" -n $NAMESPACE | grep -A5 "Pipeline Ref"

# Follow logs
tkn pipelinerun logs "$PR_NAME" -n $NAMESPACE -f

# Check which tasks were skipped (when skip-checks: "true")
oc get pipelinerun "$PR_NAME" -n $NAMESPACE \
  -o jsonpath='{.status.skippedTasks[*].name}' | tr ' ' '\n'

# When running on a full cluster โ€” remove skip-checks and verify all tasks run:
# Remove the skip-checks param block from both files, then:
git add .tekton/
git commit -m "chore: enable post-build checks (full cluster)"
git push origin main

Key Takeaways

  • A pipeline bundle is a complete Pipeline resource packaged as an OCI image โ€” referencing it via pipelineRef + resolver: bundles replaces hundreds of lines of inline YAML with a single digest
  • Always pin by digest (@sha256:โ€ฆ), never by tag โ€” tags are mutable; a digest guarantees the exact same pipeline runs every time
  • The skip-checks param is built into the Konflux bundle; it gates all post-build scan tasks via internal when conditions โ€” no manual task editing needed
  • On CRC arm64 (Silicon Mac), skip-checks: "true" is required โ€” the scan task images are amd64-only and cannot be pulled on an arm64 node
  • pipelineRef does not support appending custom tasks โ€” switch to inline pipelineSpec (item 17) if you need a print-build-summary or any other custom step
  • Push pipeline: cancel-in-progress: false โ€” production images must complete. PR pipeline: cancel-in-progress: true โ€” disposable feedback loops are safe to cancel

Pipeline as Code โ€” Customizing Your Konflux Build Pipeline

What You'll Learn

How Pipeline as Code (PaC) works โ€” events, annotations, resolution
Customizing the push pipeline: adding params, tweaking tasks
Adding a custom Task to the generated pipeline (e.g., go vet, helm lint)
Skipping non-critical checks with skip-checks parameter
Separate push vs. pull-request pipeline strategies
How PaC handles CEL expressions for event filtering

Steps

  1. Open both .tekton/ files and compare them โ€” note the key differences between push and pull-request pipelines: cancel-in-progress, image-expires-after, the on-cel-expression annotation, and the output image tag prefix (on-pr-).
  2. Add the print-build-summary custom task โ€” copy the inline taskSpec block from the YAML Files section into the push pipeline's task list, positioned after build-image-index with runAfter: [build-image-index].
  3. Add the pr-build-summary task to the pull-request pipeline โ€” use the corresponding taskSpec from the YAML Files section, which also prints the image-expires-after value to make the disposable nature of PR images explicit.
  4. Commit the updated .tekton/ files and push to main โ€” PaC reads the new pipeline definition from the repository and uses it for all subsequent PipelineRuns.
  5. Open a pull request to trigger the PR pipeline โ€” confirm that the pr-build-summary task appears in the PipelineRun logs with the correct image URL, digest, and expiry.
  6. Review the PaC annotations โ€” on-cel-expression controls which git events trigger the pipeline, cancel-in-progress determines whether concurrent runs on the same PR are cancelled, and max-keep-runs limits the number of stored PipelineRun objects per component.

YAML Files

.tekton/testrepo-push.yaml (customized push pipeline โ€” adds print-build-summary task to the default testrepo pipeline)
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  annotations:
    # Replace YOUR-USERNAME with your GitHub username (owner of your testrepo fork)
    build.appstudio.openshift.io/repo: https://github.com/YOUR-USERNAME/testrepo?rev={{revision}}
    build.appstudio.redhat.com/commit_sha: '{{revision}}'
    build.appstudio.redhat.com/target_branch: '{{target_branch}}'
    pipelinesascode.tekton.dev/cancel-in-progress: "false"
    pipelinesascode.tekton.dev/max-keep-runs: "3"
    pipelinesascode.tekton.dev/on-cel-expression: event == "push" && target_branch == "main"
  labels:
    # Must match the Application CR name created in Phase 5 (metadata.name of the Application)
    appstudio.openshift.io/application: my-first-app
    # Must match the Component CR name created in Phase 5 (metadata.name of the Component)
    appstudio.openshift.io/component: testrepo
    pipelines.appstudio.openshift.io/type: build
  # Conventionally -on-push; must be unique within the namespace
  name: testrepo-on-push
  # Your tenant namespace โ€” where the Application and Component CRs were created
  namespace: default-tenant
spec:
  params:
  - name: git-url
    value: '{{source_url}}'
  - name: revision
    value: '{{revision}}'
  - name: output-image
    # Replace YOUR-ORG with your quay.io username or org
    # Must match the containerImage field in your Component CR (Phase 5)
    value: quay.io/YOUR-ORG/testrepo:{{revision}}
  - name: dockerfile
    value: Dockerfile
  pipelineSpec:
    description: |
      Customized Konflux build pipeline for testrepo. Extends the default
      minimal OCI trusted-artifacts pipeline with a custom `print-build-summary`
      task (see inline taskSpec below). All standard scan tasks run in parallel
      after the build and are controlled by the skip-checks parameter. Scans
      are always enabled on the push (production) pipeline.
    params:
    - description: Source Repository URL
      name: git-url
      type: string
    - default: ""
      description: Revision of the Source Repository
      name: revision
      type: string
    - description: Fully Qualified Output Image
      name: output-image
      type: string
    - default: .
      description: Path to the source code of an application's component from where
        to build image.
      name: path-context
      type: string
    - default: Dockerfile
      description: Path to the Dockerfile inside the context specified by parameter
        path-context
      name: dockerfile
      type: string
    - default: "false"
      description: Skip checks against built image
      name: skip-checks
      type: string
    - default: "false"
      description: Execute the build with network isolation
      name: hermetic
      type: string
    - default: ""
      description: Build dependencies to be prefetched
      name: prefetch-input
      type: string
    - default: ""
      description: Image tag expiration time, time values could be something like
        1h, 2d, 3w for hours, days, and weeks, respectively.
      name: image-expires-after
      type: string
    - default: "false"
      description: Add built image into an OCI image index
      name: build-image-index
      type: string
    - default: docker
      description: The format for the resulting image's mediaType. Valid values are
        oci or docker.
      name: buildah-format
      type: string
    - default: "false"
      description: Enable cache proxy configuration
      name: enable-cache-proxy
    - default: "true"
      description: Use the package registry proxy when prefetching dependencies
      name: enable-package-registry-proxy
    - default: .
      description: Target directories in component's source code to scan with SAST
        tools. Multiple values should be separated with commas.
      name: sast-target-dirs
      type: string
    - default: []
      description: Array of --build-arg values ("arg=value" strings) for buildah
      name: build-args
      type: array
    - default: ""
      description: Path to a file with build arguments for buildah, see https://www.mankier.com/1/buildah-build#--build-arg-file
      name: build-args-file
      type: string
    - default: "false"
      description: Whether to enable privileged mode, should be used only with remote
        VMs
      name: privileged-nested
      type: string
    results:
    - description: ""
      name: IMAGE_URL
      value: $(tasks.build-image-index.results.IMAGE_URL)
    - description: ""
      name: IMAGE_DIGEST
      value: $(tasks.build-image-index.results.IMAGE_DIGEST)
    - description: ""
      name: CHAINS-GIT_URL
      value: $(tasks.clone-repository.results.url)
    - description: ""
      name: CHAINS-GIT_COMMIT
      value: $(tasks.clone-repository.results.commit)
    tasks:
    # โ”€โ”€ DEFAULT TASKS (unchanged from the generated pipeline) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    - name: init
      params:
      - name: enable-cache-proxy
        value: $(params.enable-cache-proxy)
      taskRef:
        params:
        - name: name
          value: init
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-init:0.4.2@sha256:421003a5c077ecb820460e71637125ec9093d2101c749a32ede28e190283e9db
        - name: kind
          value: task
        resolver: bundles
    - name: clone-repository
      params:
      - name: url
        value: $(params.git-url)
      - name: revision
        value: $(params.revision)
      - name: ociStorage
        value: $(params.output-image).git
      - name: ociArtifactExpiresAfter
        value: $(params.image-expires-after)
      runAfter:
      - init
      taskRef:
        params:
        - name: name
          value: git-clone-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-git-clone-oci-ta-min:0.2@sha256:83bffb5ef4589830c8d0f606325fb27e8cd680c8404d5a935f6660174c9e118c
        - name: kind
          value: task
        resolver: bundles
      workspaces:
      - name: basic-auth
        workspace: git-auth
    - name: prefetch-dependencies
      params:
      - name: input
        value: $(params.prefetch-input)
      - name: enable-package-registry-proxy
        value: $(params.enable-package-registry-proxy)
      - name: SOURCE_ARTIFACT
        value: $(tasks.clone-repository.results.SOURCE_ARTIFACT)
      - name: ociStorage
        value: $(params.output-image).prefetch
      - name: ociArtifactExpiresAfter
        value: $(params.image-expires-after)
      runAfter:
      - clone-repository
      taskRef:
        params:
        - name: name
          value: prefetch-dependencies-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-prefetch-dependencies-oci-ta-min:0.3.2@sha256:7f344093a3387d05eeedad1f929743e197212b50ee95ceaebdc74bbe5df05d03
        - name: kind
          value: task
        resolver: bundles
      workspaces:
      - name: git-basic-auth
        workspace: git-auth
      - name: netrc
        workspace: netrc
    - name: build-container
      params:
      - name: IMAGE
        value: $(params.output-image)
      - name: DOCKERFILE
        value: $(params.dockerfile)
      - name: CONTEXT
        value: $(params.path-context)
      - name: HERMETIC
        value: $(params.hermetic)
      - name: PREFETCH_INPUT
        value: $(params.prefetch-input)
      - name: IMAGE_EXPIRES_AFTER
        value: $(params.image-expires-after)
      - name: COMMIT_SHA
        value: $(tasks.clone-repository.results.commit)
      - name: BUILD_ARGS
        value:
        - $(params.build-args[*])
      - name: BUILD_ARGS_FILE
        value: $(params.build-args-file)
      - name: PRIVILEGED_NESTED
        value: $(params.privileged-nested)
      - name: SOURCE_URL
        value: $(tasks.clone-repository.results.url)
      - name: BUILDAH_FORMAT
        value: $(params.buildah-format)
      - name: HTTP_PROXY
        value: $(tasks.init.results.http-proxy)
      - name: NO_PROXY
        value: $(tasks.init.results.no-proxy)
      - name: SOURCE_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.SOURCE_ARTIFACT)
      - name: CACHI2_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)
      runAfter:
      - prefetch-dependencies
      taskRef:
        params:
        - name: name
          value: buildah-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-buildah-oci-ta-min:0.10@sha256:72c599425e0bda63c30e10127dc493d139c75e45b986d8b7627a0d1b94aafff8
        - name: kind
          value: task
        resolver: bundles
    - name: build-image-index
      params:
      - name: IMAGE
        value: $(params.output-image)
      - name: ALWAYS_BUILD_INDEX
        value: $(params.build-image-index)
      - name: IMAGES
        value:
        - $(tasks.build-container.results.IMAGE_URL)@$(tasks.build-container.results.IMAGE_DIGEST)
      - name: BUILDAH_FORMAT
        value: $(params.buildah-format)
      runAfter:
      - build-container
      taskRef:
        params:
        - name: name
          value: build-image-index-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-build-image-index-min:0.3@sha256:3279101e95f35b768877ae9e7620e92984bd72907e79ea5fefade39d56cc93a3
        - name: kind
          value: task
        resolver: bundles
    # โ”€โ”€ CUSTOM TASK: print-build-summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    # This task is NOT part of the default pipeline โ€” it demonstrates how to
    # add a custom step using an inline taskSpec (no separate Task CRD or OCI
    # bundle required). It runs in the post-build parallel fan-out alongside
    # the scan tasks and never blocks them.
    #
    # Key concepts shown here:
    #   taskSpec   โ€” embeds the Task definition directly in the PipelineRun
    #   runAfter   โ€” places this task after build-image-index in the DAG
    #   params     โ€” receives upstream task results via $(tasks..results.)
    #
    # To add your own step: copy this block, rename it, change the image and
    # script, and update runAfter to the task it depends on.
    - name: print-build-summary
      params:
      - name: IMAGE_URL
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: IMAGE_DIGEST
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: GIT_URL
        value: $(tasks.clone-repository.results.url)
      - name: GIT_COMMIT
        value: $(tasks.clone-repository.results.commit)
      runAfter:
      - build-image-index
      taskSpec:
        params:
        - name: IMAGE_URL
          type: string
        - name: IMAGE_DIGEST
          type: string
        - name: GIT_URL
          type: string
        - name: GIT_COMMIT
          type: string
        steps:
        - name: summarize
          image: registry.access.redhat.com/ubi9/ubi-minimal:latest
          script: |
            #!/bin/bash
            echo "=================================================="
            echo "           KONFLUX BUILD SUMMARY"
            echo "=================================================="
            printf '%-22s %s\n' "Source repository:"  "$(params.GIT_URL)"
            printf '%-22s %s\n' "Git commit:"         "$(params.GIT_COMMIT)"
            printf '%-22s %s\n' "Built image URL:"    "$(params.IMAGE_URL)"
            printf '%-22s %s\n' "Image digest:"       "$(params.IMAGE_DIGEST)"
            echo "=================================================="
            echo ""
            echo "Pull the image:"
            echo "  podman pull $(params.IMAGE_URL)@$(params.IMAGE_DIGEST)"
            echo ""
            echo "Verify the signature (Tekton Chains signs after PipelineRun):"
            echo "  cosign verify --key cosign.pub --insecure-ignore-tlog \\"
            echo "    $(params.IMAGE_URL)@$(params.IMAGE_DIGEST)"
    # โ”€โ”€ END CUSTOM TASK โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    - name: deprecated-base-image-check
      params:
      - name: IMAGE_URL
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: IMAGE_DIGEST
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: deprecated-image-check
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-deprecated-image-check:0.5@sha256:e78d0d3baf3c8cfc1a5ad278196b74032d9568b143a87c7a79ab780fedfb296e
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: clamav-scan
      params:
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: clamav-scan-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-clamav-scan-min:0.3@sha256:908e356d7a3bc3e472a9075a1ce1370486361007b3a86e3f61c2a6af7136a335
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: sast-shell-check
      params:
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: TARGET_DIRS
        value: $(params.sast-target-dirs)
      - name: SOURCE_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.SOURCE_ARTIFACT)
      - name: CACHI2_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: sast-shell-check-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-sast-shell-check-oci-ta-min:0.1@sha256:ecfae10944b45b91988ddc6311c7232bf2c98ba73c5fc6261861ecfd33434db0
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: sast-unicode-check
      params:
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: TARGET_DIRS
        value: $(params.sast-target-dirs)
      - name: SOURCE_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.SOURCE_ARTIFACT)
      - name: CACHI2_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: sast-unicode-check-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-sast-unicode-check-oci-ta-min:0.4@sha256:96badf0b06d83fc1e7cf50048f94091e257819ee537f063830541f9c97295200
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: rpms-signature-scan
      params:
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: rpms-signature-scan
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-rpms-signature-scan:0.2@sha256:65370ccb44ff82e4ce128addd913f3c96b298607b3760ee1339ed10011a4bd6b
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: tpa-scan
      params:
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: tpa-scan
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-tpa-scan:0.1@sha256:8375c9e4ee2ee417881120187be1281c50b85a8b23d8d24785e1dc96a3018c8e
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    workspaces:
    - name: git-auth
      optional: true
    - name: netrc
      optional: true
  taskRunTemplate:
    # Auto-created by Build Service as build-pipeline-
    # For a component named "testrepo" this is automatically build-pipeline-testrepo
    serviceAccountName: build-pipeline-testrepo
  workspaces:
  - name: git-auth
    secret:
      secretName: '{{ git_auth_secret }}'
status: {}
.tekton/testrepo-pull-request.yaml (PR pipeline โ€” cancel-in-progress, 5d image expiry, custom pr-build-summary task)
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  annotations:
    # Replace YOUR-USERNAME with your GitHub username (owner of your testrepo fork)
    build.appstudio.openshift.io/repo: https://github.com/YOUR-USERNAME/testrepo?rev={{revision}}
    build.appstudio.redhat.com/commit_sha: '{{revision}}'
    build.appstudio.redhat.com/pull_request_number: '{{pull_request_number}}'
    build.appstudio.redhat.com/target_branch: '{{target_branch}}'
    pipelinesascode.tekton.dev/cancel-in-progress: "true"
    pipelinesascode.tekton.dev/max-keep-runs: "3"
    pipelinesascode.tekton.dev/on-cel-expression: event == "pull_request" && target_branch == "main"
  labels:
    # Must match the Application CR name created in Phase 5 (metadata.name of the Application)
    appstudio.openshift.io/application: my-first-app
    # Must match the Component CR name created in Phase 5 (metadata.name of the Component)
    appstudio.openshift.io/component: testrepo
    pipelines.appstudio.openshift.io/type: build
  # Conventionally -on-pull-request; must be unique within the namespace
  name: testrepo-on-pull-request
  # Your tenant namespace โ€” where the Application and Component CRs were created
  namespace: default-tenant
spec:
  params:
  - name: git-url
    value: '{{source_url}}'
  - name: revision
    value: '{{revision}}'
  - name: output-image
    # Replace YOUR-ORG with your quay.io username or org
    # Must match the containerImage field in your Component CR (Phase 5)
    # The on-pr- prefix and short expiry mark this as a disposable PR image
    value: quay.io/YOUR-ORG/testrepo:on-pr-{{revision}}
  - name: image-expires-after
    value: 5d
  - name: dockerfile
    value: Dockerfile
  pipelineSpec:
    description: |
      Customized Konflux PR pipeline for testrepo. Extends the default minimal
      OCI trusted-artifacts pipeline with a custom `pr-build-summary` task.
      Key differences from the push pipeline:
        cancel-in-progress: true  โ€” a new push to the same PR cancels the prior run
        image-expires-after: 5d   โ€” PR images are disposable, not stored long-term
        skip-checks: false        โ€” scans still run; set to "true" for faster PRs
    params:
    - description: Source Repository URL
      name: git-url
      type: string
    - default: ""
      description: Revision of the Source Repository
      name: revision
      type: string
    - description: Fully Qualified Output Image
      name: output-image
      type: string
    - default: .
      description: Path to the source code of an application's component from where
        to build image.
      name: path-context
      type: string
    - default: Dockerfile
      description: Path to the Dockerfile inside the context specified by parameter
        path-context
      name: dockerfile
      type: string
    - default: "false"
      description: Skip checks against built image
      name: skip-checks
      type: string
    - default: "false"
      description: Execute the build with network isolation
      name: hermetic
      type: string
    - default: ""
      description: Build dependencies to be prefetched
      name: prefetch-input
      type: string
    - default: ""
      description: Image tag expiration time, time values could be something like
        1h, 2d, 3w for hours, days, and weeks, respectively.
      name: image-expires-after
      type: string
    - default: "false"
      description: Add built image into an OCI image index
      name: build-image-index
      type: string
    - default: docker
      description: The format for the resulting image's mediaType. Valid values are
        oci or docker.
      name: buildah-format
      type: string
    - default: "false"
      description: Enable cache proxy configuration
      name: enable-cache-proxy
    - default: "true"
      description: Use the package registry proxy when prefetching dependencies
      name: enable-package-registry-proxy
    - default: .
      description: Target directories in component's source code to scan with SAST
        tools. Multiple values should be separated with commas.
      name: sast-target-dirs
      type: string
    - default: []
      description: Array of --build-arg values ("arg=value" strings) for buildah
      name: build-args
      type: array
    - default: ""
      description: Path to a file with build arguments for buildah, see https://www.mankier.com/1/buildah-build#--build-arg-file
      name: build-args-file
      type: string
    - default: "false"
      description: Whether to enable privileged mode, should be used only with remote
        VMs
      name: privileged-nested
      type: string
    results:
    - description: ""
      name: IMAGE_URL
      value: $(tasks.build-image-index.results.IMAGE_URL)
    - description: ""
      name: IMAGE_DIGEST
      value: $(tasks.build-image-index.results.IMAGE_DIGEST)
    - description: ""
      name: CHAINS-GIT_URL
      value: $(tasks.clone-repository.results.url)
    - description: ""
      name: CHAINS-GIT_COMMIT
      value: $(tasks.clone-repository.results.commit)
    tasks:
    # โ”€โ”€ DEFAULT TASKS (unchanged from the generated pipeline) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    - name: init
      params:
      - name: enable-cache-proxy
        value: $(params.enable-cache-proxy)
      taskRef:
        params:
        - name: name
          value: init
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-init:0.4.2@sha256:421003a5c077ecb820460e71637125ec9093d2101c749a32ede28e190283e9db
        - name: kind
          value: task
        resolver: bundles
    - name: clone-repository
      params:
      - name: url
        value: $(params.git-url)
      - name: revision
        value: $(params.revision)
      - name: ociStorage
        value: $(params.output-image).git
      - name: ociArtifactExpiresAfter
        value: $(params.image-expires-after)
      runAfter:
      - init
      taskRef:
        params:
        - name: name
          value: git-clone-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-git-clone-oci-ta-min:0.2@sha256:83bffb5ef4589830c8d0f606325fb27e8cd680c8404d5a935f6660174c9e118c
        - name: kind
          value: task
        resolver: bundles
      workspaces:
      - name: basic-auth
        workspace: git-auth
    - name: prefetch-dependencies
      params:
      - name: input
        value: $(params.prefetch-input)
      - name: enable-package-registry-proxy
        value: $(params.enable-package-registry-proxy)
      - name: SOURCE_ARTIFACT
        value: $(tasks.clone-repository.results.SOURCE_ARTIFACT)
      - name: ociStorage
        value: $(params.output-image).prefetch
      - name: ociArtifactExpiresAfter
        value: $(params.image-expires-after)
      runAfter:
      - clone-repository
      taskRef:
        params:
        - name: name
          value: prefetch-dependencies-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-prefetch-dependencies-oci-ta-min:0.3.2@sha256:7f344093a3387d05eeedad1f929743e197212b50ee95ceaebdc74bbe5df05d03
        - name: kind
          value: task
        resolver: bundles
      workspaces:
      - name: git-basic-auth
        workspace: git-auth
      - name: netrc
        workspace: netrc
    - name: build-container
      params:
      - name: IMAGE
        value: $(params.output-image)
      - name: DOCKERFILE
        value: $(params.dockerfile)
      - name: CONTEXT
        value: $(params.path-context)
      - name: HERMETIC
        value: $(params.hermetic)
      - name: PREFETCH_INPUT
        value: $(params.prefetch-input)
      - name: IMAGE_EXPIRES_AFTER
        value: $(params.image-expires-after)
      - name: COMMIT_SHA
        value: $(tasks.clone-repository.results.commit)
      - name: BUILD_ARGS
        value:
        - $(params.build-args[*])
      - name: BUILD_ARGS_FILE
        value: $(params.build-args-file)
      - name: PRIVILEGED_NESTED
        value: $(params.privileged-nested)
      - name: SOURCE_URL
        value: $(tasks.clone-repository.results.url)
      - name: BUILDAH_FORMAT
        value: $(params.buildah-format)
      - name: HTTP_PROXY
        value: $(tasks.init.results.http-proxy)
      - name: NO_PROXY
        value: $(tasks.init.results.no-proxy)
      - name: SOURCE_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.SOURCE_ARTIFACT)
      - name: CACHI2_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)
      runAfter:
      - prefetch-dependencies
      taskRef:
        params:
        - name: name
          value: buildah-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-buildah-oci-ta-min:0.10@sha256:72c599425e0bda63c30e10127dc493d139c75e45b986d8b7627a0d1b94aafff8
        - name: kind
          value: task
        resolver: bundles
    - name: build-image-index
      params:
      - name: IMAGE
        value: $(params.output-image)
      - name: ALWAYS_BUILD_INDEX
        value: $(params.build-image-index)
      - name: IMAGES
        value:
        - $(tasks.build-container.results.IMAGE_URL)@$(tasks.build-container.results.IMAGE_DIGEST)
      - name: BUILDAH_FORMAT
        value: $(params.buildah-format)
      runAfter:
      - build-container
      taskRef:
        params:
        - name: name
          value: build-image-index-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-build-image-index-min:0.3@sha256:3279101e95f35b768877ae9e7620e92984bd72907e79ea5fefade39d56cc93a3
        - name: kind
          value: task
        resolver: bundles
    # โ”€โ”€ CUSTOM TASK: pr-build-summary โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    # Demonstrates PR-specific pipeline customization. Runs after build-image-index
    # in parallel with the scan tasks. Notice the "on-pr-" prefix in the image tag
    # โ€” this is set by the output-image param above and signals this is a throwaway
    # PR image, not a production artifact.
    #
    # Experiment: add a `when` condition (like the scan tasks) so this task only
    # runs when skip-checks is "true" (fast-PR mode). Flip the operator to "notin"
    # and the values to ["false"] โ€” the structure is identical to the scan tasks.
    - name: pr-build-summary
      params:
      - name: IMAGE_URL
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: IMAGE_DIGEST
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: GIT_URL
        value: $(tasks.clone-repository.results.url)
      - name: GIT_COMMIT
        value: $(tasks.clone-repository.results.commit)
      - name: IMAGE_EXPIRES_AFTER
        value: $(params.image-expires-after)
      runAfter:
      - build-image-index
      taskSpec:
        params:
        - name: IMAGE_URL
          type: string
        - name: IMAGE_DIGEST
          type: string
        - name: GIT_URL
          type: string
        - name: GIT_COMMIT
          type: string
        - name: IMAGE_EXPIRES_AFTER
          type: string
        steps:
        - name: pr-summary
          image: registry.access.redhat.com/ubi9/ubi-minimal:latest
          script: |
            #!/bin/bash
            echo "=================================================="
            echo "          KONFLUX PR BUILD SUMMARY"
            echo "=================================================="
            printf '%-22s %s\n' "Source repository:"   "$(params.GIT_URL)"
            printf '%-22s %s\n' "PR commit:"           "$(params.GIT_COMMIT)"
            printf '%-22s %s\n' "PR image URL:"        "$(params.IMAGE_URL)"
            printf '%-22s %s\n' "Image digest:"        "$(params.IMAGE_DIGEST)"
            printf '%-22s %s\n' "Expires after:"       "$(params.IMAGE_EXPIRES_AFTER)"
            echo "=================================================="
            echo ""
            echo "NOTE: This is a disposable PR image. It expires in"
            echo "$(params.IMAGE_EXPIRES_AFTER) and will NOT be released."
            echo ""
            echo "Pull for local testing:"
            echo "  podman pull $(params.IMAGE_URL)@$(params.IMAGE_DIGEST)"
    # โ”€โ”€ END CUSTOM TASK โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    - name: deprecated-base-image-check
      params:
      - name: IMAGE_URL
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: IMAGE_DIGEST
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: deprecated-image-check
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-deprecated-image-check:0.5@sha256:e78d0d3baf3c8cfc1a5ad278196b74032d9568b143a87c7a79ab780fedfb296e
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: clamav-scan
      params:
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: clamav-scan-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-clamav-scan-min:0.3@sha256:908e356d7a3bc3e472a9075a1ce1370486361007b3a86e3f61c2a6af7136a335
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: sast-shell-check
      params:
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: TARGET_DIRS
        value: $(params.sast-target-dirs)
      - name: SOURCE_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.SOURCE_ARTIFACT)
      - name: CACHI2_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: sast-shell-check-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-sast-shell-check-oci-ta-min:0.1@sha256:ecfae10944b45b91988ddc6311c7232bf2c98ba73c5fc6261861ecfd33434db0
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: sast-unicode-check
      params:
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: TARGET_DIRS
        value: $(params.sast-target-dirs)
      - name: SOURCE_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.SOURCE_ARTIFACT)
      - name: CACHI2_ARTIFACT
        value: $(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: sast-unicode-check-oci-ta-min
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-sast-unicode-check-oci-ta-min:0.4@sha256:96badf0b06d83fc1e7cf50048f94091e257819ee537f063830541f9c97295200
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: rpms-signature-scan
      params:
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: rpms-signature-scan
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-rpms-signature-scan:0.2@sha256:65370ccb44ff82e4ce128addd913f3c96b298607b3760ee1339ed10011a4bd6b
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    - name: tpa-scan
      params:
      - name: image-digest
        value: $(tasks.build-image-index.results.IMAGE_DIGEST)
      - name: image-url
        value: $(tasks.build-image-index.results.IMAGE_URL)
      runAfter:
      - build-image-index
      taskRef:
        params:
        - name: name
          value: tpa-scan
        - name: bundle
          value: quay.io/konflux-ci/tekton-catalog/task-tpa-scan:0.1@sha256:8375c9e4ee2ee417881120187be1281c50b85a8b23d8d24785e1dc96a3018c8e
        - name: kind
          value: task
        resolver: bundles
      when:
      - input: $(params.skip-checks)
        operator: in
        values:
        - "false"
    workspaces:
    - name: git-auth
      optional: true
    - name: netrc
      optional: true
  taskRunTemplate:
    # Auto-created by Build Service as build-pipeline-
    # For a component named "testrepo" this is automatically build-pipeline-testrepo
    serviceAccountName: build-pipeline-testrepo
  workspaces:
  - name: git-auth
    secret:
      secretName: '{{ git_auth_secret }}'
status: {}
.tekton/tasks/print-build-summary.yaml (standalone Task CRD โ€” same logic as the inline taskSpec, extracted for reuse across pipelines)
# The inline taskSpec blocks in the two PipelineRun files above are convenient
# for one-off steps that only apply to a single pipeline. Once a task is stable
# and you want to share it across multiple pipelines, extract it to a standalone
# Tekton Task CRD like this one.
#
# After applying this Task to your namespace, reference it from any PipelineRun:
#
#   - name: print-build-summary
#     taskRef:
#       name: print-build-summary       # resolves by name in the same namespace
#     params:
#     - name: IMAGE_URL
#       value: $(tasks.build-image-index.results.IMAGE_URL)
#     - name: IMAGE_DIGEST
#       value: $(tasks.build-image-index.results.IMAGE_DIGEST)
#     - name: GIT_URL
#       value: $(tasks.clone-repository.results.url)
#     - name: GIT_COMMIT
#       value: $(tasks.clone-repository.results.commit)
#     runAfter:
#     - build-image-index
#
# Apply to your tenant namespace:
#   oc apply -f .tekton/tasks/print-build-summary.yaml -n default-tenant
apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: print-build-summary
  namespace: default-tenant
  annotations:
    tekton.dev/displayName: "Print Build Summary"
    tekton.dev/tags: "notification, summary, debugging"
spec:
  description: >-
    Prints a human-readable summary of a completed Konflux build including
    the image URL, digest, and source commit. Use this as a template for
    custom notification steps (Slack, PagerDuty, JIRA webhooks, etc.) โ€”
    replace the echo statements with your notification tool of choice.
  params:
    - name: IMAGE_URL
      type: string
      description: "Fully qualified image URL from build-image-index.results.IMAGE_URL"
    - name: IMAGE_DIGEST
      type: string
      description: "Image digest (sha256:...) from build-image-index.results.IMAGE_DIGEST"
    - name: GIT_URL
      type: string
      description: "Source repository URL from clone-repository.results.url"
    - name: GIT_COMMIT
      type: string
      description: "Git commit SHA from clone-repository.results.commit"
  results:
    - name: SUMMARY
      description: "One-line summary โ€” useful as input to downstream notification tasks"
  steps:
    - name: summarize
      image: registry.access.redhat.com/ubi9/ubi-minimal:latest
      script: |
        #!/bin/bash
        echo "=================================================="
        echo "           KONFLUX BUILD SUMMARY"
        echo "=================================================="
        printf '%-22s %s\n' "Source repository:"  "$(params.GIT_URL)"
        printf '%-22s %s\n' "Git commit:"         "$(params.GIT_COMMIT)"
        printf '%-22s %s\n' "Built image URL:"    "$(params.IMAGE_URL)"
        printf '%-22s %s\n' "Image digest:"       "$(params.IMAGE_DIGEST)"
        echo "=================================================="
        echo ""
        echo "Pull the image:"
        echo "  podman pull $(params.IMAGE_URL)@$(params.IMAGE_DIGEST)"
        echo ""
        echo "Verify the signature (Tekton Chains signs after PipelineRun completes):"
        echo "  cosign verify --key cosign.pub --insecure-ignore-tlog \\"
        echo "    $(params.IMAGE_URL)@$(params.IMAGE_DIGEST)"
        echo ""
        # Write a one-liner result for downstream tasks (e.g. a Slack notifier task)
        SUMMARY="Built $(params.IMAGE_URL)@$(params.IMAGE_DIGEST) from $(params.GIT_COMMIT)"
        printf '%s' "$SUMMARY" | tee "$(results.SUMMARY.path)"

Commands

terminal
NAMESPACE="your-username-tenant"

# โ”€โ”€ Inspect PaC configuration โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# On OCP, the PaC controller runs in openshift-pipelines (not pipelines-as-code)
oc get pods -n openshift-pipelines \
  -l app.kubernetes.io/part-of=pipelines-as-code

# See all PaC Repositories (registered webhooks)
# The NAME column from this output is what you pass to describe below
oc get repository -n $NAMESPACE

# Describe your repository to see webhook status
# Use the exact NAME shown by the command above โ€” it typically matches
# the component name but use the value from oc get repository, not a guess
REPO_NAME=$(oc get repository -n $NAMESPACE -o jsonpath='{.items[0].metadata.name}')
oc describe repository "$REPO_NAME" -n $NAMESPACE

# โ”€โ”€ After pushing .tekton changes โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Push a commit to trigger the updated pipeline
git add .tekton/
git commit -m "feat: add print-build-summary custom task to pipeline"
git push origin main

# Watch the new PipelineRun appear
oc get pipelinerun -n $NAMESPACE -w

# Verify the custom task ran (look for the KONFLUX BUILD SUMMARY banner)
tkn pipelinerun logs --last -n $NAMESPACE | grep -A 20 "print-build-summary"

# โ”€โ”€ List all tasks in the pipeline โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
PR_NAME=$(oc get pipelinerun -n $NAMESPACE \
  --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{.items[-1].metadata.name}')

oc get taskruns -n $NAMESPACE \
  --selector=tekton.dev/pipelineRun=$PR_NAME \
  -o 'custom-columns=NAME:.metadata.name,TASK:.metadata.labels.tekton\.dev/pipelineTask,STATUS:.status.conditions[0].reason'

# โ”€โ”€ Inspect the resolved pipeline definition โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# See which OCI bundle version was actually used
oc get pipelinerun $PR_NAME -n $NAMESPACE \
  -o jsonpath='{.status.pipelineSpec.tasks[*].name}' | tr ' ' '\n'

# โ”€โ”€ Debug PaC webhook delivery issues โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# On OCP, PaC controller runs in openshift-pipelines
oc logs -n openshift-pipelines \
  -l app.kubernetes.io/component=controller,app.kubernetes.io/part-of=pipelines-as-code --tail=50

# List recent events from PaC
oc get events -n $NAMESPACE \
  --sort-by=.lastTimestamp | tail -20

Key Takeaways

  • PaC reads .tekton/ files on every webhook event
  • Push pipeline: full scans, real push, no expiry
  • PR pipeline: faster, skip-checks: true, short expiry
  • Add custom tasks by embedding taskSpec in the PipelineRun
  • Never disable scans (skip-checks: true) on the push pipeline
  • PaC CEL annotations control which branches/events trigger builds

Snapshots & Integration Tests โ€” Testing Your Application as a Whole

What You'll Learn

What a Snapshot is and when it gets created automatically
The Snapshot lifecycle: created โ†’ tested โ†’ marked passed/failed
Creating an IntegrationTestScenario โ€” pointing to a test pipeline
Writing a real integration test pipeline that deploys and tests the Snapshot
Interpreting test results and the TEST_OUTPUT result format
Manually creating a Snapshot for testing purposes

Steps

  1. Apply integration-runner-rbac.yaml before anything else โ€” the integration test pipeline creates a Kubernetes Job in the tenant namespace, and the konflux-integration-runner service account has no batch/jobs permission by default. Skipping this step causes the PipelineRun to fail immediately with a Forbidden error.
  2. Trigger a build and inspect the resulting Snapshot โ€” push a change to your testrepo fork. Once the PipelineRun succeeds, run oc get snapshot -n default-tenant to confirm the Integration Service automatically created a Snapshot and examine its spec.components.
  3. Apply integration-test-scenario.yaml โ€” this CR registers your test pipeline with the Integration Service. Check that the git resolver URL and pathInRepo point to integration-tests/testrepo-integration.yaml in your fork.
  4. Read the test pipeline โ€” open integration-tests/testrepo-integration.yaml in your repo and trace how it pulls the built image, runs it as a Kubernetes Job, and asserts the expected output. The git resolver in the IntegrationTestScenario fetches this file automatically at test time.
  5. Push another change to trigger a full cycle โ€” after the build succeeds and a new Snapshot is created, watch the integration test PipelineRun appear automatically. Follow its logs to see the Job run and the assertion pass.
  6. Check the Snapshot status โ€” run oc get snapshot -o yaml on the latest Snapshot and read the AppStudioTestSucceeded condition to confirm the integration test result is recorded on the Snapshot.
  7. Apply manual-snapshot.yaml to re-run tests without a new build โ€” this lets you run integration tests against an already-built image, which is useful for iterating on the test pipeline itself.

YAML Files

integration-runner-rbac.yaml (apply this first โ€” grants the integration runner permission to manage Jobs and Pods, and links regcred pull secret to the default service account)
# โ”€โ”€ PREREQUISITE 1: RBAC for konflux-integration-runner โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
#
# WHY THIS IS NEEDED:
# The integration test pipeline (testrepo-integration.yaml) creates a Kubernetes
# Job in the tenant namespace to run the built container image, then reads its logs.
# By default the konflux-integration-runner service account only has narrow
# permissions and cannot create, delete, or list batch/jobs or pod logs in
# the tenant namespace. Without this RBAC the test PipelineRun fails immediately:
#
#   Error from server (Forbidden): jobs.batch "test-hello" is forbidden:
#   User "system:serviceaccount:default-tenant:konflux-integration-runner"
#   cannot delete resource "jobs" in API group "batch" in the namespace "default-tenant"
#
# This is a known gap in the standard Konflux install โ€” the same Role+RoleBinding
# pattern is used in the official Konflux e2e test fixtures:
#   https://github.com/konflux-ci/konflux-ci/blob/main/test/resources/demo-users/user/ns2/appstudio-pipeline-integration-runner-rbac.yaml
#
# Apply this once before creating the IntegrationTestScenario:
#   oc apply -f integration-runner-rbac.yaml
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  # Must match your tenant namespace
  namespace: default-tenant
  name: default-tenant-pod-viewer-job-creator
rules:
- apiGroups: [""]
  resources:
    - pods
  verbs: ["get", "list", "watch", "delete"]
- apiGroups: ["batch"]
  resources:
    - jobs
  verbs: ["create", "delete", "get", "list", "watch"]
- apiGroups: [""]
  resources:
    - pods/log
  verbs: ["get", "list"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  # Must match your tenant namespace
  namespace: default-tenant
  name: default-tenant-pod-viewer-job-creator-binding
subjects:
# Grants the job viewer/creator permissions to konflux-integration-runner
# which is the SA Konflux uses to run integration test PipelineRuns
- kind: ServiceAccount
  name: konflux-integration-runner
  namespace: default-tenant
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: default-tenant-pod-viewer-job-creator
Prerequisite 2 โ€” Link regcred pull secret to the default service account
# WHY THIS IS NEEDED:
# The Kubernetes Job created by the test pipeline runs as the "default" service
# account in default-tenant. The regcred secret (your quay.io push/pull credentials
# created in Phase 4) must be explicitly linked to this SA or the Job pod will
# fail to pull the testrepo image with an image pull error:
#
#   Failed to pull image "quay.io/...": ...unauthorized: access to the requested
#   resource is not authorized

# Link regcred to the default service account in your tenant namespace
oc secrets link default regcred --for=pull -n default-tenant

# Verify the link was applied โ€” output must include {"name": "regcred"}
oc get sa default -n default-tenant \
  -o jsonpath='{.imagePullSecrets}' | python3 -m json.tool
integration-test-scenario.yaml
apiVersion: appstudio.redhat.com/v1beta2
kind: IntegrationTestScenario
metadata:
  # Name for this test scenario โ€” shown in the Konflux UI test results panel
  name: testrepo-integration-tests
  # Must match your tenant namespace โ€” where the Application and Component CRs live
  namespace: default-tenant
  labels:
    test.appstudio.openshift.io/optional: "false"   # failure blocks release
spec:
  # Must match the Application CR name created in Phase 5
  application: my-first-app

  # Points to the integration test pipeline inside your git repository.
  # The Integration Service resolves this file at test time using the git
  # resolver โ€” no need to apply the pipeline to the cluster separately.
  resolverRef:
    resolver: git
    params:
      - name: url
        # Replace YOUR-USERNAME with your GitHub username (owner of the testrepo fork)
        value: https://github.com/YOUR-USERNAME/testrepo
      - name: revision
        value: main
      - name: pathInRepo
        # Path to the test pipeline file inside the repository
        # This file already exists in the testrepo โ€” do not rename it
        value: integration-tests/testrepo-integration.yaml
integration-tests/testrepo-integration.yaml (runs the built image as a Kubernetes Job and asserts "hello world" output)
# Konflux integration test for testrepo.
# Runs the built container image as a Kubernetes Job and asserts its logs
# contain "hello world" โ€” the output produced by entrypoint.sh.
#
# This file is already present in the testrepo repository at:
#   integration-tests/testrepo-integration.yaml
#
# The IntegrationTestScenario (integration-test-scenario.yaml) points the
# Integration Service here via the git resolver, so this pipeline is fetched
# from the repo at test time. You do NOT apply it manually to the cluster.
#
# IMPORTANT: Keep the grep assertion ("hello world") in sync with entrypoint.sh.
# If you change what entrypoint.sh prints, update the grep below too.
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: testrepo-integration-test
spec:
  params:
    - name: SNAPSHOT
      type: string
      description: |
        JSON snapshot injected by the Konflux Integration Service.
        Contains the built image reference for the testrepo component:
          {
            "components": [
              {
                "containerImage": "quay.io/YOUR-ORG/testrepo@sha256:..."
              }
            ]
          }
  tasks:
    - name: test-hello-world
      params:
        - name: SNAPSHOT
          value: "$(params.SNAPSHOT)"
      taskSpec:
        params:
          - name: SNAPSHOT
            type: string
        steps:
          - name: test-output
            # oc image with jq โ€” resolves the image from the snapshot and
            # runs it as a Job to verify the container prints "hello world"
            image: quay.io/ongres/kubectl@sha256:4be5050c456a4751fe3d70086c0387d65e8973a176fe965de4eaeeb8643b2e0a
            env:
              - name: SNAPSHOT
                value: "$(params.SNAPSHOT)"
              - name: NAMESPACE
                # Injected automatically by Tekton โ€” the namespace the PipelineRun runs in
                value: "$(context.pipelineRun.namespace)"
            script: |
              #!/usr/bin/bash
              set -euxo pipefail

              # Extract the built image from the snapshot (first component)
              IMAGE=$(echo "$SNAPSHOT" | jq -r '.components[0].containerImage')
              echo "Extracted image: $IMAGE"

              # Clean up any leftover job or pods from a previous run
              kubectl delete job --ignore-not-found test-hello -n $NAMESPACE
              kubectl delete pods --ignore-not-found -l job-name=test-hello -n $NAMESPACE

              # Run the testrepo image as a short-lived Job
              kubectl create job test-hello -n $NAMESPACE --image=$IMAGE

              # Wait up to 120s for the Job to complete (entrypoint.sh exits after printing)
              kubectl wait --for=condition=complete job/test-hello --timeout=120s

              # Capture the pod logs and assert the expected output
              LOGS=$(kubectl logs -l job-name=test-hello -n $NAMESPACE)

              # Clean up the Job after capturing logs
              kubectl delete job --ignore-not-found test-hello -n $NAMESPACE

              # This is the actual test assertion โ€” fails the pipeline if output is wrong
              echo $LOGS | grep "hello world"
manual-snapshot.yaml (trigger integration tests without a new build โ€” useful during test development)
apiVersion: appstudio.redhat.com/v1alpha1
kind: Snapshot
metadata:
  # A descriptive name for this manually created snapshot
  name: testrepo-manual-snapshot
  # Must match your tenant namespace
  namespace: default-tenant
  labels:
    # Must match the Application CR name created in Phase 5
    appstudio.openshift.io/application: my-first-app
    # "override" marks this as a manually created snapshot (not from a build)
    test.appstudio.openshift.io/type: override
spec:
  # Must match the Application CR name created in Phase 5
  application: my-first-app
  displayName: "Manual test snapshot for integration test development"
  components:
    # Must match the Component CR name created in Phase 5
    - name: testrepo
      # Replace with the actual image digest from a previous successful push build.
      # Get the image URL and digest from the last push PipelineRun:
      #   NS="default-tenant"
      #   PR=$(oc get pipelinerun -n $NS --sort-by=.metadata.creationTimestamp \
      #          -o jsonpath='{.items[-1].metadata.name}')
      #   oc get pipelinerun $PR -n $NS \
      #     -o jsonpath='{.status.results[?(@.name=="IMAGE_URL")].value}'
      #   oc get pipelinerun $PR -n $NS \
      #     -o jsonpath='{.status.results[?(@.name=="IMAGE_DIGEST")].value}'
      # Then combine as: quay.io/YOUR-ORG/testrepo@sha256:
      containerImage: quay.io/YOUR-ORG/testrepo@sha256:abc123def456abc123def456abc123def456abc123def456abc123def456abc123

Commands

terminal
NAMESPACE="your-username-tenant"

# โ”€โ”€ Inspect snapshots โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
oc get snapshots -n $NAMESPACE
oc get snapshot -n $NAMESPACE -o wide

# Describe a snapshot โ€” see components and test status
oc describe snapshot -n $NAMESPACE

# Get the snapshot in full YAML (includes test result annotations)
oc get snapshot -n $NAMESPACE -o yaml | \
  grep -A 20 "annotations\|status"

# โ”€โ”€ Prerequisites before applying anything else โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€

# Step 1: Grant RBAC โ€” the test pipeline creates a Job in the tenant namespace;
# the integration runner service account has no batch/jobs permission by default.
oc apply -f integration-runner-rbac.yaml

# Verify the Role and RoleBinding were created
oc get role default-tenant-pod-viewer-job-creator -n $NAMESPACE
oc get rolebinding default-tenant-pod-viewer-job-creator-binding -n $NAMESPACE

# Step 2: Link regcred pull secret to the default service account.
# The Job pod runs as "default" SA โ€” without this link it cannot pull the
# testrepo image from quay.io and the pod fails with an image pull error.
oc secrets link default regcred --for=pull -n $NAMESPACE

# Verify the link is in place
oc get sa default -n $NAMESPACE \
  -o jsonpath='{.imagePullSecrets}' | python3 -m json.tool
# Expected output includes: {"name": "regcred"}

# โ”€โ”€ Apply the IntegrationTestScenario โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
oc apply -f integration-test-scenario.yaml

# Verify it was created
oc get integrationtestscenario -n $NAMESPACE

# โ”€โ”€ Manually create a snapshot to trigger integration tests โ”€โ”€โ”€โ”€
oc apply -f manual-snapshot.yaml

# Watch the integration test PipelineRun appear
oc get pipelinerun -n $NAMESPACE -w | grep integration

# Follow the integration test logs
tkn pipelinerun logs --last -f -n $NAMESPACE

# โ”€โ”€ Check test results in the snapshot status โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
SNAPSHOT_NAME=$(oc get snapshot -n $NAMESPACE \
  --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{.items[-1].metadata.name}')

# Check the Snapshot conditions โ€” the Integration Service writes pass/fail here
oc get snapshot $SNAPSHOT_NAME -n $NAMESPACE \
  -o jsonpath='{.status.conditions}' | python3 -m json.tool

# Find the integration test PipelineRun for this snapshot
# (the label appstudio.openshift.io/snapshot is set by the Integration Service)
oc get pipelinerun -n $NAMESPACE \
  --selector=appstudio.openshift.io/snapshot=$SNAPSHOT_NAME

# Check whether the integration test PipelineRun succeeded or failed
# NOTE: testrepo-integration.yaml does not write a TEST_OUTPUT result โ€”
# it passes or fails purely on the exit code of "grep hello world".
# The authoritative result is the PipelineRun's Succeeded condition:
oc get pipelinerun -n $NAMESPACE \
  --selector=appstudio.openshift.io/snapshot=$SNAPSHOT_NAME \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.conditions[?(@.type=="Succeeded")].status}{"\t"}{.status.conditions[?(@.type=="Succeeded")].reason}{"\n"}{end}'

Key Takeaways

  • Snapshot = immutable record of image digests for all components
  • Auto-created after every successful component build
  • IntegrationTestScenario points to a test pipeline via the git resolver โ€” no manual pipeline apply needed
  • Grant RBAC first: konflux-integration-runner needs batch/jobs + pods/log permission to run Job-based tests
  • Failed tests block the Snapshot from being released
  • Manually create Snapshots for integration test development without triggering a new build

Release Planning & Gating โ€” Controlled Software Delivery with Konflux

What You'll Learn

The full release flow: Snapshot โ†’ Tests โ†’ ReleasePlan โ†’ Release
Creating a ReleasePlan โ€” what to release and where
ReleasePlanAdmission โ€” how managed services gate your releases
Writing a release pipeline that pushes to production
Triggering a Release manually and watching the release pipeline
Automated vs manual release approval flows

Steps

  1. Create the staging-registry-secret โ€” this is a one-time prerequisite in default-tenant that provides the release pipeline's push task with credentials to authenticate against quay.io. Without it, the push step fails with an authentication error.
  2. Apply enterprise-contract-policy.yaml first โ€” the ReleasePlanAdmission references this CR by name. If it does not exist before a Release is triggered, every Release fails immediately with a "not found" error.
  3. Apply release-plan.yaml and release-plan-admission.yaml โ€” both go into default-tenant. Run oc get releaseplan and oc get releaseplanadmission to confirm they are matched to each other.
  4. Commit release/release-pipeline.yaml to your testrepo fork โ€” the git resolver in the ReleasePlanAdmission fetches this file from your repository at release time. It must be present on the main branch before any Release fires.
  5. Push a change to trigger the full release cycle โ€” with auto-release: "true" on the ReleasePlan, the Release Service automatically creates a Release object after the Snapshot passes integration tests. Follow the release PipelineRun with tkn pipelinerun logs --last -f -n default-tenant.
  6. Inspect the Release CRD status โ€” run oc get release -n default-tenant and describe the latest Release object to read the conditions that capture the release outcome, including the promoted image reference.

YAML Files

enterprise-contract-policy.yaml (must be applied before triggering a Release โ€” referenced by release-plan-admission.yaml)
# The ReleasePlanAdmission references this policy by name. If this CR does not
# exist, every Release fails immediately with:
#   EnterpriseContractPolicy.appstudio.redhat.com "testrepo-ec-policy" not found
#
# This policy mirrors the cluster-managed default from enterprise-contract-service/default,
# applied in default-tenant so the release pipeline can reference it via
# "default-tenant/testrepo-ec-policy" without cross-namespace access.
# It enforces the @redhat collection (excluding hermetic_task, source_image, rpm_repos)
# using the same pinned policy and data bundles as the cluster default.
#
# โ”€โ”€ ALTERNATIVE: create directly from the cluster default (no YAML file needed) โ”€โ”€
# Instead of applying this file, you can copy the default policy from the
# enterprise-contract-service namespace and apply it to default-tenant in one command:
#
#   oc get enterprisecontractpolicy default \
#     -n enterprise-contract-service \
#     -o json \
#     | jq 'del(.metadata.resourceVersion,
#               .metadata.uid,
#               .metadata.creationTimestamp,
#               .metadata.generation,
#               .metadata.ownerReferences,
#               .metadata.labels,
#               .metadata.managedFields,
#               .status)
#           | .metadata.name = "testrepo-ec-policy"
#           | .metadata.namespace = "default-tenant"' \
#     | oc apply -f -
#
#   # Verify it was created
#   oc get enterprisecontractpolicy testrepo-ec-policy -n default-tenant
apiVersion: appstudio.redhat.com/v1alpha1
kind: EnterpriseContractPolicy
metadata:
  name: testrepo-ec-policy
  # Must match the namespace where the ReleasePlanAdmission lives
  namespace: default-tenant
spec:
  description: >-
    Mirror of the cluster default EnterpriseContractPolicy from
    enterprise-contract-service/default, applied in default-tenant for the
    testrepo release pipeline. Enforces the @redhat rule collection with the
    same exclusions and pinned bundle references as the cluster-managed policy.
    Omit ownerReferences โ€” this CR is managed manually, not by the operator.
  name: Default
  # Public key used to verify Tekton Chains image signatures.
  # References the cosign.pub key in openshift-pipelines/public-key secret.
  # Extract with: oc get secret public-key -n openshift-pipelines \
  #                 -o jsonpath='{.data.cosign\.pub}' | base64 -d
  publicKey: k8s://openshift-pipelines/public-key
  sources:
    - name: Default
      config:
        exclude:
          - hermetic_task
          - source_image
          - rpm_repos
        include:
          - '@redhat'
      data:
        - oci::quay.io/konflux-ci/tekton-catalog/data-acceptable-bundles:latest@sha256:1d39bdf428eb0273938d672ae9d65c26e42aad64db106ed643e685ce14b09e69
        - github.com/redhat-appstudio/tsf-conforma-data//data?ref=1966f21842d507441a7a5e1c7de9071cf3f9ec53
      policy:
        - oci::quay.io/conforma/release-policy:latest@sha256:4b56dc3d04f3dd372b1db1b38fa02ef8bad0799f354de0c011ac5eba106e0541
release-plan.yaml (applied in the tenant namespace โ€” your team's side of the release contract)
apiVersion: appstudio.redhat.com/v1alpha1
kind: ReleasePlan
metadata:
  # Conventionally -release-plan
  name: testrepo-release-plan
  # Must match your tenant namespace
  namespace: default-tenant
  labels:
    # "false" = manual release: you create a Release object yourself
    # "true"  = auto-release: Integration Service creates a Release on every passing Snapshot
    release.appstudio.openshift.io/auto-release: "true"
    # REQUIRED for auto-release โ€” tells the Release Service who authorises automated releases.
    # "standing-attribution: true" means: attribute every automated Release to whoever
    # created this ReleasePlan (no hardcoded username needed).
    # Without this label, automated releases fail with:
    #   "no author in the ReleasePlan found for automated release"
    # Safe to include even when auto-release is "false" โ€” it has no effect for manual releases.
    release.appstudio.openshift.io/standing-attribution: "true"
spec:
  # Must match the Application CR name created in Phase 5
  application: my-first-app

  # The namespace where the ReleasePlanAdmission lives.
  # For this learning setup both the ReleasePlan and ReleasePlanAdmission live
  # in the same default-tenant namespace โ€” no separate managed namespace needed.
  target: default-tenant

  # Snapshot must have passed integration tests within this many days to be releasable
  releaseGracePeriodDays: 7
release-plan-admission.yaml (applied in the same default-tenant namespace as the ReleasePlan)
apiVersion: appstudio.redhat.com/v1alpha1
kind: ReleasePlanAdmission
metadata:
  # Conventionally -admission
  name: testrepo-admission
  # Same namespace as the ReleasePlan โ€” no separate managed namespace for this learning setup
  namespace: default-tenant
spec:
  # Must match the namespace where the ReleasePlan lives
  origin: default-tenant

  # Must match the Application CR name created in Phase 5
  applications:
    - my-first-app

  # The Tekton pipeline to run when a Release is triggered.
  # The git resolver fetches the pipeline from the repo at release time โ€”
  # no need to apply it to the cluster separately.
  pipeline:
    pipelineRef:
      resolver: git
      params:
        - name: url
          # Replace YOUR-USERNAME with your GitHub username (owner of the testrepo fork).
          # In production this would typically be a dedicated release-pipelines repository.
          value: https://github.com/YOUR-USERNAME/testrepo
        - name: revision
          value: main
        - name: pathInRepo
          # Path to the release pipeline inside the repository
          value: release/release-pipeline.yaml

  # EnterpriseContractPolicy to enforce before the release pipeline runs.
  # The CR must exist in the managed namespace before a Release is triggered.
  # See enterprise-contract-policy.yaml below for how to create it.
  policy: testrepo-ec-policy
release/release-pipeline.yaml (store in your repo at this path โ€” fetched by the git resolver in release-plan-admission.yaml)
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: testrepo-release-to-staging
  namespace: default-tenant
spec:
  description: >-
    Production release pipeline for testrepo.
    Validates the built image with Enterprise Contract (Conforma) then promotes
    it to the staging registry. Registry credentials come from the regcred secret
    (kubernetes.io/dockerconfigjson) mounted as a projected volume โ€” this bypasses
    Tekton cred-init and provides stable credentials to all OCI tools (skopeo,
    cosign, ec) regardless of the step user identity.
  params:
    - name: release
      type: string
      description: "The Release CRD name โ€” injected by the Release Service"
    - name: releasePlan
      type: string
      description: "The ReleasePlan CRD name"
    - name: releasePlanAdmission
      type: string
      description: "The ReleasePlanAdmission CRD name"
    - name: snapshot
      type: string
      description: "Snapshot reference โ€” injected by the Release Service as namespace/name"
    - name: enterpriseContractPolicy
      type: string
      description: "Enterprise Contract policy name (not used directly โ€” policy is hardcoded below)"

  tasks:

    # โ”€โ”€ Task 1: Extract the image reference from the Snapshot โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    # The Release Service passes snapshot as "namespace/name" (e.g.
    # "default-tenant/my-first-app-abc123"). This task fetches the Snapshot CR
    # from the cluster and extracts the containerImage for the testrepo component.
    - name: extract-images
      taskSpec:
        params:
          - name: snapshot
        results:
          - name: image
            description: "containerImage for the testrepo component"
        steps:
          - name: extract
            image: quay.io/ongres/kubectl@sha256:4be5050c456a4751fe3d70086c0387d65e8973a176fe965de4eaeeb8643b2e0a
            env:
              - name: SNAPSHOT_REF
                value: "$(params.snapshot)"
            script: |
              #!/bin/bash
              set -e

              echo "Snapshot reference: $SNAPSHOT_REF"

              if [[ "$SNAPSHOT_REF" == *"/"* ]]; then
                SNAP_NS="${SNAPSHOT_REF%%/*}"
                SNAP_NAME="${SNAPSHOT_REF##*/}"
              else
                SNAP_NS="default-tenant"
                SNAP_NAME="$SNAPSHOT_REF"
              fi

              echo "Fetching snapshot '$SNAP_NAME' from namespace '$SNAP_NS'"

              IMAGE=$(kubectl get snapshot "$SNAP_NAME" -n "$SNAP_NS" -o json | \
                jq -r '.spec.components[] | select(.name=="testrepo") | .containerImage')

              if [ -z "$IMAGE" ] || [ "$IMAGE" = "null" ]; then
                echo "ERROR: component 'testrepo' not found in snapshot" >&2
                kubectl get snapshot "$SNAP_NAME" -n "$SNAP_NS" -o yaml >&2
                exit 1
              fi

              echo "Extracted image: $IMAGE"
              printf '%s' "$IMAGE" > $(results.image.path)
      params:
        - name: snapshot
          value: "$(params.snapshot)"

    # โ”€โ”€ Task 2: Enterprise Contract validation โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    # Runs ec validate image against the Enterprise Contract policy.
    #
    # Credential approach (projected volume):
    #   regcred (.dockerconfigjson) is mounted as /var/docker-config/config.json
    #   via a projected volume and DOCKER_CONFIG is set to that directory.
    #   This gives all tools (ec, cosign) stable access to registry credentials
    #   regardless of which user the step runs as โ€” avoids the Tekton cred-init
    #   permission race that causes "permission denied on /tekton/home/.docker/config.json".
    #
    # Tekton cred-init permission fix:
    #   The write-snapshot step (first to run) pre-creates /tekton/home/.docker/
    #   and sets it to mode 777 so every subsequent step's cred-init can write
    #   its copy of credentials without hitting permission denied.
    #
    # Policy: enterprise-contract-service/default
    #   Uses the @redhat collection (excluding hermetic_task, source_image, rpm_repos).
    #   Note: slsa_source_correlated rules will produce a violation because satisfying
    #   them requires EC config files in the source repository at the exact build commit.
    #   STRICT is set to false so the violation is reported but does not block the release.
    #   Set STRICT to true after adding EC config to your source repo.
    #
    # IGNORE_REKOR: true
    #   Required on self-hosted Konflux โ€” images are signed by Tekton Chains using
    #   the cluster's internal cosign key. No entry is written to the public
    #   Sigstore Rekor at rekor.sigstore.dev.
    - name: validate-enterprise-contract
      runAfter:
        - extract-images
      taskSpec:
        params:
          - name: image
            type: string
          - name: POLICY_CONFIGURATION
            type: string
            default: "enterprise-contract-service/default"
          - name: PUBLIC_KEY
            type: string
            default: "k8s://openshift-pipelines/public-key"
          - name: IGNORE_REKOR
            type: string
            default: "true"
          - name: STRICT
            type: string
            default: "false"
          - name: WORKERS
            type: string
            default: "4"
        results:
          - name: TEST_OUTPUT
            description: "EC policy evaluation result โ€” SUCCESS, WARNING, or FAILURE"
        volumes:
          - name: workdir
            emptyDir: {}
          - name: trusted-ca
            configMap:
              name: trusted-ca
              items:
                - key: ca-bundle.crt
                  path: ca-bundle.crt
              optional: true
          - name: docker-config
            projected:
              sources:
                - secret:
                    name: regcred
                    items:
                      - key: .dockerconfigjson
                        path: config.json
        stepTemplate:
          volumeMounts:
            - mountPath: /var/workdir
              name: workdir
            - mountPath: /mnt/trusted-ca
              name: trusted-ca
              readOnly: true
            - mountPath: /var/docker-config
              name: docker-config
              readOnly: true
          env:
            - name: HOME
              value: /tekton/home
            - name: DOCKER_CONFIG
              value: /var/docker-config

        steps:
          # Step 1: Write snapshot JSON and fix Tekton cred-init directory permissions.
          # Creates /tekton/home/.docker/ with mode 777 so all subsequent steps'
          # cred-init can write their credentials copy without permission denied errors.
          - name: write-snapshot
            image: registry.access.redhat.com/ubi9/ubi-minimal:latest
            env:
              - name: IMAGE
                value: "$(params.image)"
            script: |
              #!/bin/bash
              set -euo pipefail

              # Write the snapshot JSON for ec validate image
              mkdir -p /var/workdir
              printf '%s' \
                "{\"components\":[{\"name\":\"testrepo\",\"containerImage\":\"$IMAGE\"}]}" \
                > /var/workdir/snapshot.json
              echo "snapshot.json:"
              cat /var/workdir/snapshot.json

              # Fix Tekton cred-init directory permissions.
              # Cred-init runs before each step's script and tries to write
              # /tekton/home/.docker/config.json. If this directory was created
              # by a previous step as root (mode 700), subsequent non-root steps
              # get "permission denied". Setting mode 777 on the directory (and
              # 666 on the file if it already exists) lets every step's cred-init
              # succeed without errors.
              mkdir -p /tekton/home/.docker
              chmod 777 /tekton/home/.docker
              if [ -f /tekton/home/.docker/config.json ]; then
                chmod 666 /tekton/home/.docker/config.json
              fi

          # Step 2: Skip TUF โ€” cluster uses internal Tekton Chains key, not public Sigstore
          - name: initialize-tuf
            image: quay.io/conforma/cli:latest
            script: |
              echo 'TUF_MIRROR not configured โ€” skipping TUF root initialization.'

          # Step 3: Enterprise Contract validation.
          # WHY --strict=false in the ec command:
          #   ec always exits 0 and writes all output files (TEST_OUTPUT, text report,
          #   JSON report) even when violations exist. This ensures every subsequent
          #   step (detailed-report, summary, version) has the data it needs for the
          #   audit trail. The actual release-blocking decision is made by the assert
          #   step, which reads TEST_OUTPUT and honours the STRICT param.
          - name: validate
            image: quay.io/conforma/cli:latest
            onError: continue
            env:
              - name: POLICY_CONFIGURATION
                value: "$(params.POLICY_CONFIGURATION)"
              - name: PUBLIC_KEY
                value: "$(params.PUBLIC_KEY)"
              - name: IGNORE_REKOR
                value: "$(params.IGNORE_REKOR)"
              - name: WORKERS
                value: "$(params.WORKERS)"
            script: |
              #!/bin/bash
              set -euo pipefail

              if [ -f "/mnt/trusted-ca/ca-bundle.crt" ]; then
                export SSL_CERT_FILE="/mnt/trusted-ca/ca-bundle.crt"
              fi

              ec validate image \
                --images=/var/workdir/snapshot.json \
                --policy="${POLICY_CONFIGURATION}" \
                --public-key="${PUBLIC_KEY}" \
                --ignore-rekor="${IGNORE_REKOR}" \
                --workers="${WORKERS}" \
                --strict=false \
                --info=true \
                --timeout=0 \
                --output="text=/var/workdir/text-report.txt?show-successes=false" \
                --output="json=/var/workdir/report-json.json" \
                --output="appstudio=$(results.TEST_OUTPUT.path)"

          # Step 4: Human-readable EC report (violations and warnings only)
          - name: detailed-report
            image: quay.io/conforma/cli:latest
            onError: continue
            command: ["cat"]
            args: ["/var/workdir/text-report.txt"]

          # Step 5: TEST_OUTPUT summary (SUCCESS / WARNING / FAILURE + counts)
          - name: summary
            image: quay.io/conforma/cli:latest
            onError: continue
            command: ["jq"]
            args: [".", "$(results.TEST_OUTPUT.path)"]

          # Step 6: ec version โ€” recorded for reproducibility and audit
          - name: version
            image: quay.io/conforma/cli:latest
            command: ["ec"]
            args: ["version"]

          # Step 7: Assert.
          # With STRICT=false: always exits 0 โ€” pipeline continues regardless of violations.
          # With STRICT=true:  exits non-zero if result is FAILURE โ€” pipeline and release blocked.
          # Change STRICT param to "true" once all EC rules pass cleanly.
          - name: assert
            image: quay.io/conforma/cli:latest
            env:
              - name: STRICT
                value: "$(params.STRICT)"
            command: ["jq"]
            args:
              - "--argjson"
              - "strict"
              - "$(params.STRICT)"
              - "-e"
              - '.result == "SUCCESS" or .result == "WARNING" or ($strict | not)'
              - "$(results.TEST_OUTPUT.path)"
      params:
        - name: image
          value: "$(tasks.extract-images.results.image)"
        - name: POLICY_CONFIGURATION
          value: "enterprise-contract-service/default"
        - name: PUBLIC_KEY
          value: "k8s://openshift-pipelines/public-key"
        - name: IGNORE_REKOR
          value: "true"
        - name: STRICT
          value: "false"

    # โ”€โ”€ Task 3: Push the released image to the staging registry โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    # Runs only after EC validation completes (STRICT=false means it always runs;
    # set STRICT=true to block here on violations).
    # Uses skopeo with regcred credentials written inline to /tmp/auth/config.json.
    - name: push-to-staging-registry
      runAfter:
        - validate-enterprise-contract
      taskSpec:
        params:
          - name: source-image
            type: string
          - name: target-registry
            type: string
            # Replace YOUR-ORG with your staging quay.io org/username
            default: "quay.io/YOUR-ORG"
        steps:
          - name: copy-image
            image: quay.io/skopeo/stable:latest
            env:
              - name: SOURCE
                value: "$(params.source-image)"
              - name: TARGET
                value: "$(params.target-registry)/testrepo:latest"
              - name: DOCKER_CONFIG_JSON
                valueFrom:
                  secretKeyRef:
                    name: regcred
                    key: .dockerconfigjson
            script: |
              #!/bin/bash
              set -e
              mkdir -p /tmp/auth
              printf '%s' "$DOCKER_CONFIG_JSON" > /tmp/auth/config.json
              echo "Copying: $SOURCE โ†’ $TARGET"
              skopeo copy \
                --authfile /tmp/auth/config.json \
                "docker://$SOURCE" \
                "docker://$TARGET"
              echo "Image pushed to staging registry."
              rm -f /tmp/auth/config.json
      params:
        - name: source-image
          value: "$(tasks.extract-images.results.image)"

    # โ”€โ”€ Task 4: Tag the released image as stable โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    - name: post-release-actions
      runAfter:
        - push-to-staging-registry
      taskSpec:
        params:
          - name: image
            type: string
          - name: release-name
            type: string
          - name: target-registry
            type: string
            # Replace YOUR-ORG with your staging quay.io org/username
            default: "quay.io/YOUR-ORG"
        steps:
          - name: tag-stable
            image: quay.io/skopeo/stable:latest
            env:
              - name: IMAGE
                value: "$(params.image)"
              - name: RELEASE
                value: "$(params.release-name)"
              - name: TARGET_REGISTRY
                value: "$(params.target-registry)"
              - name: DOCKER_CONFIG_JSON
                valueFrom:
                  secretKeyRef:
                    name: regcred
                    key: .dockerconfigjson
            script: |
              #!/bin/bash
              set -e

              mkdir -p /tmp/auth
              printf '%s' "$DOCKER_CONFIG_JSON" > /tmp/auth/config.json

              echo "=================================================="
              echo "           RELEASE SUMMARY"
              echo "=================================================="
              echo "Release name   : $RELEASE"
              echo "Released image : $IMAGE"
              echo "=================================================="

              STABLE_TAG="$TARGET_REGISTRY/testrepo:stable"
              echo ""
              echo "Tagging as stable: $STABLE_TAG"
              skopeo copy \
                --authfile /tmp/auth/config.json \
                "docker://$IMAGE" \
                "docker://$STABLE_TAG"
              echo "Stable tag updated."
              rm -f /tmp/auth/config.json
      params:
        - name: image
          value: "$(tasks.extract-images.results.image)"
        - name: release-name
          value: "$(params.release)"
        - name: target-registry
          value: "quay.io/YOUR-ORG"
release.yaml (apply this to manually trigger a release against a passing Snapshot)
apiVersion: appstudio.redhat.com/v1alpha1
kind: Release
metadata:
  # Give each release a unique, descriptive name
  name: testrepo-release-v1
  # Must match your tenant namespace
  namespace: default-tenant
spec:
  # The Snapshot to release โ€” must have passed all integration tests.
  # Get the name of the LATEST passing Snapshot (sort by creationTimestamp descending):
  #   oc get snapshot -n default-tenant -o json | \
  #     jq -r '[.items[] | select(.status.conditions // [] | any(.type == "AppStudioTestSucceeded" and .status == "True"))] | sort_by(.metadata.creationTimestamp) | reverse | .[0].metadata.name'
  snapshot: testrepo-manual-snapshot

  # Must match the ReleasePlan name in release-plan.yaml
  releasePlan: testrepo-release-plan

Commands

terminal
NAMESPACE="default-tenant"

# Apply the EnterpriseContractPolicy FIRST
# (ReleasePlanAdmission references it by name โ€” release fails with "not found" without this)
oc apply -f enterprise-contract-policy.yaml -n $NAMESPACE

# Verify it was created
oc get enterprisecontractpolicy testrepo-ec-policy -n $NAMESPACE

# Apply the ReleasePlan and ReleasePlanAdmission โ€” both in default-tenant
oc apply -f release-plan.yaml -n $NAMESPACE
oc apply -f release-plan-admission.yaml -n $NAMESPACE

# Verify they are matched
oc get releaseplan -n $NAMESPACE
oc get releaseplanadmission -n $NAMESPACE

# โ”€โ”€ Next: commit the release pipeline to your testrepo fork โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# With auto-release: "true" set on the ReleasePlan, the Release Service will
# automatically create a Release CR after every Snapshot that passes integration
# tests. No manual "oc apply -f release.yaml" is needed.
#
# In your local testrepo fork clone, run:
#   mkdir -p release
#   cp /path/to/07-release-planning/release/release-pipeline.yaml release/
#   # Replace YOUR-ORG-staging with your Quay.io org:
#   perl -pi -e 's/YOUR-ORG-staging/your-quay-org/g' release/release-pipeline.yaml
#   git add release/release-pipeline.yaml
#   git commit -m "feat: add release pipeline for staging promotion"
#   git push origin main
#
# The git push triggers the push pipeline โ†’ integration tests โ†’ Snapshot is
# created with AppStudioTestSucceeded=True โ†’ Release Service auto-creates a
# Release CR โ†’ release pipeline runs automatically.

# โ”€โ”€ Wait for push pipeline โ†’ integration tests โ†’ release to complete โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Before running the watch commands below, ensure:
#   1. The push pipeline PipelineRun has reached Succeeded
#   2. Integration tests have passed (AppStudioTestSucceeded=True on Snapshot)
#   3. The Release CR has been auto-created by the Release Service

# Watch the Release status (auto-created โ€” no manual oc apply -f release.yaml)
oc get release -n $NAMESPACE -w

RELEASE_NAME=$(oc get release -n $NAMESPACE \
  --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{.items[-1].metadata.name}')
oc describe release "$RELEASE_NAME" -n $NAMESPACE

# Watch the release PipelineRun (runs in the same default-tenant namespace)
tkn pipelinerun list -n $NAMESPACE
tkn pipelinerun logs --last -f -n $NAMESPACE

# See the full Release status with conditions
oc get release "$RELEASE_NAME" -n $NAMESPACE \
  -o jsonpath='{.status.conditions}' | python3 -m json.tool

# โ”€โ”€ Manual release alternative (if auto-release is not triggered) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Check that a Snapshot has passed all integration tests
oc get snapshot -n $NAMESPACE -o wide

# Get the name of a passing Snapshot
# Collect all passing snapshots, sort by creationTimestamp descending, pick the latest.
# Using an array ([ ]) + sort_by + reverse ensures we always get the newest passing
# snapshot, not the oldest (which may be an ephemeral PR build snapshot).
SNAPSHOT=$(oc get snapshot -n $NAMESPACE -o json | \
  jq -r '[.items[] |
    select(
      .status.conditions // [] |
      any(.type == "AppStudioTestSucceeded" and .status == "True")
    )] | sort_by(.metadata.creationTimestamp) | reverse | .[0].metadata.name')
echo "Passing snapshot: $SNAPSHOT"

# Before applying, edit release.yaml and set the snapshot field to the
# value printed by the command above, then apply:
#
#   spec:
#     snapshot: <paste the $SNAPSHOT value here>  # e.g. my-first-app-abc123def
#     releasePlan: testrepo-release-plan
#
# Linux / macOS:
#   sed -i 's/testrepo-manual-snapshot/'"$SNAPSHOT"'/g' release.yaml          # Linux
#   sed -i '' 's/testrepo-manual-snapshot/'"$SNAPSHOT"'/g' release.yaml       # macOS
#   perl -pi -e "s/testrepo-manual-snapshot/$SNAPSHOT/g" release.yaml         # cross-platform

# Trigger the release manually (only if auto-release did not fire)
oc apply -f release.yaml

Key Takeaways

  • ReleasePlan + ReleasePlanAdmission both in default-tenant for this learning setup โ€” no separate managed namespace needed
  • Release object triggers the release pipeline against a Snapshot
  • Release pipeline can push to a staging registry, tag as stable, update GitOps, etc.
  • Auto-release: set label auto-release: "true" on ReleasePlan (also requires standing-attribution: "true")
  • Enterprise Contract validates images before any release proceeds (commented out due to OPA version incompatibility โ€” re-enable with a compatible EC task)
  • Release status is captured in the Release CRD conditions

Enterprise Contract & SLSA โ€” Policy-Driven Release Gating

What You'll Learn

What Enterprise Contract (EC) is and how it gates releases in the pipeline
Installing the ec CLI for standalone validation outside the pipeline
Validating your own Konflux-built and signed testrepo image with ec validate image
Interpreting EC output โ€” violations, warnings, successes โ€” in both text and JSON format
Verifying the cosign image signature and inspecting the SLSA provenance attestation
Understanding which rules the @redhat collection enforces and what violations mean

EC Validation in the Release Pipeline โ€” What Worked

โœ… EC validation runs successfully in the release pipeline

The release pipeline (release/release-pipeline.yaml) includes an inline validate-enterprise-contract task that uses quay.io/conforma/cli:latest directly โ€” no external task bundle. The task validates the released image against the enterprise-contract-service/default policy and produces a detailed report.

  • 144 successes โ€” all @redhat collection rules pass (signing, attestation, trusted tasks, etc.)
  • 1 violation โ€” slsa_source_correlated.source_code_reference_provided: satisfying this requires EC config files committed to the source repo at the exact build commit. Non-blocking with STRICT: false.
  • 4 warnings โ€” trusted_task.current: some task bundles in the build pipeline have newer versions available. Fix by updating the digests in .tekton/testrepo-push.yaml.

The standalone ec CLI commands below let you explore the same validation interactively against your own built image.

Commands

terminal
# โ”€โ”€ Step 1: Install the Enterprise Contract (Conforma) CLI โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Apple Silicon (macOS arm64):
curl -sLO https://github.com/conforma/cli/releases/latest/download/ec_darwin_arm64
chmod 755 ec_darwin_arm64
sudo mv ec_darwin_arm64 /usr/local/bin/ec

# Intel Mac (macOS amd64):
curl -sLO https://github.com/conforma/cli/releases/latest/download/ec_darwin_amd64
chmod 755 ec_darwin_amd64
sudo mv ec_darwin_amd64 /usr/local/bin/ec

# Linux (amd64):
curl -sLO https://github.com/conforma/cli/releases/latest/download/ec_linux_amd64
chmod 755 ec_linux_amd64
sudo mv ec_linux_amd64 /usr/local/bin/ec

ec version

# โ”€โ”€ Step 2: Inspect the EC policy in use โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# The release pipeline uses enterprise-contract-service/default (cluster managed).
# We created a mirror of it in default-tenant as testrepo-ec-policy (item 19).

# Inspect the cluster-managed default policy
oc get enterprisecontractpolicy default -n enterprise-contract-service -o yaml

# Inspect the testrepo mirror applied in default-tenant
oc get enterprisecontractpolicy testrepo-ec-policy -n default-tenant
oc describe enterprisecontractpolicy testrepo-ec-policy -n default-tenant

# โ”€โ”€ Step 3: Get the Tekton Chains public key โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Our policy uses k8s://openshift-pipelines/public-key โ€” this is the secret
# written by Tekton Chains containing the cosign public key for the cluster.
oc get secret public-key -n openshift-pipelines \
  -o jsonpath='{.data.cosign\.pub}' | base64 -d > /tmp/chains-public-key.pub
echo "Public key saved to /tmp/chains-public-key.pub"
cat /tmp/chains-public-key.pub

# โ”€โ”€ Step 4: Derive the image URL and digest โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# The release PipelineRun does not expose IMAGE_URL/IMAGE_DIGEST as Tekton results
# (those come from the build pipeline). Instead, read the image reference from the
# post-release-actions task log which prints "Released image : " explicitly.
NS="default-tenant"

# Get the last release PipelineRun name
PR=$(oc get pipelinerun -n $NS \
  --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{.items[-1].metadata.name}')
echo "Last PipelineRun: $PR"

# Extract the full image reference from the post-release-actions task logs
IMAGE_REF=$(tkn pipelinerun logs "$PR" -n $NS 2>/dev/null | \
  grep "Released image" | \
  awk '{print $NF}' | \
  tail -1)
echo "Image reference : $IMAGE_REF"

# Split into URL and digest (format: quay.io/org/repo@sha256:...)
IMAGE_URL="${IMAGE_REF%@*}"
IMAGE_DIGEST="${IMAGE_REF#*@}"

echo "Image URL   : $IMAGE_URL"
echo "Image Digest: $IMAGE_DIGEST"
echo "Full ref    : ${IMAGE_URL}@${IMAGE_DIGEST}"

# โ”€โ”€ Step 5: Export the policy to a local YAML file โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# ec v0.9.60 cannot parse k8s:// policy references โ€” it fails with:
#   "cannot unmarshal string into Go value of type v1alpha1.EnterpriseContractPolicySpec"
# The workaround: export the CR to a YAML file and pass the file path instead.
oc get enterprisecontractpolicy testrepo-ec-policy -n default-tenant -o yaml \
  > /tmp/testrepo-ec-policy.yaml
echo "Policy exported to /tmp/testrepo-ec-policy.yaml"

# โ”€โ”€ Step 6: Validate your testrepo image with the EC policy (text output) โ”€โ”€โ”€โ”€โ”€โ”€
# This runs the same validation the release pipeline performs.
# --ignore-rekor: required on self-hosted Konflux โ€” images are signed with the
#   cluster's internal Tekton Chains key, not recorded in the public Sigstore Rekor.
# --public-key:   the cosign public key that Tekton Chains used to sign the image.
# --policy:       local YAML file path (k8s:// reference not supported by this ec version).
ec validate image \
  --image "${IMAGE_URL}@${IMAGE_DIGEST}" \
  --policy /tmp/testrepo-ec-policy.yaml \
  --public-key /tmp/chains-public-key.pub \
  --ignore-rekor \
  --output text

# โ”€โ”€ Step 7: Full JSON output โ€” programmatic inspection โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
ec validate image \
  --image "${IMAGE_URL}@${IMAGE_DIGEST}" \
  --policy /tmp/testrepo-ec-policy.yaml \
  --public-key /tmp/chains-public-key.pub \
  --ignore-rekor \
  --output json > /tmp/ec-result.json

# Overall result and counts
jq '{result: .components[0].success,
     violations: (.components[0].violations | length),
     warnings:   (.components[0].warnings   | length),
     successes:  (.components[0].successes  | length)}' /tmp/ec-result.json

# List violations with their rule code and reason
jq -r '.components[0].violations[] | "VIOLATION \(.metadata.code): \(.msg)"' \
  /tmp/ec-result.json

# List warnings with their rule code and reason
jq -r '.components[0].warnings[] | "WARNING \(.metadata.code): \(.msg)"' \
  /tmp/ec-result.json

# โ”€โ”€ Step 8: Verify the cosign image signature directly โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# This confirms Tekton Chains produced and stored a valid cryptographic signature.
# --insecure-ignore-tlog: skip the public Rekor transparency log lookup โ€”
#   the signature was recorded by the cluster's internal Chains, not public Rekor.
cosign verify \
  --key /tmp/chains-public-key.pub \
  --insecure-ignore-tlog \
  "${IMAGE_URL}@${IMAGE_DIGEST}" | jq .

# โ”€โ”€ Step 9: Inspect the SLSA provenance attestation โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# Tekton Chains generates an in-toto SLSA v0.2 attestation encoding every build
# input: git commit, task images, parameters. EC validates this attestation.
cosign verify-attestation \
  --key /tmp/chains-public-key.pub \
  --insecure-ignore-tlog \
  --type slsaprovenance \
  "${IMAGE_URL}@${IMAGE_DIGEST}" \
  | jq -r '.payload | @base64d | fromjson | {
      predicateType,
      builder: .predicate.builder.id,
      buildType: .predicate.buildType,
      commit: .predicate.materials[0].digest.sha1,
      repo: .predicate.materials[0].uri,
      startedOn: .predicate.metadata.buildStartedOn,
      finishedOn: .predicate.metadata.buildFinishedOn
    }'

# โ”€โ”€ Step 10: Download and inspect the SBOM โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# The build pipeline attaches an SBOM (Software Bill of Materials) to the image.
# cosign download sbom fetches the SBOM OCI artifact attached to the image digest.
cosign download sbom "${IMAGE_URL}@${IMAGE_DIGEST}" | python3 -m json.tool | head -60

# โ”€โ”€ Step 11: Test with an unsigned image โ€” confirm EC catches it โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# alpine:latest has no cosign signature and no SLSA attestation.
# EC will always fail on this image โ€” useful to understand what failure looks like.
ec validate image \
  --image "docker.io/library/alpine:latest" \
  --policy /tmp/testrepo-ec-policy.yaml \
  --public-key /tmp/chains-public-key.pub \
  --ignore-rekor \
  --output text || true
# Expected: FAILURE โ€” no signature, no SLSA provenance attestation

Key Takeaways

  • EC validation runs in the release pipeline via an inline taskSpec using quay.io/conforma/cli:latest โ€” no external task bundle needed
  • The @redhat collection enforces 144+ rules: image signing, SLSA provenance, trusted task bundles, SBOM presence, and more
  • For cluster-signed images use --ignore-rekor and --public-key k8s://openshift-pipelines/public-key โ€” signatures are written by the internal Tekton Chains, not the public Sigstore Rekor
  • --policy accepts k8s://namespace/policy-name (cluster CR) โ€” the OCI bundle URL belongs inside the policy CR's sources field, not on the CLI directly
  • STRICT: false reports violations without blocking the release; switch to true for full enforcement once all rules pass
  • Update task bundle digests in .tekton/testrepo-push.yaml to eliminate trusted_task.current warnings โ€” EC reads these from the SLSA attestation of the built image

Multi-Architecture Builds โ€” amd64, arm64, and s390x with Konflux

What You'll Learn

Why multi-arch matters: OpenShift on IBM Z, ARM edge, and x86 cloud
How Konflux's matrix builds produce per-arch images in parallel
Merging per-arch digests into an OCI multi-arch manifest list
Enabling multi-arch on an existing Component โ€” no new repo or Dockerfile needed
Why the multi-arch pipeline must replace the single-arch push pipeline โ€” not coexist with it
Verifying the manifest list with skopeo inspect and releasing it with the same release pipeline

Steps

  1. Remove the single-arch push pipeline and add the multi-arch one โ€” in your testrepo fork, run git rm .tekton/testrepo-push.yaml and add .tekton/multiarch-push.yaml from the YAML Files section. Both files share the same CEL expression (event == "push" && target_branch == "main"), so the old file must be removed to prevent two PipelineRuns firing on every push.
  2. Update the placeholders and review the build-platforms list โ€” replace YOUR-USERNAME and YOUR-ORG, then confirm build-platforms lists linux/amd64, linux/arm64, and linux/s390x and the pipeline bundle is pinned by digest.
  3. Confirm release/release-pipeline.yaml is present in your repo โ€” this file was committed in item 19 and already contains skopeo copy --all, which copies the entire OCI manifest list. No changes to the release pipeline are needed for multi-arch.
  4. Commit and push to main โ€” PaC fires the testrepo-multiarch-on-push PipelineRun. Run oc get taskruns -n default-tenant -w and watch three build-images TaskRuns appear simultaneously, one per architecture.
  5. Verify the OCI Image Index โ€” once the PipelineRun completes, run skopeo inspect --raw against the output image digest. Confirm mediaType is application/vnd.oci.image.index.v1+json and that entries for all three platforms appear in the manifests array.
  6. Confirm the auto-release promotes the multi-arch index โ€” after integration tests pass, the Release Service creates a Release object automatically. Follow the release PipelineRun and verify the full OCI Image Index lands in the staging registry.

Prerequisite: Multi-Platform Controller

Multi-arch builds require the Multi-Platform Controller โ€” it is NOT included in a default CRC install

The pipeline-docker-build-multi-platform-oci-ta bundle does not use QEMU emulation or local cross-compilation. For every platform entry in build-platforms, it creates a TaskRun that waits for a secret named multi-platform-ssh-<taskrun-name>. That secret is created by the Multi-Platform Controller, a separate Konflux add-on that provisions remote VMs (or selects from a static pool) and handles SSH key exchange via a one-time-password server. If the controller is not running, the build pod will hang indefinitely waiting for a secret that never arrives โ€” even if you only specify a single platform.

What the controller does (lifecycle per build):

  1. Detects the waiting TaskRun (identified by the missing secret and the PLATFORM param)
  2. Provisions a VM of the correct native architecture from a cloud provider (AWS, IBM Z, IBM Power) or selects a host from a static pool
  3. Creates a per-build non-privileged user and SSH keypair on the remote host
  4. Sends the private key to an OTP server; writes the one-time password into the secret
  5. The build task redeems the OTP once to get the SSH key, SSHes into the host, and runs buildah natively
  6. On completion, a cleanup task removes the per-build user from the remote host

Supported host allocation modes:

  • Static pool โ€” a fixed set of machines you already own, configured with address, SSH key, and concurrency limit in the host-config ConfigMap
  • Dynamic โ€” cloud VMs spun up per build and torn down after; supports AWS (Graviton for arm64), IBM Z (s390x), IBM Power (ppc64le)
  • Dynamic pool โ€” shared VMs that scale to zero when idle and are recycled after a configurable max-age
  • Local โ€” no provisioning; the secret points to localhost; the task runs inside the cluster pod itself (useful for native-arch builds on the cluster node)

Configuration is driven entirely by a ConfigMap named host-config in the multi-platform-controller namespace โ€” no CRDs required. For the full architecture, configuration reference, and cloud provider field lists, refer to the official architecture document: konflux-ci/architecture โ€” multi-platform-controller.md .

On CRC (OpenShift Local): the controller is not installed and there are no remote build hosts. The build pods will always hang with MountVolume.SetUp failed โ€ฆ secret "multi-platform-ssh-โ€ฆ" not found. The practical options are:

  • Use the hosted Konflux (console.redhat.com/application-pipeline) where the controller and native arm64/amd64/s390x hosts are pre-configured
  • Install the controller on a full OpenShift cluster and register remote hosts in the host-config ConfigMap
  • Use the standard single-arch bundle (pipeline-docker-build-oci-ta) on CRC โ€” on a Silicon Mac the CRC node is natively arm64, so you get a real linux/arm64 image without any remote SSH machinery

YAML Files

Merging multi-arch into the existing push pipeline

There is no need to create a new repository or onboard a new Component. The testrepo fork deployed in item 14 already has a Dockerfile and is registered as a Konflux Component.

The multi-arch pipeline is a full replacement, not a stripped-down build. The pipeline-docker-build-multi-platform-oci-ta bundle referenced via pipelineRef is a complete Konflux pipeline โ€” it contains all the same standard tasks as the original single-arch pipeline (init, clone, prefetch, SAST, Clair scan, ClamAV, EC scan, apply-tags, push-dockerfile) plus the Tekton Matrix that fans out build-images across platforms and the build-image-index task that merges the per-arch digests into a single OCI Image Index. No checks are dropped.

The only thing not carried over is the custom print-build-summary task added in item 17 as a Pipeline-as-Code teaching example โ€” that was illustrative, not a real check.

Why replace, not add alongside: .tekton/multiarch-push.yaml must replace the existing .tekton/testrepo-push.yaml. Both files carry the same CEL expression (event == "push" && target_branch == "main"), so keeping both causes two PipelineRuns to fire on every push โ€” the old single-arch one and the new multi-arch one simultaneously. Delete testrepo-push.yaml and commit multiarch-push.yaml in its place.

.tekton/multiarch-push.yaml (replaces .tekton/testrepo-push.yaml โ€” switches to the multi-platform bundle and adds build-platforms)
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  annotations:
    # Replace YOUR-USERNAME with your GitHub username (owner of your testrepo fork)
    build.appstudio.openshift.io/repo: https://github.com/YOUR-USERNAME/testrepo?rev={{revision}}
    build.appstudio.redhat.com/commit_sha: '{{revision}}'
    build.appstudio.redhat.com/target_branch: '{{target_branch}}'
    pipelinesascode.tekton.dev/cancel-in-progress: "false"
    pipelinesascode.tekton.dev/max-keep-runs: "3"
    pipelinesascode.tekton.dev/on-cel-expression: event == "push" && target_branch == "main"
  labels:
    # Must match the Application and Component CRs created in item 14
    appstudio.openshift.io/application: my-first-app
    appstudio.openshift.io/component: testrepo
    pipelines.appstudio.openshift.io/type: build
  name: testrepo-multiarch-on-push
  namespace: default-tenant
spec:
  params:
  - name: git-url
    value: '{{source_url}}'
  - name: revision
    value: '{{revision}}'
  # Replace YOUR-ORG with your quay.io username or org
  # Must match the containerImage field in your Component CR
  - name: output-image
    value: quay.io/YOUR-ORG/testrepo:{{revision}}
  - name: dockerfile
    value: Dockerfile
  # โ”€โ”€ The key addition: build-platforms โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
  # Tekton Matrix fans out one build-container TaskRun per entry, all running
  # in parallel. The per-arch images are then merged into a single OCI Image Index.
  - name: build-platforms
    value:
    - linux/amd64
    - linux/arm64
    - linux/s390x
  pipelineRef:
    # Switch from the standard docker-build bundle to the multi-platform variant.
    # This bundle is a COMPLETE pipeline โ€” it includes all standard Konflux tasks
    # (init, clone, SAST, Clair scan, ClamAV, EC scan, apply-tags, push-dockerfile)
    # plus the Tekton Matrix that fans out build-images per platform and the
    # build-image-index task that merges per-arch digests into an OCI Image Index.
    # No checks from the original testrepo-push.yaml are lost by switching here.
    resolver: bundles
    params:
    - name: bundle
      value: quay.io/konflux-ci/tekton-catalog/pipeline-docker-build-multi-platform-oci-ta@sha256:17a3c94c33742e19f35c2516bbd7fab4ed702838c7a6d47daea70100d350e266
    - name: name
      value: docker-build-multi-platform-oci-ta
    - name: kind
      value: pipeline
  taskRunTemplate:
    serviceAccountName: build-pipeline-testrepo
  workspaces:
  - name: git-auth
    secret:
      secretName: '{{ git_auth_secret }}'
status: {}
release/release-pipeline.yaml (already in your testrepo fork from item 19 โ€” no changes needed for multi-arch)
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: testrepo-release-to-staging
  namespace: default-tenant
spec:
  description: >-
    Production release pipeline for testrepo.
    Validates the built image with Enterprise Contract (Conforma) then promotes
    it to the staging registry. Registry credentials come from the regcred secret
    (kubernetes.io/dockerconfigjson) mounted as a projected volume โ€” this bypasses
    Tekton cred-init and provides stable credentials to all OCI tools (skopeo,
    cosign, ec) regardless of the step user identity.
  params:
    - name: release
      type: string
      description: "The Release CRD name โ€” injected by the Release Service"
    - name: releasePlan
      type: string
      description: "The ReleasePlan CRD name"
    - name: releasePlanAdmission
      type: string
      description: "The ReleasePlanAdmission CRD name"
    - name: snapshot
      type: string
      description: "Snapshot reference โ€” injected by the Release Service as namespace/name"
    - name: enterpriseContractPolicy
      type: string
      description: "Enterprise Contract policy name (not used directly โ€” policy is hardcoded below)"

  tasks:

    # โ”€โ”€ Task 1: Extract the image reference from the Snapshot โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    # The Release Service passes snapshot as "namespace/name" (e.g.
    # "default-tenant/my-first-app-abc123"). This task fetches the Snapshot CR
    # from the cluster and extracts the containerImage for the testrepo component.
    # For a multi-arch build the containerImage is the OCI Image Index digest โ€”
    # skopeo copy in Task 3 handles manifest lists transparently.
    - name: extract-images
      taskSpec:
        params:
          - name: snapshot
        results:
          - name: image
            description: "containerImage for the testrepo component"
        steps:
          - name: extract
            image: quay.io/ongres/kubectl@sha256:4be5050c456a4751fe3d70086c0387d65e8973a176fe965de4eaeeb8643b2e0a
            env:
              - name: SNAPSHOT_REF
                value: "$(params.snapshot)"
            script: |
              #!/bin/bash
              set -e

              echo "Snapshot reference: $SNAPSHOT_REF"

              if [[ "$SNAPSHOT_REF" == *"/"* ]]; then
                SNAP_NS="${SNAPSHOT_REF%%/*}"
                SNAP_NAME="${SNAPSHOT_REF##*/}"
              else
                SNAP_NS="default-tenant"
                SNAP_NAME="$SNAPSHOT_REF"
              fi

              echo "Fetching snapshot '$SNAP_NAME' from namespace '$SNAP_NS'"

              IMAGE=$(kubectl get snapshot "$SNAP_NAME" -n "$SNAP_NS" -o json | \
                jq -r '.spec.components[] | select(.name=="testrepo") | .containerImage')

              if [ -z "$IMAGE" ] || [ "$IMAGE" = "null" ]; then
                echo "ERROR: component 'testrepo' not found in snapshot" >&2
                kubectl get snapshot "$SNAP_NAME" -n "$SNAP_NS" -o yaml >&2
                exit 1
              fi

              echo "Extracted image: $IMAGE"
              printf '%s' "$IMAGE" > $(results.image.path)
      params:
        - name: snapshot
          value: "$(params.snapshot)"

    # โ”€โ”€ Task 2: Enterprise Contract validation โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    - name: validate-enterprise-contract
      runAfter:
        - extract-images
      taskSpec:
        params:
          - name: image
            type: string
          - name: POLICY_CONFIGURATION
            type: string
            default: "enterprise-contract-service/default"
          - name: PUBLIC_KEY
            type: string
            default: "k8s://openshift-pipelines/public-key"
          - name: IGNORE_REKOR
            type: string
            default: "true"
          - name: STRICT
            type: string
            default: "false"
          - name: WORKERS
            type: string
            default: "4"
        results:
          - name: TEST_OUTPUT
            description: "EC policy evaluation result โ€” SUCCESS, WARNING, or FAILURE"
        volumes:
          - name: workdir
            emptyDir: {}
          - name: trusted-ca
            configMap:
              name: trusted-ca
              items:
                - key: ca-bundle.crt
                  path: ca-bundle.crt
              optional: true
          - name: docker-config
            projected:
              sources:
                - secret:
                    name: regcred
                    items:
                      - key: .dockerconfigjson
                        path: config.json
        stepTemplate:
          volumeMounts:
            - mountPath: /var/workdir
              name: workdir
            - mountPath: /mnt/trusted-ca
              name: trusted-ca
              readOnly: true
            - mountPath: /var/docker-config
              name: docker-config
              readOnly: true
          env:
            - name: HOME
              value: /tekton/home
            - name: DOCKER_CONFIG
              value: /var/docker-config

        steps:
          - name: write-snapshot
            image: registry.access.redhat.com/ubi9/ubi-minimal:latest
            env:
              - name: IMAGE
                value: "$(params.image)"
            script: |
              #!/bin/bash
              set -euo pipefail
              mkdir -p /var/workdir
              printf '%s' \
                "{\"components\":[{\"name\":\"testrepo\",\"containerImage\":\"$IMAGE\"}]}" \
                > /var/workdir/snapshot.json
              echo "snapshot.json:"
              cat /var/workdir/snapshot.json
              mkdir -p /tekton/home/.docker
              chmod 777 /tekton/home/.docker
              if [ -f /tekton/home/.docker/config.json ]; then
                chmod 666 /tekton/home/.docker/config.json
              fi

          - name: initialize-tuf
            image: quay.io/conforma/cli:latest
            script: |
              echo 'TUF_MIRROR not configured โ€” skipping TUF root initialization.'

          - name: validate
            image: quay.io/conforma/cli:latest
            onError: continue
            env:
              - name: POLICY_CONFIGURATION
                value: "$(params.POLICY_CONFIGURATION)"
              - name: PUBLIC_KEY
                value: "$(params.PUBLIC_KEY)"
              - name: IGNORE_REKOR
                value: "$(params.IGNORE_REKOR)"
              - name: WORKERS
                value: "$(params.WORKERS)"
            script: |
              #!/bin/bash
              set -euo pipefail
              if [ -f "/mnt/trusted-ca/ca-bundle.crt" ]; then
                export SSL_CERT_FILE="/mnt/trusted-ca/ca-bundle.crt"
              fi
              ec validate image \
                --images=/var/workdir/snapshot.json \
                --policy="${POLICY_CONFIGURATION}" \
                --public-key="${PUBLIC_KEY}" \
                --ignore-rekor="${IGNORE_REKOR}" \
                --workers="${WORKERS}" \
                --strict=false \
                --info=true \
                --timeout=0 \
                --output="text=/var/workdir/text-report.txt?show-successes=false" \
                --output="json=/var/workdir/report-json.json" \
                --output="appstudio=$(results.TEST_OUTPUT.path)"

          - name: detailed-report
            image: quay.io/conforma/cli:latest
            onError: continue
            command: ["cat"]
            args: ["/var/workdir/text-report.txt"]

          - name: summary
            image: quay.io/conforma/cli:latest
            onError: continue
            command: ["jq"]
            args: [".", "$(results.TEST_OUTPUT.path)"]

          - name: version
            image: quay.io/conforma/cli:latest
            command: ["ec"]
            args: ["version"]

          - name: assert
            image: quay.io/conforma/cli:latest
            env:
              - name: STRICT
                value: "$(params.STRICT)"
            command: ["jq"]
            args:
              - "--argjson"
              - "strict"
              - "$(params.STRICT)"
              - "-e"
              - '.result == "SUCCESS" or .result == "WARNING" or ($strict | not)'
              - "$(results.TEST_OUTPUT.path)"
      params:
        - name: image
          value: "$(tasks.extract-images.results.image)"
        - name: POLICY_CONFIGURATION
          value: "enterprise-contract-service/default"
        - name: PUBLIC_KEY
          value: "k8s://openshift-pipelines/public-key"
        - name: IGNORE_REKOR
          value: "true"
        - name: STRICT
          value: "false"

    # โ”€โ”€ Task 3: Push the released image to the staging registry โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    # skopeo copy handles OCI manifest lists transparently โ€” when the source image
    # is a multi-arch index, the entire index (all per-arch manifests) is copied.
    - name: push-to-staging-registry
      runAfter:
        - validate-enterprise-contract
      taskSpec:
        params:
          - name: source-image
            type: string
          - name: target-registry
            type: string
            # Replace YOUR-ORG with your staging quay.io org/username
            default: "quay.io/YOUR-ORG"
        steps:
          - name: copy-image
            image: quay.io/skopeo/stable:latest
            env:
              - name: SOURCE
                value: "$(params.source-image)"
              - name: TARGET
                value: "$(params.target-registry)/testrepo:latest"
              - name: DOCKER_CONFIG_JSON
                valueFrom:
                  secretKeyRef:
                    name: regcred
                    key: .dockerconfigjson
            script: |
              #!/bin/bash
              set -e
              mkdir -p /tmp/auth
              printf '%s' "$DOCKER_CONFIG_JSON" > /tmp/auth/config.json
              echo "Copying: $SOURCE โ†’ $TARGET"
              skopeo copy \
                --authfile /tmp/auth/config.json \
                --all \
                "docker://$SOURCE" \
                "docker://$TARGET"
              echo "Multi-arch index pushed to staging registry."
              rm -f /tmp/auth/config.json
      params:
        - name: source-image
          value: "$(tasks.extract-images.results.image)"

    # โ”€โ”€ Task 4: Tag the released image as stable โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    - name: post-release-actions
      runAfter:
        - push-to-staging-registry
      taskSpec:
        params:
          - name: image
            type: string
          - name: release-name
            type: string
          - name: target-registry
            type: string
            # Replace YOUR-ORG with your staging quay.io org/username
            default: "quay.io/YOUR-ORG"
        steps:
          - name: tag-stable
            image: quay.io/skopeo/stable:latest
            env:
              - name: IMAGE
                value: "$(params.image)"
              - name: RELEASE
                value: "$(params.release-name)"
              - name: TARGET_REGISTRY
                value: "$(params.target-registry)"
              - name: DOCKER_CONFIG_JSON
                valueFrom:
                  secretKeyRef:
                    name: regcred
                    key: .dockerconfigjson
            script: |
              #!/bin/bash
              set -e
              mkdir -p /tmp/auth
              printf '%s' "$DOCKER_CONFIG_JSON" > /tmp/auth/config.json
              echo "=================================================="
              echo "           RELEASE SUMMARY"
              echo "=================================================="
              echo "Release name   : $RELEASE"
              echo "Released image : $IMAGE"
              echo "=================================================="
              STABLE_TAG="$TARGET_REGISTRY/testrepo:stable"
              echo ""
              echo "Tagging as stable: $STABLE_TAG"
              skopeo copy \
                --authfile /tmp/auth/config.json \
                --all \
                "docker://$IMAGE" \
                "docker://$STABLE_TAG"
              echo "Stable tag updated."
              rm -f /tmp/auth/config.json
      params:
        - name: image
          value: "$(tasks.extract-images.results.image)"
        - name: release-name
          value: "$(params.release)"
        - name: target-registry
          value: "quay.io/YOUR-ORG"
verify-manifest.sh (run after the build to confirm the OCI Image Index was created correctly)
#!/bin/bash
# Verify that the built testrepo image is a proper multi-arch OCI Image Index.
# Run this after the multiarch-push pipeline completes.
#
# Usage:
#   IMAGE_URL=quay.io/YOUR-ORG/testrepo
#   DIGEST=$(oc get pipelinerun -n default-tenant \
#     --sort-by=.metadata.creationTimestamp \
#     -o jsonpath='{.items[-1].status.results[?(@.name=="IMAGE_DIGEST")].value}')
#   bash verify-manifest.sh "$IMAGE_URL" "$DIGEST"

IMAGE="${1:-quay.io/YOUR-ORG/testrepo}"
DIGEST="${2:-sha256:REPLACE_WITH_ACTUAL_DIGEST}"

echo "=== Multi-Arch Manifest Verification ==="
echo "Image : $IMAGE"
echo "Digest: $DIGEST"
echo ""

# 1. Raw manifest โ€” mediaType must be application/vnd.oci.image.index.v1+json
echo "--- 1. Raw OCI Image Index manifest ---"
skopeo inspect --raw "docker://${IMAGE}@${DIGEST}" | python3 -m json.tool

echo ""
echo "--- 2. Architectures in the manifest list ---"
skopeo inspect --raw "docker://${IMAGE}@${DIGEST}" | \
  python3 -c "
import sys, json
data = json.load(sys.stdin)
media = data.get('mediaType', data.get('schemaVersion', '?'))
print(f'mediaType: {media}')
print('platforms:')
for m in data.get('manifests', []):
    p = m.get('platform', {})
    print(f'  {p.get(\"os\",\"?\")}/{p.get(\"architecture\",\"?\")}: {m[\"digest\"][:32]}...')
"

echo ""
echo "--- 3. Full skopeo inspect (resolves to current host arch) ---"
skopeo inspect "docker://${IMAGE}@${DIGEST}" | python3 -m json.tool

Commands

terminal
NAMESPACE="default-tenant"

# In your local testrepo fork clone, replace the single-arch push pipeline with
# the multi-arch one. Both share the same CEL trigger, so the old file must be
# removed to avoid two PipelineRuns firing on every push to main.
git rm .tekton/testrepo-push.yaml
git add .tekton/multiarch-push.yaml
git commit -m "feat: replace single-arch push pipeline with multi-arch (amd64, arm64, s390x)"
git push origin main

# PaC detects the updated .tekton/ directory and triggers testrepo-multiarch-on-push.
# Watch the matrix TaskRuns appear โ€” one per platform, all running in parallel:
oc get taskruns -n $NAMESPACE -w | grep build-container

# All three should show STATUS=Running simultaneously
oc get taskruns -n $NAMESPACE \
  --selector=tekton.dev/pipelineTask=build-container \
  -o 'custom-columns=NAME:.metadata.name,PLATFORM:.spec.params[?(@.name=="PLATFORM")].value,STATUS:.status.conditions[0].reason'

# Follow logs for the amd64 build specifically
AMD64_TR=$(oc get taskruns -n $NAMESPACE \
  --selector=tekton.dev/pipelineTask=build-container \
  -o jsonpath='{.items[?(@.spec.params[?(@.name=="PLATFORM")].value=="linux/amd64")].metadata.name}')
tkn taskrun logs "$AMD64_TR" -n $NAMESPACE -f

# After the push pipeline completes, get the manifest list digest
PR_NAME=$(oc get pipelinerun -n $NAMESPACE \
  --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{.items[-1].metadata.name}')

IMAGE_URL=$(oc get pipelinerun "$PR_NAME" -n $NAMESPACE \
  -o jsonpath='{.status.results[?(@.name=="IMAGE_URL")].value}')

MANIFEST_DIGEST=$(oc get pipelinerun "$PR_NAME" -n $NAMESPACE \
  -o jsonpath='{.status.results[?(@.name=="IMAGE_DIGEST")].value}')

echo "OCI Image Index: ${IMAGE_URL}@${MANIFEST_DIGEST}"

# Quick check โ€” mediaType should be application/vnd.oci.image.index.v1+json
skopeo inspect --raw "docker://${IMAGE_URL}@${MANIFEST_DIGEST}" | python3 -m json.tool

# Full verification script
bash verify-manifest.sh "$IMAGE_URL" "$MANIFEST_DIGEST"

# โ”€โ”€ Watch the auto-release fire โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# After integration tests pass on the new Snapshot, the Release Service
# auto-creates a Release object (auto-release: "true" on the ReleasePlan).
# The release pipeline from item 19 runs and promotes the multi-arch index:
oc get release -n $NAMESPACE -w

RELEASE_NAME=$(oc get release -n $NAMESPACE \
  --sort-by=.metadata.creationTimestamp \
  -o jsonpath='{.items[-1].metadata.name}')
oc describe release "$RELEASE_NAME" -n $NAMESPACE

# Watch the release PipelineRun logs
tkn pipelinerun logs --last -f -n $NAMESPACE

# Verify the staging registry received the full OCI index (all three arches)
skopeo inspect --raw "docker://quay.io/YOUR-ORG/testrepo:latest" | python3 -m json.tool

Key Takeaways

  • No new repo or Component needed โ€” .tekton/multiarch-push.yaml replaces (not adds alongside) .tekton/testrepo-push.yaml; both share the same CEL trigger so coexistence causes duplicate PipelineRuns on every push
  • The pipeline-docker-build-multi-platform-oci-ta bundle is a complete pipeline โ€” all standard checks (SAST, Clair, ClamAV, EC scan, etc.) are included; the only thing not carried over is the custom print-build-summary task from the item 17 tutorial
  • The only two changes needed versus the single-arch pipeline: pin the pipelineRef bundle to the multi-platform variant (by digest, not :latest) and add the build-platforms array param
  • Tekton Matrix fans out one build-container TaskRun per platform entry, all running in parallel from the same Dockerfile
  • Result is an OCI Image Index (manifest list) โ€” one digest that resolves to the correct arch image on any platform
  • The release pipeline from item 19 needs only --all added to the skopeo copy call to copy the entire manifest list to the staging registry
  • Verify with skopeo inspect --raw โ€” mediaType must be application/vnd.oci.image.index.v1+json; all per-arch images get individual SLSA attestations from Tekton Chains