OpenShift Serverless + OpenShift Service Mesh 3.0 integration

Resources

• Latest blogpost about OSSM 3.x
• 3.0 installation readme
• OSSM 3.x code base
• Using gateway injection
• sail-operator
• tech preview announcement
• OSSM 3.x docs

Future things to look at

• Enable mTLS set-up
• Check if we can move the gateways, OCP routes and so on to our own namespace knative-serving-ingress instead of istio-system
• Make all Serverless-Operator tests pass (Serving only)
• For the future: Relying on Gateway API resources with net-gateway-api instead of net-istio
• Update all usages of istio-inject annotation to the label, as per docs

Contents

• Installing OSSM 3.x
• Analysis of injection

Findings

• Basically, it is upstream istio with a helm operator to install it. You can use all helm values to configure istio.
• The injection is different to OSSM 2.x. We will need to have the istio-injection: enabled label on namespaces where we want proxies. This has the "downside" that all pods are injected with the istio-proxy. Where this is not applicable, we need to opt-out of it using the sidecar.istio.io/inject: 'false' label (or annotation on the StatefulSets of Eventing). More here.
• The changes to make tests pass are here: openshift-knative/serverless-operator#2928.
• We will probably need to CI jobs to test OSSM 2.x and OSSM 3.x as these are very different. To be decided by PM.
• To enforce mTLS on the mesh, we need a PeerAuthentication.
• We need a new DestinationRule to make mTLS (as there is no mtls: true anymore) work with DomainMappings.
• OSSM 3.x has no default istio-ingressgateway, we need to create our own deployment and RBAC for it. But with that, we can now host this in knative-serving-ingress namespace instead of istio-system. With that, this is aligned with Kourier.
• We need to deploy an istio-cni instance, otherwise OSSM 3.x does not work.
• The configuration is migrated from SMCP to Istio CR.
• As per discussion with the Service Mesh and RHOAI team, we omit the creation of NetworkPolices per default. If we add those, it becomes a "catch-all" policy, which denies everything else. As we don't know the setup up of a customer, we should rather document what communication paths we need and let the customer create the Policies as needed.

More details in the linked PR.