function-patch-and-transform
A Crossplane Composition Function that implements P&T-style Composition.
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
name: function-patch-and-transform
spec:
package: xpkg.upbound.io/crossplane-contrib/function-patch-and-transform:v0.1.3What is this?
This Composition Function does everything Crossplane's built-in Patch &
Transform Composition does. Instead of specifying spec.resources in your
Composition, you can use this Function.
Note that this is a beta-style Function. It won't work with Crossplane v1.13 or
earlier - it targets the implementation of Functions coming with
Crossplane v1.14 in late October.
Take this example from https://docs.crossplane.io. Using
this Function, it would look like this:
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: dynamo-with-bucket
spec:
compositeTypeRef:
apiVersion: database.example.com/v1alpha1
kind: NoSQL
mode: Pipeline
pipeline:
- step: patch-and-transform
functionRef:
name: function-patch-and-transform
input:
apiVersion: pt.fn.crossplane.io/v1beta1
kind: Resources
resources:
- name: s3Bucket
base:
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
name: crossplane-quickstart-bucket
spec:
forProvider:
region: us-east-2
patches:
- type: FromCompositeFieldPath
fromFieldPath: "location"
toFieldPath: "spec.forProvider.region"
transforms:
- type: map
map:
EU: "eu-north-1"
US: "us-east-2"
- name: dynamoDB
base:
apiVersion: dynamodb.aws.upbound.io/v1beta1
kind: Table
metadata:
name: crossplane-quickstart-database
spec:
forProvider:
region: "us-east-2"
writeCapacity: 1
readCapacity: 1
attribute:
- name: S3ID
type: S
hashKey: S3ID
patches:
- type: FromCompositeFieldPath
fromFieldPath: "spec.location"
toFieldPath: "spec.forProvider.region"
transforms:
- type: map
map:
EU: "eu-north-1"
US: "us-east-2"Notice that it looks pretty much identical to the example from the Crossplane
documentation. The key difference is that everything that used to be under
spec.resources is now nested a little deeper. Specifically, it's under
spec.pipeline[0].input.resources (i.e. in the Function's input).
Okay, but why?
I think there are a lot of good reasons to implement P&T Composition as a
Function. In fact, I would go so far as to propose that once Functions are a GA
feature we deprecate support for 'native' P&T (i.e. spec.resources). We can't
remove it - that would be a breaking change - but we can freeze its API and
suggest folks use the P&T Function instead.
Run P&T anywhere in your pipeline
Native P&T can only run before the Composition Function pipeline. In the draft
beta implementation of Functions Crossplane does all the patching
and transforming first, then sends the results through the Function pipeline.
This is handy, but what if you wanted to run another Function (like rendering
some Go templates) first, then pass the result of that Function to be patched
and transformed? With this Function you can do that:
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: dynamo-with-bucket
spec:
compositeTypeRef:
apiVersion: database.example.com/v1alpha1
kind: NoSQL
# This pipeliene renders some Go templates, then passes them to P&T
pipeline:
- step: render-go-templates
functionRef:
name: function-go-templates
input: {} # Omitted for brevity :)
- step: patch-and-transform
functionRef:
name: function-patch-and-transform
input:
apiVersion: pt.fn.crossplane.io/v1beta1
kind: Resources
resources:
# Notice that my-cool-bucket doesn't have a base template. As long as
# the render-go-templates step above rendered a composed resource with
# this name, this Function will patch it.
- name: my-cool-bucket
patches:
- type: FromCompositeFieldPath
fromFieldPath: "location"
toFieldPath: "spec.forProvider.region"
transforms:
- type: map
map:
EU: "eu-north-1"
US: "us-east-2"It's not just patches either - you can use P&T to derive XR connection details
from a resource produced by another Function too, or use it to determine whether
a resource produced by another Function is ready
Decouple P&T development from Crossplane core
When P&T development happens in a Function, it's not coupled to the Crossplane
release cycle. The Function developers could cut releases more frequently to add
new features to P&T.
Plus, because it's just a Function, it becomes easier to fork. You could fork
this Function, add a new kind of transform and try it out for a few weeks in
your development environment before sending a PR upstream. Or, if your new
feature is controversial, it's now a lot less work to maintain your own fork
long term.
Makes P&T code more portable
A lot of building a better developer experience around Composition comes down to
shifting left - letting you run and test your Compositions when you're
developing them. Historically this has been tough. You need to spin up a kind
cluster, install Crossplane, install providers, etc.
$ xp composition render xr.yaml composition.yamlYou could imagine a CLI tool like the above helping a lot. The problem with
building tools like this in the past has been that they need to share
Crossplane's Composition logic. We could make Composition a library, but then
you'd need to make sure that the version of xp on your laptop used the same
Composition library as your control planes, or you might see different results
than you expected in production.
When all Composition logic is encapsulated in Functions - i.e. versioned OCI
containers with a standard RPC - building a tool like this becomes much easier.
Just tell the CLI what Function versions you're using in production and it can
pull them down and use them to render your Composition.
Makes Crossplane's Composition implementation simpler
If we can make the native P&T implementation and Functions mutually exclusive,
Crossplane's Composition implementation is dramatically less complex. This means
it's easier to maintain and much less likely to be buggy.
Moving P&T inside a Function makes this possible - you can still use 'both' P&T
and Functions, you'd just do it by... using Functions.
Eventually, if enough P&T users switch to this Function we may be able to remove
native support for P&T altogether.
Differences from the native implementation
This Function has a few small, intentional breaking changes compared to the
native implementation. Making the below fields required makes P&T configuration
a lot more explicit and less ambiguous.
resources[i].nameis now a required field.resources[i].connectionDetails[i].nameis now a required fieldresources[i].connectionDetails[i].typeis now a required fieldresources[i].patches[i].policy.mergeOptionsis no longer supported
Functions use Kubernetes server-side apply, not mergeOptions, to intelligently
merge arrays and objects. This requires merge configuration to be specified at
the composed resource schema level (i.e. in CRDs) per #4617.
Known issues
The initial implementation has the following limitations:
EnvironmentConfigand its associated patches aren't supported yet. This is
just because Crossplane doesn't yet send theEnvironmentConfigalong with
theRunFunctionRequest. Once we do, these should be easy to (re)implement.
Adding support at the Functions level is tracked in #4632.
Developing
This Function doesn't use the typical Crossplane build submodule and Makefile,
since we'd like Functions to have a less heavyweight developer experience.
It mostly relies on regular old Go tools:
# Run code generation - see input/generate.go
$ go generate ./...
# Run tests
$ go test -cover ./...
? github.com/crossplane-contrib/function-patch-and-transform/input/v1beta1 [no test files]
ok github.com/crossplane-contrib/function-patch-and-transform 0.021s coverage: 76.1% of statements
# Lint the code
$ docker run --rm -v $(pwd):/app -v ~/.cache/golangci-lint/v1.54.2:/root/.cache -w /app golangci/golangci-lint:v1.54.2 golangci-lint run
# Build a Docker image - see Dockerfile
$ docker build .