MetalLB - admission webhooks - denied the request: resource must be created in operators namespace. What does it mean? - k3s

Question

I am trying to create a self-hosted k3s cluster to explore the world of k8s.
I have used kubectl apply to apply the manifest at (not at the same time):
https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml
https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-frr.yaml
https://raw.githubusercontent.com/metallb/metallb/v0.13.6/config/manifests/metallb-frr.yaml
I am trying to add ipAddressPool, BGPPeer and BGPAdvertisement to MetalLB and am facing an issue where the relevant webhooks:
ipaddresspoolvalidationwebhook.metallb.io
bgppeersvalidationwebhook.metallb.io
bgpadvertisementvalidationwebhook.metallb.io
Are returning with: admission webhook denied the request: resource must be created in operators namespace
I don't know what this response means or how to resolve it.
Manifests:
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
namespace: metallb-system
name: bgp-pool
spec:
addresses:
- 172.31.20.1/24
- 2001:db8::c00b:beef::/80
apiVersion: metallb.io/v1beta2
kind: BGPPeer
metadata:
name: router
namespace: metallb-system
spec:
myASN: 64521
peerASN: 64520
peerAddress: 172.30.0.1
bfdProfile: bfdprofile
apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
name: bgp-pool
namespace: metallb-system
spec:
ipAddressPools:
- bgp-pool
Pod Status:
$ kpod
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-node-f4dsm 1/1 Running 0 68m
kube-system local-path-provisioner-79f67d76f8-scwdt 1/1 Running 0 92m
kube-system coredns-597584b69b-ljxd9 1/1 Running 0 92m
kube-system calico-kube-controllers-798cc86c47-9h6mz 1/1 Running 0 68m
kube-system calico-node-dhtt7 1/1 Running 0 68m
kube-system calico-node-dbm2f 1/1 Running 0 68m
olm olm-operator-56cf65dbf9-t4r9k 1/1 Running 0 54m
olm catalog-operator-6b8c45596c-m6fxf 1/1 Running 0 54m
olm packageserver-6d7b8cd74-dqgdl 1/1 Running 0 40m
olm packageserver-6d7b8cd74-ncphw 1/1 Running 0 40m
olm operatorhubio-catalog-np4f7 1/1 Running 0 40m
olm 00ae99ed4d0c9f0380e2866691b8643f32e9c345efa7942e3572d090556hpc8 0/1 Completed 0 37m
operators metallb-operator-controller-manager-f59767f58-mtd48 1/1 Running 0 36m
operators metallb-operator-webhook-server-7f79999bb7-v94hm 1/1 Running 0 36m
metallb-system controller-66f6c8999f-kvxkp 1/1 Running 0 21m
metallb-system speaker-bnxxd 4/4 Running 0 21m
metallb-system speaker-dqcn9 4/4 Running 0 21m
Logs
metallb-operator-webhook-server
$ k logs metallb-operator-webhook-server-7f79999bb7-v94hm -n operators
{"branch":"dev","caller":"main.go:155","commit":"dev","goversion":"gc / go1.18.3 / amd64","level":"info","msg":"MetalLB controller starting (commit dev, branch dev)","ts":"2022-12-29T20:40:23Z","version":""}
{"caller":"k8s.go:389","level":"info","msg":"Starting Manager","op":"Run","ts":"2022-12-29T20:40:24Z"}
{"action":"webhooks enabled","caller":"webhook.go:55","level":"info","op":"startup","ts":"2022-12-29T20:40:24Z"}
{"level":"info","ts":1672346424.0781536,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called","GVK":"metallb.io/v1beta1, Kind=AddressPool"}
{"level":"info","ts":1672346424.0788455,"logger":"controller-runtime.builder","msg":"Registering a validating webhook","GVK":"metallb.io/v1beta1, Kind=AddressPool","path":"/validate-metallb-io-v1beta1-addresspool"}
{"level":"info","ts":1672346424.0794268,"logger":"controller-runtime.webhook.webhooks","msg":"Starting webhook server"}
{"level":"info","ts":1672346424.0799024,"logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-metallb-io-v1beta1-addresspool"}
{"level":"info","ts":1672346424.080218,"logger":"controller-runtime.certwatcher","msg":"Updated current TLS certificate"}
{"level":"info","ts":1672346424.080664,"logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/convert"}
{"level":"info","ts":1672346424.0814717,"logger":"controller-runtime.builder","msg":"Conversion webhook enabled","GVK":"metallb.io/v1beta1, Kind=AddressPool"}
{"level":"info","ts":1672346424.081652,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called","GVK":"metallb.io/v1beta1, Kind=IPAddressPool"}
{"level":"info","ts":1672346424.081719,"logger":"controller-runtime.builder","msg":"Registering a validating webhook","GVK":"metallb.io/v1beta1, Kind=IPAddressPool","path":"/validate-metallb-io-v1beta1-ipaddresspool"}
{"level":"info","ts":1672346424.0819142,"logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-metallb-io-v1beta1-ipaddresspool"}
{"level":"info","ts":1672346424.0821607,"logger":"controller-runtime.certwatcher","msg":"Starting certificate watcher"}
{"level":"info","ts":1672346424.0823011,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called","GVK":"metallb.io/v1beta2, Kind=BGPPeer"}
{"level":"info","ts":1672346424.083003,"logger":"controller-runtime.webhook","msg":"Serving webhook server","host":"","port":9443}
{"level":"info","ts":1672346424.0831902,"logger":"controller-runtime.builder","msg":"Registering a validating webhook","GVK":"metallb.io/v1beta2, Kind=BGPPeer","path":"/validate-metallb-io-v1beta2-bgppeer"}
{"level":"info","ts":1672346424.0839014,"logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-metallb-io-v1beta2-bgppeer"}
{"level":"info","ts":1672346424.084423,"logger":"controller-runtime.builder","msg":"Conversion webhook enabled","GVK":"metallb.io/v1beta2, Kind=BGPPeer"}
{"level":"info","ts":1672346424.0847147,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called","GVK":"metallb.io/v1beta1, Kind=BGPAdvertisement"}
{"level":"info","ts":1672346424.0849228,"logger":"controller-runtime.builder","msg":"Registering a validating webhook","GVK":"metallb.io/v1beta1, Kind=BGPAdvertisement","path":"/validate-metallb-io-v1beta1-bgpadvertisement"}
{"level":"info","ts":1672346424.0854847,"logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-metallb-io-v1beta1-bgpadvertisement"}
{"level":"info","ts":1672346424.0860505,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called","GVK":"metallb.io/v1beta1, Kind=L2Advertisement"}
{"level":"info","ts":1672346424.0861676,"logger":"controller-runtime.builder","msg":"Registering a validating webhook","GVK":"metallb.io/v1beta1, Kind=L2Advertisement","path":"/validate-metallb-io-v1beta1-l2advertisement"}
{"level":"info","ts":1672346424.0863633,"logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-metallb-io-v1beta1-l2advertisement"}
{"level":"info","ts":1672346424.086669,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called","GVK":"metallb.io/v1beta1, Kind=Community"}
{"level":"info","ts":1672346424.087227,"logger":"controller-runtime.builder","msg":"Registering a validating webhook","GVK":"metallb.io/v1beta1, Kind=Community","path":"/validate-metallb-io-v1beta1-community"}
{"level":"info","ts":1672346424.0876715,"logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-metallb-io-v1beta1-community"}
{"level":"info","ts":1672346424.087925,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, object does not implement admission.Defaulter or WithDefaulter wasn't called","GVK":"metallb.io/v1beta1, Kind=BFDProfile"}
{"level":"info","ts":1672346424.0879838,"logger":"controller-runtime.builder","msg":"Registering a validating webhook","GVK":"metallb.io/v1beta1, Kind=BFDProfile","path":"/validate-metallb-io-v1beta1-bfdprofile"}
{"level":"info","ts":1672346424.0881443,"logger":"controller-runtime.webhook","msg":"Registering webhook","path":"/validate-metallb-io-v1beta1-bfdprofile"}
metallb-operator-controller-manager
$ k logs metallb-operator-controller-manager-f59767f58-mtd48 -n operators
1.6723464002770052e+09 INFO setup git commit: {"id": "=dev"}
1.6723464006334805e+09 INFO controller-runtime.metrics Metrics server is starting to listen {"addr": ":0"}
1.6723464006350648e+09 INFO platform detecting platform version...
1.6723464006471622e+09 INFO platform PlatformInfo [Name: Kubernetes, K8SVersion: 1.25, OS: linux/amd64]
1.6723464006474972e+09 INFO setup starting manager
1.6723464006485376e+09 INFO Starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:33419"}
I1229 20:40:00.650000 1 leaderelection.go:248] attempting to acquire leader lease operators/metallb.io.metallboperator...
I1229 20:40:00.679813 1 leaderelection.go:258] successfully acquired lease operators/metallb.io.metallboperator
1.672346400680695e+09 INFO Starting EventSource {"controller": "metallb", "controllerGroup": "metallb.io", "controllerKind": "MetalLB", "source": "kind source: *v1beta1.MetalLB"}
1.6723464006810224e+09 INFO Starting Controller {"controller": "metallb", "controllerGroup": "metallb.io", "controllerKind": "MetalLB"}
1.6723464006801696e+09 DEBUG events Normal {"object": {"kind":"Lease","namespace":"operators","name":"metallb.io.metallboperator","uid":"4aed740a-9ffd-4207-940f-941905ac353e","apiVersion":"coordination.k8s.io/v1","resourceVersion":"2812"}, "reason": "LeaderElection", "message": "metallb-operator-controller-manager-f59767f58-mtd48_bb8c2886-579c-4666-ade5-95d9f366f330 became leader"}
1.6723464007836418e+09 INFO Starting workers {"controller": "metallb", "controllerGroup": "metallb.io", "controllerKind": "MetalLB", "worker count": 1}
What does work
The BFDProfile does get accepted by the validation hooks. The ipAddressPool, BGPPeer and BGPAdvertisement manifests have been built using the documentation at https://metallb.universe.tf/configuration/.
I have tried removing the metallb-operator and the whole metallb manifest and even destroying and recreating the cluster and cannot find any reference to this error after hours searching.
I expect the admission webhooks to accept the manifests as they have been built using the official documentation.