Issue with transparent cache.nixos.org proxy
Issue with transparent cache.nixos.org proxy
Hi! I would like to host a transparent proxy for cache.nixos.org on my local kubernetes cluster.
I took the following NGINX config https://nixos.wiki/wiki/FAQ/Private_Cache_Proxy and created all the folders on the mounted storage.
This is the kubernetes deployment:
yaml
apiVersion: v1 kind: PersistentVolume metadata: name: nix-cache-volume spec: capacity: storage: 500Gi storageClassName: manual accessModes: - ReadWriteOnce hostPath: path: "/mnt/k8s/nix-cache" # Needs exists before PV is created! persistentVolumeReclaimPolicy: Retain --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: nix-cache-pvc spec: accessModes: - ReadWriteOnce storageClassName: manual resources: requests: storage: 500Gi --- apiVersion: apps/v1 kind: Deployment metadata: name: nix-cache spec: replicas: 1 selector: matchLabels: app: nix-cache template: metadata: labels: app: nix-cache name: nix-cache spec: volumes: - name: nix-cache-storage persistentVolumeClaim: claimName: nix-cache-pvc - name: nix-cache-config configMap: name: nix-cache-config containers: - name: nix-cache image: nginx:1.27.0 ports: - containerPort: 80 volumeMounts: - name: nix-cache-storage mountPath: /data - name: nix-cache-config mountPath: /etc/nginx/sites-available/default resources: limits: memory: "512Mi" cpu: "300m" requests: memory: "256Mi" cpu: "200m" --- apiVersion: v1 kind: Service metadata: name: nix-cache spec: selector: app: nix-cache ports: - protocol: TCP port: 80 targetPort: 80 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: nix-cache-ingress annotations: traefik.ingress.kubernetes.io/router.tls: "true" spec: rules: - host: "nix-cache.raspi.home" http: paths: - pathType: Prefix path: "/" backend: service: name: nix-cache port: number: 80 tls: - secretName: nix-cache-raspi-home-tls hosts: - "nix-cache.raspi.home" --- apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: nix-cache.raspi.home spec: commonName: nix-cache.raspi.home dnsNames: - "nix-cache.raspi.home" secretName: nix-cache-raspi-home-tls issuerRef: name: ca-issuer kind: ClusterIssuer --- apiVersion: v1 kind: ConfigMap metadata: name: nix-cache-config data: nginx.conf: | server { listen 80; server_name nix-cache.raspi.home; location ~ ^/nix-cache-info { proxy_store on; proxy_store_access user:rw group:rw all:r; proxy_temp_path /data/nginx/nix-cache-info/temp; root /data/nginx/nix-cache-info/store; proxy_set_header Host "cache.nixos.org"; proxy_pass https://cache.nixos.org/; } location ~^/nar/.+$ { proxy_store on; proxy_store_access user:rw group:rw all:r; proxy_temp_path /data/nginx/nar/temp; root /data/nginx/nar/store; proxy_set_header Host "cache.nixos.org"; proxy_pass https://cache.nixos.org/; } }
To use the cache I added it to the substituters
.
nix
nix.settings.substituters = [ "https://nix-cache.raspi.home/" ];
But when I try to use it, get the error:
bash
# Trigger a download nix develop nixpkgs#just # Error message warning: 'https://nix-cache.raspi.home/' does not appear to be a binary cache
In the logs of the NGINX I see the following error:
2024/08/03 12:09:30 [error] 31#31: *3 open() "/usr/share/nginx/html/nix-cache-info" failed (2: No such file or directory), client: 10.42.2.7, server: localhost, request: "GET /nix-cache-info HTTP/1 │ │ 10.42.2.7 - - [03/Aug/2024:12:09:30 +0000] "GET /nix-cache-info HTTP/1.1" 404 153 "-" "curl/8.8.0 Nix/2.18.5" "10.42.2.1" │ │ 10.42.2.7 - - [03/Aug/2024:12:09:30 +0000] "PUT /nix-cache-info HTTP/1.1" 405 157 "-" "curl/8.8.0 Nix/2.18.5" "10.42.2.1"
Any ideas whats wrong? I'm neither an nix nor an nginx expert, so maybe it is something really simple but I cannot figure it out.