Few changes for default helm chart on github:

  1. Few new features only available in new components images(e.g authorization), so update your images.

  2. New Fron50 requires adding Credentials into /home/spinnaker/.aws, so make sure you mount it correctly:

      "volumeMounts": [
        {
          "name": "spinnaker-spinnaker-spinnaker-config",
          "mountPath": "/opt/spinnaker/config"
        },
        {
          "name": "spinnaker-spinnaker-s3-config",
          "mountPath": "/root/.aws"
        },
        {
          "name": "spinnaker-spinnaker-s3-config",
          "mountPath": "/home/spinnaker/.aws"
        }
      ],
    
  3. To enable authentication and authorization, you need to configure Gate and Fiat. Default chart has no Fiat, and it has wrong fiat.yml-local(should be fiat-local.yml) in the configmap.

      apiVersion: extensions/v1beta1
      kind: Deployment
      metadata:
        labels:
          app: spinnaker-fiat
          component: fiat
        name: spinnaker-spinnaker-fiat
        namespace: devops
      spec:
        replicas: 1
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            app: spinnaker-fiat
            component: fiat
        strategy:
          rollingUpdate:
            maxSurge: 25%
            maxUnavailable: 25%
          type: RollingUpdate
        template:
          metadata:
            labels:
              app: spinnaker-fiat
              component: fiat
          spec:
            containers:
            - image: gcr.io/spinnaker-marketplace/fiat:1.0.0-20180626022808
              imagePullPolicy: IfNotPresent
              name: fiat
              ports:
              - containerPort: 7003
                protocol: TCP
              readinessProbe:
                httpGet:
                  path: /health
                  port: 7003
                initialDelaySeconds: 30
                timeoutSeconds: 2
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
              volumeMounts:
              - mountPath: /opt/spinnaker/config
                name: spinnaker-spinnaker-spinnaker-config
            volumes:
            - name: spinnaker-spinnaker-spinnaker-config
              configMap:
                defaultMode: 420
                name: spinnaker-spinnaker-spinnaker-config
      ----
      apiVersion: v1
      kind: Service
      metadata:
        labels:
          app: spinnaker-fiat
          component: fiat
        name: spinnaker-spinnaker-fiat
        namespace: devops
      spec:
        ports:
        - port: 7003
          protocol: TCP
          targetPort: 7003
        selector:
          app: spinnaker-fiat
          component: fiat
        sessionAffinity: None
        type: ClusterIP
    

Change Deck settings.js along with related configmaps. 4. The way that Spinnaker uses to Authentcate requires redirect traffic, which leads to troublesome if you want to use ingress for both deck and gate. So I just give it up and use loadbalancer instead. 5. After enable Fiat, the trigger feature requires a service account to work. Offical Link

  • Create fiat serviceaccount on front50:
    curl -X POST \
      -H "Content-type: application/json" \
      -d '{ "name": "[email protected]", "memberOf": ["devops"] }' \
      http://spinnaker-spinnaker-front50:8080/serviceAccounts
    
  • Check accoutn existence on front50: curl http://spinnaker-spinnaker-front50:8080/serviceAccounts.
  • check account detail on fiat: curl http://spinnaker-spinnaker-fiat:7003/authorize/myApp-svc-account.
  1. Change deck in Spinnaker-local.yml to make slack send correct URL notes:
deck:
    #host: spinnaker-spinnaker-deck
    host: spinnaker.tdlab.ca
  1. Spinnaker uses ELB, Eureka or Consul to determine weather the instance is ‘up’, if using Openstack as infra, there’s no way for it to determine if the servergroup(Heat instances) it creates is up or not. So make sure that the Consider Only Cloud Provider Health When Executing Tasks under config is checked, this will force spinnaker to ignore checking status.
  2. To use Canary, we need to manually install Kayenta:
      apiVersion: extensions/v1beta1
      kind: Deployment
      metadata:
        labels:
          app: spinnaker-kayenta
          component: kayenta
        name: spinnaker-spinnaker-kayenta
        namespace: devops
      spec:
        progressDeadlineSeconds: 600
        replicas: 1
        revisionHistoryLimit: 10
        selector:
          matchLabels:
            app: spinnaker-kayenta
            component: kayenta
        strategy:
          rollingUpdate:
            maxSurge: 25%
            maxUnavailable: 25%
          type: RollingUpdate
        template:
          metadata:
            labels:
              app: spinnaker-kayenta
              component: kayenta
          spec:
            containers:
            - image: gcr.io/spinnaker-marketplace/kayenta:0.3.0-20180703115601
              imagePullPolicy: IfNotPresent
              name: kayenta
              ports:
              - containerPort: 8090
                protocol: TCP
              readinessProbe:
                failureThreshold: 3
                httpGet:
                  path: /health
                  port: 8090
                  scheme: HTTP
                initialDelaySeconds: 30
                periodSeconds: 10
                successThreshold: 1
                timeoutSeconds: 2
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
              volumeMounts:
              - mountPath: /opt/spinnaker/config
                name: spinnaker-spinnaker-spinnaker-config
              - mountPath: /root/.aws
                name: spinnaker-spinnaker-s3-config
              - mountPath: /home/spinnaker/.aws
                name: spinnaker-spinnaker-s3-config
            dnsPolicy: ClusterFirst
            restartPolicy: Always
            schedulerName: default-scheduler
            terminationGracePeriodSeconds: 30
            volumes:
            - configMap:
                defaultMode: 420
                name: spinnaker-spinnaker-spinnaker-config
              name: spinnaker-spinnaker-spinnaker-config
            - configMap:
                defaultMode: 420
                name: spinnaker-spinnaker-s3-config
              name: spinnaker-spinnaker-s3-config
      ---
      apiVersion: v1
      kind: Service
      metadata:
        labels:
          app: spinnaker-kayenta
          component: kayenta
        name: spinnaker-spinnaker-kayenta
        namespace: devops
      spec:
        ports:
        - port: 8090
          protocol: TCP
          targetPort: 8090
        selector:
          app: spinnaker-kayenta
          component: kayenta
        sessionAffinity: None
        type: ClusterIP
    

Note that we mount credential file at ``/home/spinnaker/.aws, this is required for new Kayenta's images. Then create kayenta's config part: ``` redis: connection: ${services.redis.connection:redis://localhost:6379} server: ssl: enabled: false port: '8090' address: 0.0.0.0 security: basic: enabled: true kayenta: prometheus: enabled: true accounts: - name: prometheus endpoint: baseUrl: http://prometheus-server.logging:80 supportedTypes: - METRICS_STORE aws: enabled: true accounts: - name: minio-s3 bucket: spinnaker rootFolder: kayenta endpoint: http://spinnaker-minio:9000 accessKeyId: xxx secretAccessKey: xxx supportedTypes: - CONFIGURATION_STORE - OBJECT_STORE s3: enabled: true ``` And also need to add Kayenta config in spinnaker.yml` to make entire spinnaker aware the existence of Kayenta.

Redis Issue

Sometimes Redis may stuck at Bad file format reading the append only file: make a backup of your AOF file, then use ./redis-check-aof --fix <filename> and goes dead instantly, unfortunately you can’t run this command by appending onto a dead container or run commands based on current container, because data and config are not mounted. Luckily usually these data are mounted from shared drive like nfs, so we can mount them elsewhere and do some changes from there. So go find that Redis folder, in my case is /var/lib/kubelet/pods/1def3102-d3ac-11e8-ad66-fa163ed25b38/volumes/kubernetes.io~nfs/nfs-client-root/spinnaker-spinnaker-redis-pvc-48d0acd3-94cd-11e8-8502-fa163ed25b38/redis/data and run redis-check-aof --fix ./appendonly.aof, it will then start truncating AOF. Account sync may be needed, try curl -X POST http://spinnaker-spinnaker-fiat:7003/roles/sync then get curl http://spinnaker-spinnaker-fiat:7003/authorize/[email protected] to see if account gets created.