Setting Your Pipeline

• setpipeline script
• secrets .yml
• Pipeline Inception

setpipeline script

Create a script in your repo that will set your pipeline:

#!/bin/bash
fly -t MYTARGET sp -c pipeline.yml -p MY-PIPELINE-NAME -l ~/.ssh/secrets.yml

Don't forget to grant execution permissions:

setpipeline

$ chmod +x setpipeline

Now each time you want to set the pipeline, just run:

$ ./setpipeline

secrets .yml

Any secrets that you have that you do not want committed to your repo can be kept in an external .yml file (e.g. concourse.yml)

github-private-key: |
  -----BEGIN RSA PRIVATE KEY-----
  MIIJKgIBAAKCAgEArWVvzbHG++Tiyjjmhib5QkwmCD4Zw6wz7TxXsoeso5H6VxW+
  ...
  fXBH4JFKByP7DKuc9TBZgm6JRzVA4p60SFKyWA8MTJIec4RHbDhxCiMhGkcH4A==
  -----END RSA PRIVATE KEY-----
github-token: abcdefghijklmnopef8a10af6a8a468284423
github-username: svc_str_ops_sft_ci
github-access-token: abcdefghijklmnop4a920a00a4b12a8484ad699718a90
cloud-foundry-service-account-user: svc_str_ops_sft_ci
cloud-foundry-service-account-password: abcdefghijklmnop

resources:
- name: resource-git
  type: git
  source:
    branch: master
    private_key: {{github-private-key}}

In the pipeline.yml file, the secrets can be referenced with surrounding curly braces {{name}}

Pipeline Inception

https://github.homedepot.com/Back-Office-And-In-Aisle-Systems/pipeline-inception

• Pipeline that sets other pipelines based on json configuration
• Great for:
  • Handling secrets (not everyone needs secrets)
  • Setting pipeline by running concourse job instead of fly command
  • Can automatically stand up all pipelines with one fly set-pipeline command (helpful for server-on-demand expiration)

[
    {
        "server": "http://concourse-core-tech-mobile.apps-np.homedepot.com",
        "team": "main",
        "pipelineName": "kiosk-shell",
        "pipelineRepo": "github.homedepot.com/store-app-platform/kiosk-shell-pipeline.git"
    },
    {
        "server": "http://concourse-core-tech-mobile.apps-np.homedepot.com",
        "team": "main",
        "pipelineName": "cash-manager",
        "pipelineRepo": "github.homedepot.com/store-app-platform/cash-manager-pipeline.git"
    },
    {
        "server": "http://concourse-core-tech-mobile.apps-np.homedepot.com",
        "team": "main",
        "pipelineName": "shellconfiguration",
        "pipelineRepo": "github.homedepot.com/Back-Office-And-In-Aisle-Systems/shellconfiguration.git"
    },
    {
        "server": "http://concourse-core-tech-mobile.apps-np.homedepot.com",
        "team": "main",
        "pipelineName": "shellconfiguration-router",
        "pipelineRepo": "github.homedepot.com/Back-Office-And-In-Aisle-Systems/shellconfiguration-router.git"
    },
    {
        "server": "http://concourse-core-tech-mobile.apps-np.homedepot.com",
        "team": "main",
        "pipelineName": "health-alert-mobile-shell",
        "pipelineRepo": "github.homedepot.com/Back-Office-And-In-Aisle-Systems/health-alert-mobile-shell.git"
    }
]

Pulling Github Resources

• Webhooks
• Polling

Webhooks

Idea: instead of polling github repo for changes, let's just notify of changes when there is a commit

Don't be on the shame-bot list

Easy to Setup:

• Make sure concourse is updated
• Configure pipeline
• Add webhook

Walkthrough:

https://github.homedepot.com/bxr2249/concourse-webhooks

Polling

Make sure every pipeline resource has a `check_every` value, otherwise it defaults to 1 minute

Again, don't be on the shame-bot list

resources:
- name: resource-git
  type: git
  source:
    branch: master
    private_key: {{github-private-key}}
    uri: git@github.homedepot.com:bxr2249/repo.git
    git_config:
    - name: http.sslVerify
      value: false
    disable_ci_skip: true
  check_every: 1h

Best Practices:

• ALWAYS SET check_every
• For external resources that update infrequently, set it to 24h (e.g. sos-secrets)