Month: February 2025

… and keeping them out of the code in GIT? 

Let’s say you have a larger config file with a pile of items that you want to fill in while deploying, but don’t want to keep in git, such as settings or credentials?

At the same time, you want to test things while developing, without having to set up credentials etc all the time, copying files in and out, or quickly configure things for deploying / testing against different targets..

This obviously does not take away from the use of live secret management,
such as aws secrets manager and others, but is suitable for more “static” solutions,
or fundamental configurations required for base setups.

Pre-requisites; 

• 1Password command line CLI agent. ( https://1password.com/downloads/command-line )
• jq (the json parser)

Steps: 

• Create a new vault in the 1Password, like “CICD”
• Create the item, and name it as you please, no spaces in the name, like.. “app-config”.
• In the item, say a secure note called ‘general-creds’ , create a field and name it “json”. 
• In this json field, store the json as an array of key/value sets, like this: 

  {
     "cred": [
          { "key_1": "value 1" }, 
          { "key_2": "value 2" }, 
          .... and so on. 
     }
  }

• and this shellscript; 

  #!/usr/bin/env bash
  ROOT="$(dirname "$(realpath "$0")")"
  
  # Check if the required arguments are provided
  if [ "$#" -ne 2 ]; then
     echo "Usage: $0 <1pass vault/item> <template_file>"
     exit 1
  fi
  
  # Read the JSON file and template file
  path="$1"
  template_file="$2"
  
  # Get the credentials from the 1Password
  op read "op://${path}/json" > 1p-credfile.json
  
  # Validate that the JSON file exists
  if [ ! -f "1p-credfile.json" ]; then
      echo "Error: JSON file '1p-credfile.json' does not exist."
      exit 1
  fi
  
  # Validate that the template file exists
  if [ ! -f "$template_file" ]; then
      echo "Error: Template file '$template_file' does not exist."
      exit 1
  fi
  
  # Read the JSON content and template content from the files
  json="$(cat "1p-credfile.json")"
  template="$(cat "$ROOT/$template_file")"
  
  # Use jq to traverse the JSON array and replace placeholders
  result=$(echo "$json" | jq -r --arg template "$template" '
     reduce .cred[] as $item ($template; 
     reduce ($item | to_entries[]) as $kv (.;
       gsub("\\[\\[" + $kv.key + "\\]\\]"; $kv.value))
     )
   ')
  
  # Output the replaced string
  echo "$result"
  
  # Clean up after ourselves. 
  if [ -e 1p-credfile.json ] ; then rm -f 1p-credfile.json ; fi

• .. and a config template file (or other file), like this, inserting the key names in double hard brackets, such as [[keyname]]

  {
     "configValue1": "[[key_1]]",
     "configValue2": "[[key_2]]"
  }

./MapCreds-onepass.sh CICD/app-config config-template.json > config.json

{
   "configValue1": "value 1",
   "configValue2": "value 2"
}

… just don’t forget to mark the resulting “config.json” as an exluded file in git!
(and obviously, this would work on other text files as well,  including source code for replacing values / settings having the source in 1pass.)

Njoy!