About This Article
This article organizes its content by leveraging generative AI while verifying official and primary sources.Verification Status: 🧪 Syntax and Logic Verified
Syntax and conversion tests using dummy sensitive information were performed in a Linux / GNU Bash environment. Please verify it in your own environment before applying it to actual configuration files.
By incorporating a step to mechanically mask sensitive information before handing configuration files over to AI, you can reduce the number of forgotten redactions compared to relying solely on manual work. However, automatic masking is not a mechanism that eliminates the need for a final check. It should be used under the premise of combining automated processing with human visual inspection. Here, we will create a Bash script that creates a separate masked file without modifying the original file.
What We Want to Protect with This Script
The targets are values that commonly appear in configuration files, such as the following:
password / passwd
token
api_key / api-key
secret / client_secret
access_key
user:password@ within URLs
PEM-formatted PRIVATE KEY bodies
On the other hand, we do not assume that it can completely identify unique key names or complex JSON/YAML structures.
flowchart LR
A[元の設定ファイル] -->|読み取り| B[mask-secrets.sh]
B --> C{秘密情報らしいか}
C -->|Yes| D[値を **** に置換]
C -->|No| E[そのまま出力]
D --> F[.masked ファイル]
E --> F
F --> G[機械チェック]
G --> H[人間が目視]
H --> I[AIへ渡す]
Do Not Overwrite the Original File
To err on the side of safety, this sample does not rewrite the input file.
app.conf ↓ Read mask-secrets.sh ↓ app.conf.masked
This is to ensure that even if there is a bug in the masking process, the original configuration file is not corrupted. Furthermore, if a .masked file with the same name already exists, it stops without overwriting it.
Bash Script
Save it as mask-secrets.sh.
#!/usr/bin/env bash
set -euo pipefail
if [[ $# -ne 1 ]]; then
echo "Usage: $0 <config-file>" >&2
exit 2
fi
input=$1
if [[ ! -f "$input" ]]; then
echo "ERROR: file not found: $input" >&2
exit 2
fi
output="${input}.masked"
if [[ -e "$output" ]]; then
echo "ERROR: output already exists: $output" >&2
exit 3
fi
umask 077
tmp=$(mktemp "${output}.tmp.XXXXXX")
trap 'rm -f "$tmp"' EXIT
awk '
BEGIN { in_private_key = 0 }
{
line = $0
if (line ~ /-----BEGIN ([A-Z0-9 ]+ )?PRIVATE KEY-----/) {
print line
print "****"
in_private_key = 1
next
}
if (in_private_key) {
if (line ~ /-----END ([A-Z0-9 ]+ )?PRIVATE KEY-----/) {
print line
in_private_key = 0
}
next
}
gsub(/://[^/:@[:space:]]+:[^@/[:space:]]+@/, "://****:****@", line)
lower = tolower(line)
if (lower ~ /^[[:space:]]*(export[[:space:]]+)?[a-z0-9_.-]*(password|passwd|token|api[_-]?key|secret|client[_-]?secret|access[_-]?key)[a-z0-9_.-]*[[:space:]]*[:=]/) {
eq = index(line, "=")
co = index(line, ":")
if (eq == 0) sep = co
else if (co == 0) sep = eq
else if (eq < co) sep = eq
else sep = co
print substr(line, 1, sep) " ****"
next
}
print line
}
' "$input" > "$tmp"
chmod 600 "$tmp"
mv "$tmp" "$output"
trap - EXIT
printf 'Created: %sn' "$output"
We set umask 077 before creating a temporary file, and finally apply chmod 600. The purpose is to handle files during the masking process in a state that makes them difficult for other users to read.
Verify with Dummy Data
Do not use real passwords or API keys; verify using values dedicated for testing.
cat > demo.conf <<'EOF' host=localhost password = dummy-password API_KEY: dummy-api-key export ACCESS_TOKEN=dummy-token url=https://alice:dummy-pass@example.com/api note=keep-this-line -----BEGIN PRIVATE KEY----- DUMMY-PRIVATE-KEY-BODY -----END PRIVATE KEY----- EOF chmod +x mask-secrets.sh bash -n mask-secrets.sh ./mask-secrets.sh demo.conf cat demo.conf.masked
This is the expected output.
host=localhost password = **** API_KEY: **** export ACCESS_TOKEN= **** url=https://****:****@example.com/api note=keep-this-line -----BEGIN PRIVATE KEY----- **** -----END PRIVATE KEY-----
You can also check whether any known dummy values remain.
grep -En 'dummy-password|dummy-api-key|dummy-token|dummy-pass|DUMMY-PRIVATE-KEY-BODY' demo.conf.masked
If nothing is displayed, the dummy secret strings prepared this time do not remain. However, this does not prove that unknown sensitive information does not exist.
Why Automatic Masking Alone Is Not Enough
Names of sensitive information vary from system to system. For example, credentials, bearers, private_tokens, or custom variable names might be missed by the regular expressions in this sample.
flowchart TB
A[自動マスク] --> B[既知の秘密文字列をgrep]
B --> C[差分と本文を目視]
C --> D{怪しい値が残る?}
D -->|Yes| E[ルールを追加して再実行]
E --> A
D -->|No| F[必要な範囲だけAIへ渡す]
In formats where structure matters, such as JSON or YAML, using dedicated parsers like jq or yq rather than simple line processing may be safer.
Reducing the Amount of Information Passed to AI Itself
Just as important as masking is “passing only the necessary parts.” If you do not need to send the entire configuration file, extract only the sections required to reproduce the issue.
Security should be considered through a multi-layered structure like the following:
flowchart LR
A[必要範囲だけ抽出] --> B[秘密情報をマスク]
B --> C[機械チェック]
C --> D[人間が確認]
D --> E[AIへ送信]
Conclusion
Create a masked copy without modifying the original file
Restrict permissions of temporary files right from creation
Do not assume automated detection is 100% effective; retain known-value checks and visual inspection
Consider dedicated parsers for complex structured data
Keep the amount of information passed to AI to the absolute minimum
Official and Primary Sources
GNU Bash Reference Manual
https://www.gnu.org/software/bash/manual/bash.html
GNU Awk User’s Guide
https://www.gnu.org/software/gawk/manual/gawk.html
OWASP Secrets Management Cheat Sheet
https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
コメント