Erasing Secrets - How to Remove Sensitive Files from Git History
⚠ Problem
Imagine you accidentally added a sensitive file to Git, committed it, and pushed the repository to GitHub or another platform. After making several more commits, you realize that the file is still being tracked by Git. While you can delete the file and commit the removal, the file remains accessible in the commit history, posing a security risk.
To fully remove the file from the entire Git history, additional steps are required.
✔ Solution
As an example we will remove the .env file from the git history
1. Add the File to .gitignore
First, make sure your .gitignore file contains .env, so it doesn't get tracked again in the future:
echo ".env" >> .gitignore
2. Remove the File from Git History
Use git filter-repo to remove the .env file from all commits
git filter-repo --path .env --invert-paths
git-filter-repo command doesn’t come with git by default, you need to install it manually. (Github)3. Force Push the Cleaned History
Since we’re modifying the commit history, we need to force push:
git push origin --force --all
4. Ensure .env is no Longer in the Repository
Run:
git ls-files | grep .env
or:
git log --stat | grep .env
5. Inform Collaborators
If others have cloned the repo, they need to run:
git fetch origin --prune git reset --hard origin/main
This forces them to sync with the new cleaned history.
6. (Optional) Remove the File from GitHub Cache
If the .env file was exposed on GitHub, you can also request GitHub to remove cached versions from their servers:
Possible Issues
git: 'filter-repo' is not a git command. See 'git --help'.
Aborting: Refusing to destructively overwrite repo history since this does not look like a fresh clone. (expected freshly packed repo) Please operate on a fresh clone instead. If you want to proceed anyway, use --force.
--force or work on a fresh clonegit filter-repo --path .env --invert-paths --force
Advanced
Understanding the Command
This command is used to permanently remove the .env file from your Git repository's history. Let's break it down step by step:
Command Breakdown
git filter-repo --path .env --invert-paths
git filter-repo
git filter-repois a modern and more efficient alternative togit filter-branch.- It's used for rewriting commit history, particularly for removing files, directories, or sensitive data from a repository.
--path .env
- This option targets the
.envfile specifically. - It tells
git filter-repothat we want to operate on.env(remove it from history).
--invert-paths
- Inverts the selection, meaning instead of keeping
.env, it removes it from all commits. - Without
--invert-paths,git filter-repo --path .envwould only keep.envand remove everything else (which is not what we want). - By using
--invert-paths, we are saying "remove.envfrom all commits but keep everything else".
What This Command Does
When you run:
git filter-repo --path .env --invert-paths
it:
- Goes through all commits in the repository.
- Finds any occurrence of
.envin the commit history. - Removes the
.envfile from those commits (like it was never added in the first place). - Keeps everything else intact.
