Foundations & Prerequisites-2026-09-02

Git and GitHub for Bioinformatics

NM

Nasir Mahmood Abbasi, PhD

Bioinformatics Educator

Git and GitHub for Bioinformatics

Build a small local repository, record a meaningful change, and inspect what Git has captured before deciding whether it is safe to share. The examples use a deliberately data-free project; they do not process sequencing reads or establish that an analysis is scientifically valid.

Format: Hands-on lab. Beginners with a terminal, Git, and R will build and inspect a local teaching repository, then run a tiny synthetic TSV parser check. The commands create the inputs; a GitHub account and remote are optional.

Before you start

You need a terminal, Git, and R for the main script and provenance sections. Python 3 is only needed if you choose the alternative parser. Check the main tools first:

git --version
Rscript --version

If R is missing, follow the official CRAN R installation files for Linux, macOS, and Windows for your operating system, then open a new terminal and repeat the checks. The main parser uses base R, with no third-party R packages. The optional Python parser uses only the Python standard library.

A GitHub account is optional until you choose to create a remote repository. Work first in a project that contains no patient-level information, credentials, or restricted data. Git records text changes well, but ordinary Git history is a poor home for large sequencing files.

Start a local repository

Use a dedicated practice path. The preflight below stops rather than reusing an existing directory, so it will not silently overwrite prior work. If it already exists, choose another unused name before continuing:

mkdir -p ~/projects
if [ -e ~/projects/omics-demo-git-tutorial ]; then
  echo "STOP: ~/projects/omics-demo-git-tutorial already exists; choose a new unused path." >&2
else
  mkdir ~/projects/omics-demo-git-tutorial && cd ~/projects/omics-demo-git-tutorial && git init
fi

Continue only if git init ran in the new directory. The README and .gitignore heredocs below replace same-named files; because this directory was freshly created, they cannot overwrite files from an earlier run. If you choose a different path, verify it is new and adjust the later cd accordingly.

Git creates a hidden .git/ directory for repository history and settings. At this point it has no tracked project files. Run git status to see the current branch and working-tree state; the exact initial-branch wording depends on your Git configuration.

Describe the project and exclude common local files

A README should tell a collaborator what the repository contains, what inputs it expects, and how to run the analysis. This example documents the practice project without pretending it contains biological results.

cat > README.md << 'EOF'
# Omics demo

A small teaching repository for Git workflows. It contains scripts and configuration,
not raw sequencing data. Add the analysis purpose, required software, input format,
and run instructions before sharing a real project.
EOF

cat > .gitignore << 'EOF'
# Local data and common sequencing files; review project-specific needs
data/raw/
*.fastq.gz
*.bam
*.bai

# Temporary and environment files
*.pyc
__pycache__/
.ipynb_checkpoints/
.Rhistory
.RData
.Rproj.user/
.env
.env.*
.venv/
.conda/
envs/*/
!envs/*.yml
!envs/*.yaml
!envs/requirements.txt
*.key
EOF

Check the ignore rule before adding files: git check-ignore -v data/raw/example.fastq.gz prints the matching rule and its source if the path is ignored. Notice that solved environment prefix directories (such as .venv/, .conda/, or subprefixes under envs/*/) stay ignored, while declarative manifest files (envs/*.yml, envs/requirements.txt, renv.lock) remain unignored so collaborators can recreate the software environment. Patterns are project-specific: a broad pattern such as *.bam may hide a small derived file you meant to version, so inspect ignored files rather than assuming all matches are disposable.

Make a focused first commit

On a brand-new machine where Git has never committed before, configure your author identity once with git config --global user.name "Your Name" and git config --global user.email "you@example.org" so git commit does not abort with an unknown-author error. Stage only the two files you intend to record and review the staged diff before committing:

git add README.md .gitignore
git status
git diff --cached
git commit -m "Add project README and ignore rules"
git log --oneline -1

git add selects content for the next snapshot; git diff --cached lets you inspect that selection. A commit stores the selected content and a message. The short identifier from git log points to this repository snapshot, not to a result's scientific validity.

Isolate a change on a branch

A branch gives you a separate line of development for a small change. This local example uses main as the stable branch and merges the documentation update back after reviewing it:

git branch -M main
git switch -c docs/input-notes
cat >> README.md << 'EOF'

Input files must follow the format documented with each analysis script.
EOF
git diff
git add README.md
git diff --cached
git commit -m "Clarify input documentation"
git switch main
git merge docs/input-notes
git log --oneline --decorate -3

After the merge, the change is part of main and remains visible in history. If Git reports a conflict, stop and inspect the marked file before continuing; do not discard edits you need. Branching helps organize changes, while review and testing still determine whether they are ready to use.

Add a small script with an explicit input contract

This example reads a tab-separated file with headers named sample and reads. It is a teaching script for checking repository structure; its printed values are whatever the supplied file contains, not an assessment of sequencing quality.

R: main TSV parser

Save the following as scripts/qc_summary.R after creating scripts/ and results/ with mkdir -p scripts results. It reads values as text so identifiers and printed counts are not silently reformatted.

args <- commandArgs(trailingOnly = TRUE)
if (length(args) != 1L) stop("Usage: Rscript scripts/qc_summary.R samples.tsv", call. = FALSE)
frame <- read.delim(args[[1]], colClasses = "character", check.names = FALSE,
                    stringsAsFactors = FALSE, quote = "", comment.char = "")
if (!all(c("sample", "reads") %in% names(frame))) {
  stop("Input must have 'sample' and 'reads' header columns", call. = FALSE)
}
if (nrow(frame)) cat(paste(frame$sample, frame$reads), sep = "\n")
git add scripts/qc_summary.R
git diff --cached
git commit -m "Add R TSV summary example"
printf 'sample\treads\nsynthetic_A\t1200\n' > samples.tsv
Rscript --vanilla scripts/qc_summary.R samples.tsv
# Expected: synthetic_A 1200
# Deliberate negative test, only on this disposable teaching fixture:
printf 'name\treads\nsynthetic_A\t1200\n' > samples.tsv
Rscript --vanilla scripts/qc_summary.R samples.tsv
# Expected nonzero exit and the missing-header message.
# Restore the valid fixture before continuing.
printf 'sample\treads\nsynthetic_A\t1200\n' > samples.tsv

The successful run should print synthetic_A 1200. The negative test must report Input must have 'sample' and 'reads' header columns and exit nonzero. These checks test a parser contract, not sequencing quality.

Python alternative: the same TSV parser

Use this route only if you choose Python 3. Run python3 --version first; no third-party packages are needed. Its input and expected successful output are the same as the R route.

mkdir -p scripts results
cat > scripts/qc_summary.py << 'EOF'
#!/usr/bin/env python3
"""Print sample names and read counts from a two-column TSV."""
import csv
import sys

if len(sys.argv) != 2:
    raise SystemExit("Usage: qc_summary.py samples.tsv")

with open(sys.argv[1], newline="", encoding="utf-8") as handle:
    reader = csv.DictReader(handle, delimiter="\t")
    required = {"sample", "reads"}
    if reader.fieldnames is None or not required.issubset(reader.fieldnames):
        raise SystemExit("Input must have 'sample' and 'reads' header columns")
    for row in reader:
        print(row["sample"], row["reads"])
EOF

chmod +x scripts/qc_summary.py
git add scripts/qc_summary.py
git diff --cached
git commit -m "Add TSV summary example"

Create a one-row synthetic fixture, run the script from the repository root, and compare the exact output. This tests the parser's input contract only, not sequencing quality or biological validity:

cat > samples.tsv << 'EOF'
sample	reads
synthetic_A	1200
EOF
python3 scripts/qc_summary.py samples.tsv

Expected output:

synthetic_A 1200

Now deliberately test the missing-header failure. This replaces only the teaching fixture samples.tsv in the fresh practice repository; restore the valid fixture above before continuing:

cat > samples.tsv << 'EOF'
name	reads
synthetic_A	1200
EOF
python3 scripts/qc_summary.py samples.tsv

Expected error on standard error: Input must have 'sample' and 'reads' header columns (the command exits non-zero). The empty results/ directory is not tracked by Git unless it contains a file.

Record provenance without overstating reproducibility

Capture the code snapshot used for an analysis after committing the analysis code. The first command writes the current short commit ID; subsequent commands record selected environment details. Because adding these records creates another commit, the saved ID names the code state immediately before the provenance files were committed.

git rev-parse --short HEAD > COMMIT_HASH.txt
Rscript --version > ENV_INFO.txt 2>&1
Rscript --vanilla -e 'sessionInfo()' >> ENV_INFO.txt 2>&1
git add COMMIT_HASH.txt ENV_INFO.txt
git commit -m "Record analysis code revision and R session"

sessionInfo() records the R session and loaded packages, not every dependency, reference genome, annotation release, or external database. Notice also that committing COMMIT_HASH.txt creates a new commit whose own hash differs from the parent hash stored inside COMMIT_HASH.txt; in real analysis pipelines, avoid committing a self-referential hash file and instead write git describe --always --dirty directly into each run's output provenance log (for example results/run_metadata.txt) when the script executes. Use a reviewed project lockfile such as renv.lock for R package versions and record external inputs separately.

Choose how to handle larger files

Keep raw reads and restricted inputs in approved institutional storage. A .gitignore rule only affects untracked paths and does not provide storage, access control, or backup.

  • Git LFS: suitable when collaborators need large files versioned alongside Git and the hosting service's storage and bandwidth limits fit the project. Install/configure LFS, track selected patterns, and commit .gitattributes. Tracking a pattern does not convert copies already present in earlier commits.
  • DVC: can keep data content in configured external storage while Git tracks small metadata files. It requires an appropriately configured remote and permissions; do not point it at a storage location unless the data-use policy permits it.
  • Institutional storage: often the right place for controlled or very large data. Document how authorized collaborators request access and identify the input version without placing restricted data in Git.

Connect to GitHub only after a privacy review

Create an empty remote repository through GitHub's interface, then use its exact clone URL in the commands below. Replace the placeholders and verify the destination before pushing. This optional step writes to a remote account, so it is intentionally not part of the local-only practice run.

git remote add origin https://github.com/USER/REPO.git
git branch -M main
git push -u origin main

GitHub authentication may use HTTPS with an approved credential manager or SSH with a key linked to your account. Do not embed a personal access token in a URL, script, or shell command. Before a public push, inspect git status, review the staged diff, and scan files for likely secrets. A text search is only a screening aid; it cannot prove that a repository is safe to publish.

Clone, update, and contribute through review

For a first-time checkout, run the first two commands once. On later visits, skip them and start in the existing repository directory. Before updating it, check that the working tree is clean. If git status --short lists changes, inspect and preserve them before switching branches. This example assumes the shared default branch is named main:

git clone https://github.com/OWNER/REPOSITORY.git
cd REPOSITORY

# Before updating an existing clean clone, run these commands from its directory.
git status --short
# Continue only if this prints nothing.
git switch main
git pull --ff-only origin main

To propose a small change, start from the updated main branch, work on a topic branch, inspect the diff, and push that branch rather than committing directly to main:

git switch -c docs/input-notes
# Edit README.md in your editor, then inspect the change before staging.
git diff --check
git diff -- README.md
git add README.md
git diff --cached
git commit -m "Clarify project input notes"
git push -u origin docs/input-notes

Open a pull request with main as the base. A reviewer should check the complete diff, test results, and data permissions before approval and merge. If you do not have write access, fork the repository, clone your fork, and propose the change back to the original repository. After a pull request is merged, update a clean local main branch with the fast-forward-only commands above; for a fork, configure the original repository as upstream and use that remote in place of origin. If a fast-forward is not possible, stop and inspect both histories rather than using a force push or hard reset. Follow the repository's review policy and never include sensitive data.

Read the repository state

  • git status reports staged, modified, and untracked paths. A clean working tree means no uncommitted changes are visible to Git; it does not mean every required file is tracked.
  • git log --oneline --decorate summarizes local history. Focused commits make it easier to identify when a code or configuration change entered the project.
  • git diff --cached shows staged changes. Use it before committing, especially when a broad add command was used.
  • A commit ID records a repository tree and its ancestry. It does not capture an external reference genome, data object, cloud resource, or unrecorded package installation.

A successful script run checks that the script accepted that input. It does not validate the reads, the analysis design, or the biological interpretation. Reproducibility also depends on the input data and versions of dependencies outside Git.

Troubleshoot common problems

git commit fails with Author identity unknown

On a fresh workstation or container, Git refuses to commit until a name and email are configured. Run git config --global user.name "Your Name" and git config --global user.email "you@example.org" (if you prefer not to expose a personal or institutional email in public Git history, enable email privacy in GitHub settings and use your assigned ID+username@users.noreply.github.com address).

Push fails with an authentication error

Check that the remote URL is correct with git remote -v. For SSH, confirm that a key is configured with GitHub and that the SSH agent can access it. For HTTPS, use the credential prompt or your platform's credential manager. Never paste a token into a remote URL to bypass an authentication error.

A supposedly ignored file still appears in Git

Use git check-ignore -v path/to/file to see which rule applies. If the file is already tracked, ignore rules alone will not remove it from the index or history. For a non-sensitive tracked file you intend to keep locally, git rm --cached path/to/file removes it from the index for a future commit. Do not use this as a response to leaked secrets; revoke the credential first and follow your organization's incident process.

A secret or restricted file was committed

Revoke or rotate exposed credentials immediately. Removing a path in a later commit does not remove earlier copies from history. History rewriting can change commit IDs and disrupt collaborators; forks, clones, caches, and backups may retain copies. Follow GitHub's sensitive-data removal guidance and your institutional data-governance process. If personal or clinical data may have been exposed, contact the responsible privacy/security team rather than attempting an improvised cleanup.

The repository grows or clones slowly

Check whether large files entered prior commits. Adding LFS tracking now does not retroactively replace blobs already in history. Moving existing files out of history requires a planned migration and collaborator coordination. For controlled datasets, use an approved external store and document access instead of moving them to a public host.

Practice checklist

  1. Initialize a local repository and explain what Git stores in .git/.
  2. Add a README and reviewed ignore rules; confirm a test path with git check-ignore -v.
  3. Review git diff --cached, make a focused commit, and locate its ID with git log.
  4. Run the example script on a tiny synthetic TSV and verify the missing-header failure.
  5. Before any remote push, check data permissions, inspect staged content, and select an appropriate large-file strategy.
  6. Clone or update a clean copy with git pull --ff-only, propose a change on a topic branch, and send it through pull-request review.

Completion criterion: you can identify the tracked code snapshot, state what the commit does not capture, explain why ignore rules do not make sensitive files safe, and outline a branch-and-review cycle.

References

Software and package behaviour can change. Please re-check commands against the official documentation. If a command does not work, you find an error, or the tutorial is no longer reproducible, please report it through the Contact page.

Author: Nasir Mahmood Abbasi, PhD - Category: Foundations & Prerequisites

Back to topGit and GitHub for Bioinformatics