AI-Assisted Cell-Type Annotation: Models, Limits, and Review

Cell annotation tools assign labels by comparing a query dataset with learned classes, reference profiles, or embedded representations. Treat each assignment as a proposal: its usefulness depends on what the model or reference contains and whether the evidence fits your cells.

Choose a method by its input and comparison

These methods do not share one interchangeable “AI annotation” interface. CellTypist is a logistic-regression classifier with downloadable or custom models. SingleR transfers labels by comparing query expression with a labeled reference. The published CELLama method converts ranked expressed genes, and optionally metadata, into text for sentence-transformer embeddings; the study evaluates embedding-based cell typing against reference cells. Geneformer and scGPT are separate projects with distinct representations and task workflows. A model's general-purpose or foundation-model label does not establish support for every annotation task.

A shared output label does not make the methods' scores equivalent. Compare each result using its own documented diagnostics.

Primary R route: annotate against a labeled reference with SingleR

This is a reproducible R reference-transfer baseline, not an R implementation of CellTypist and not a foundation-model prediction. Prepare two SingleCellExperiment objects from public or synthetic inputs only: query_sce.rds and reference_sce.rds. Both must contain a logcounts assay with compatible gene identifiers; the reference must have a reviewed cell_type label in colData. This template uses those existing log-normalized assays as supplied. It does not normalize, map gene identifiers, choose labels, or fetch data. Review reference provenance, species, tissue, assay, and label granularity before running.

# Inputs: query_sce.rds and reference_sce.rds, prepared by the reader
# Output: singleR_predictions.csv and singleR_scores.csv
suppressPackageStartupMessages({
  library(SingleCellExperiment)
  library(SingleR)
})

query <- readRDS("query_sce.rds")
reference <- readRDS("reference_sce.rds")
if (!inherits(query, "SingleCellExperiment") || !inherits(reference, "SingleCellExperiment")) {
  stop("Both RDS inputs must contain SingleCellExperiment objects")
}
if (!"logcounts" %in% assayNames(query) || !"logcounts" %in% assayNames(reference)) {
  stop("Both objects must contain the declared logcounts assay")
}
if (!"cell_type" %in% colnames(colData(reference))) {
  stop("Reference needs a reviewed cell_type label column in colData")
}
labels <- as.character(colData(reference)$cell_type)
if (anyNA(labels) || any(!nzchar(labels))) {
  stop("Reference labels must be present and non-empty")
}
shared_genes <- intersect(rownames(query), rownames(reference))
if (length(shared_genes) < 2L) {
  stop("Too few exact shared gene identifiers; resolve the feature contract first")
}
query <- query[shared_genes, ]
reference <- reference[shared_genes, ]
if (anyDuplicated(rownames(query)) || anyDuplicated(rownames(reference))) {
  stop("Resolve duplicate gene identifiers before annotation")
}
if (is.null(colnames(query)) || anyDuplicated(colnames(query))) {
  stop("Query cell names must be present and unique")
}
if (is.null(colnames(reference)) || anyDuplicated(colnames(reference))) {
  stop("Reference cell names must be present and unique")
}
prediction <- SingleR(test = query, ref = reference, labels = labels,
                     assay.type.test = "logcounts", assay.type.ref = "logcounts",
                     prune = TRUE)
results <- data.frame(
  cell_id = rownames(prediction),
  label = prediction$labels,
  pruned_label = prediction$pruned.labels,
  delta_next = prediction$delta.next,
  stringsAsFactors = FALSE
)
write.csv(results, "singleR_predictions.csv", row.names = FALSE)
write.csv(as.data.frame(prediction$scores), "singleR_scores.csv", row.names = TRUE)
message("Annotated ", nrow(results), " query cells using ", length(shared_genes),
        " exact shared genes. Review labels, score separation, pruning, and reference fit.")

Run only after confirming the input contracts and package versions for your environment. A pruned/unassigned label is a useful uncertainty signal, not a failure to be filled automatically. Inspect score patterns, marker evidence, and reference fit; a technically successful transfer is not independent validation and cannot recover cell states absent from the reference.

Optional Python-only method: run and inspect CellTypist

CellTypist inference in this section is Python-only; it is a distinct classifier route, not a matching implementation of the R SingleR baseline. Do not interpret the two tools' scores as interchangeable. CellTypist’s input and model compatibility requirements below apply only to this optional method.

This optional Python template expects a reader-prepared my_data.h5ad made from public or synthetic data only: AnnData with cells in rows, genes in columns, and an explicitly documented raw-count matrix. Start in a dedicated working directory, create an isolated Python environment, and install the two packages (current releases and dependency resolution may change):

mkdir -p celltypist-review
cd celltypist-review
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install scanpy celltypist
python -c "import scanpy, celltypist; print('scanpy', scanpy.__version__, 'celltypist', celltypist.__version__)"
# Put your authorized input and selected model here.

Record the environment actually used with python -m pip freeze > requirements-record.txt; this captures the resolved environment, not a tested or guaranteed lock. Download a selected model from the official CellTypist model catalog into this working directory as Immune_All_Low.pkl. Inspect its catalog description and confirm the species, reference scope, tissue and assay suitability for your question. This model is only an example, not a universal choice. The code hashes the exact local model file so the run can be identified even if a catalog version label is absent.

Before execution, establish from upstream records that counts contains untransformed raw counts. The declared species and identifier conventions below are assertions you supply from provenance and model documentation. They are not inferred from feature names. Unique names only rule out duplicates: they do not establish species, symbol convention, or model compatibility. The exact feature overlap is measured against the loaded model before normalization or annotation.

Preflight and annotate

Set the query and model declarations to match documented provenance. Supply a pre-existing review_overcluster observation column whose clustering choices you have reviewed. Confirm that query and model tissue, condition, assay and reference scope suit the question, then set the explicit review flag in the code; this compatibility check is a documented human decision, not automatic inference. The 25% model-feature overlap cutoff is a configurable teaching refusal gate: it rejects a run with very sparse direct matches, but it is not a universal biological validity threshold. Calibrate a defensible cutoff for the selected model and assay before interpreting real data. Identifier-pattern checking below flags a common Ensembl-versus-symbol mismatch only; it does not infer species or validate arbitrary identifiers.

from datetime import datetime, timezone
from pathlib import Path
import hashlib
import json
import re
import sys

import anndata as ad
import celltypist
import numpy as np
import scanpy as sc
from celltypist import models

INPUT_PATH = Path("my_data.h5ad")
MODEL_PATH = Path("Immune_All_Low.pkl").resolve()
INPUT_SPECIES = "human"                 # Set from input provenance, never inferred
MODEL_SPECIES = "human"                 # Verify against the official model description
INPUT_FEATURE_ID_CONVENTION = "HGNC gene symbols"
MODEL_FEATURE_ID_CONVENTION = "HGNC gene symbols"  # Verify catalog/model context
INPUT_CONTEXT_NOTE = "Human immune-cell query; replace with documented tissue and condition"
MODEL_CONTEXT_NOTE = "Human immune-cell reference; review tissue and assay coverage"
CONTEXT_COMPATIBILITY_REVIEWED = False  # Set True only after comparing query/model context
MODEL_SOURCE_URL = "https://www.celltypist.org/models"
OVER_CLUSTERING_KEY = "review_overcluster"  # Required, reviewed AnnData obs column
MIN_MODEL_FEATURE_FRACTION = 0.25  # Teaching gate only, not a biological standard
MANIFEST_PATH = Path("celltypist-run-manifest.json")

if not INPUT_PATH.is_file() or not MODEL_PATH.is_file():
    raise FileNotFoundError("Place the authorized input and selected model in this directory")
if not INPUT_SPECIES.strip() or not MODEL_SPECIES.strip():
    raise ValueError("Declare both input and model species from documented provenance")
if INPUT_SPECIES.casefold() != MODEL_SPECIES.casefold():
    raise ValueError("Declared input/model species differ; stop and select a compatible model")
if not INPUT_FEATURE_ID_CONVENTION.strip() or not MODEL_FEATURE_ID_CONVENTION.strip():
    raise ValueError("Declare both input and model feature identifier conventions")
if INPUT_FEATURE_ID_CONVENTION.casefold() != MODEL_FEATURE_ID_CONVENTION.casefold():
    raise ValueError("Declared input/model identifier conventions differ; map IDs explicitly")
if not INPUT_CONTEXT_NOTE.strip() or not MODEL_CONTEXT_NOTE.strip() or not MODEL_SOURCE_URL.strip():
    raise ValueError("Record the query context, model context, and model source")
if CONTEXT_COMPATIBILITY_REVIEWED is not True:
    raise ValueError("Review query/model context and explicitly set CONTEXT_COMPATIBILITY_REVIEWED = True")
if not 0 < MIN_MODEL_FEATURE_FRACTION <= 1:
    raise ValueError("The overlap policy must be a fraction in (0, 1]")

input_sha256 = hashlib.sha256(INPUT_PATH.read_bytes()).hexdigest()
model_sha256 = hashlib.sha256(MODEL_PATH.read_bytes()).hexdigest()
adata = sc.read_h5ad(INPUT_PATH)
if adata.n_obs == 0 or adata.n_vars == 0:
    raise ValueError("Empty input: stop and check the file")
if not adata.var_names.is_unique:
    raise ValueError("Duplicate feature names: resolve before prediction")
if "counts" not in adata.layers:
    raise ValueError("No verified raw-count layer; stop rather than infer provenance")
if OVER_CLUSTERING_KEY not in adata.obs:
    raise ValueError("Add a reviewed over-clustering column before majority voting")
if adata.obs[OVER_CLUSTERING_KEY].isna().any():
    raise ValueError("Over-clustering labels are missing for one or more cells")

# An explicit path avoids catalog lookup; forward slashes also work on Windows.
model = models.Model.load(MODEL_PATH.as_posix())
query_features = list(map(str, adata.var_names))
model_features = list(map(str, model.features))
if not model_features:
    raise ValueError("Loaded model reports no features; stop for manual inspection")
matched_features = sorted(set(query_features).intersection(model_features))
model_feature_fraction = len(matched_features) / len(model_features)
query_feature_fraction = len(matched_features) / len(query_features)
ensembl_like = sum(bool(re.fullmatch(r"ENS[A-Z]*G\d+(?:\.\d+)?", name))
                   for name in query_features) / len(query_features)
issues = []
if "symbol" in INPUT_FEATURE_ID_CONVENTION.casefold() and ensembl_like >= 0.5:
    issues.append("Most query IDs resemble Ensembl gene IDs despite the declared symbol convention")
if model_feature_fraction < MIN_MODEL_FEATURE_FRACTION:
    issues.append("Direct model-feature overlap is below the configured refusal threshold")

description = model.description
if not description:
    raise ValueError("Loaded model has no description; establish model context before prediction")
manifest = {
    "started_at_utc": datetime.now(timezone.utc).isoformat(),
    "status": "refused_preflight" if issues else "preflight_passed_pending_count_checks",
    "input_file": INPUT_PATH.name,
    "input_sha256": input_sha256,
    "input_species_declared": INPUT_SPECIES,
    "input_feature_id_convention_declared": INPUT_FEATURE_ID_CONVENTION,
    "model_identifier": MODEL_PATH.name,
    "model_sha256": model_sha256,
    "model_species_declared": MODEL_SPECIES,
    "model_feature_id_convention_declared": MODEL_FEATURE_ID_CONVENTION,
    "model_description": description,
    "input_context_note": INPUT_CONTEXT_NOTE,
    "model_context_note": MODEL_CONTEXT_NOTE,
    "context_compatibility_reviewed": CONTEXT_COMPATIBILITY_REVIEWED,
    "model_source": MODEL_SOURCE_URL,
    "software_versions": {
        "python": sys.version.split()[0],
        "anndata": getattr(ad, "__version__", "not reported"),
        "celltypist": getattr(celltypist, "__version__", "not reported"),
        "numpy": getattr(np, "__version__", "not reported"),
        "scanpy": getattr(sc, "__version__", "not reported"),
    },
    "preprocessing_choices": {
        "counts_layer": "counts, provenance verified by the user",
        "X_before_annotation": "copy of counts, then normalize_total(target_sum=10000) and log1p",
        "majority_voting": True,
        "over_clustering_obs_key": OVER_CLUSTERING_KEY,
    },
    "overlap_policy": {
        "metric": "exact shared feature IDs divided by model feature count",
        "minimum_fraction": MIN_MODEL_FEATURE_FRACTION,
        "threshold_status": "configurable teaching refusal gate, not universal biology",
    },
    "overlap_report": {
        "query_feature_count": len(query_features),
        "model_feature_count": len(model_features),
        "exact_matched_feature_count": len(matched_features),
        "model_feature_fraction_matched": model_feature_fraction,
        "query_feature_fraction_matched": query_feature_fraction,
        "matched_examples": matched_features[:20],
        "query_only_examples": sorted(set(query_features) - set(model_features))[:20],
        "model_only_examples": sorted(set(model_features) - set(query_features))[:20],
        "ensembl_like_query_fraction_pattern_check_only": ensembl_like,
    },
    "preflight_issues": issues,
}
MANIFEST_PATH.write_text(json.dumps(manifest, indent=2, default=str) + "\n", encoding="utf-8")
if not MANIFEST_PATH.is_file():
    raise RuntimeError("Run manifest was not written; stop before annotation")
print("Preflight report:", json.dumps(manifest["overlap_report"], indent=2))
if issues:
    raise RuntimeError("Preflight refused annotation: " + "; ".join(issues))

# Numerical checks reject obvious count violations, but cannot prove provenance.
counts = adata.layers["counts"]
values = counts.data if hasattr(counts, "data") and hasattr(counts, "indptr") else np.asarray(counts)
if not np.isfinite(values).all() or (values < 0).any():
    raise ValueError("Counts contain invalid values")
if not np.allclose(values, np.rint(values)):
    raise ValueError("Values are not integer-like raw counts")
manifest["status"] = "preflight_passed"
MANIFEST_PATH.write_text(json.dumps(manifest, indent=2, default=str) + "\n", encoding="utf-8")
if not MANIFEST_PATH.is_file():
    raise RuntimeError("Validated preflight manifest is missing; stop before annotation")
adata.X = counts.copy()
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)

pred = celltypist.annotate(
    adata, model=model, majority_voting=True, over_clustering=OVER_CLUSTERING_KEY
)
labels = pred.predicted_labels
expected_ids = list(map(str, adata.obs_names))
def require_same_cell_order(frame, name):
    if list(map(str, frame.index)) != expected_ids:
        raise RuntimeError(name + " cell IDs/order do not match the input")

require_same_cell_order(labels, "Predicted labels")
if not {"predicted_labels", "majority_voting"}.issubset(labels.columns):
    raise RuntimeError("CellTypist did not return both cell and majority-vote labels")
prob = pred.probability_matrix
decision = pred.decision_matrix
require_same_cell_order(prob, "Probability matrix")
require_same_cell_order(decision, "Decision matrix")
if prob.shape[0] != adata.n_obs or prob.shape[1] < 2:
    raise RuntimeError("Probability matrix has unexpected dimensions")
if decision.shape != prob.shape:
    raise RuntimeError("Decision and probability matrices have different dimensions")

# The documented result method supplies majority-vote confidence in obs.
result_adata = pred.to_adata(
    insert_labels=True, insert_conf=True, insert_conf_by="majority_voting",
    prefix="celltypist_"
)
if list(map(str, result_adata.obs_names)) != expected_ids:
    raise RuntimeError("Result AnnData cell IDs/order do not match the input")
if "celltypist_conf_score" not in result_adata.obs:
    raise RuntimeError("Expected majority-vote confidence score is absent")
prob_sorted = np.sort(prob.to_numpy(), axis=1)
decision_sorted = np.sort(decision.to_numpy(), axis=1)
review = labels[["predicted_labels", "majority_voting"]].copy()
review["cell_probability_top"] = prob_sorted[:, -1]
review["cell_probability_margin"] = prob_sorted[:, -1] - prob_sorted[:, -2]
review["cell_decision_top"] = decision_sorted[:, -1]
review["cell_decision_margin"] = decision_sorted[:, -1] - decision_sorted[:, -2]
review["majority_voting_conf_score"] = result_adata.obs["celltypist_conf_score"].to_numpy()
review.index.name = "cell_id"
review.to_csv("celltypist-review.csv")
prob.to_csv("celltypist-probabilities.csv")
decision.to_csv("celltypist-decision-scores.csv")
result_adata.write_h5ad("celltypist-annotated.h5ad")
manifest["status"] = "outputs_written"
manifest["outputs"] = ["celltypist-review.csv", "celltypist-probabilities.csv",
                       "celltypist-decision-scores.csv", "celltypist-annotated.h5ad"]
manifest["completed_at_utc"] = datetime.now(timezone.utc).isoformat()
MANIFEST_PATH.write_text(json.dumps(manifest, indent=2, default=str) + "\n", encoding="utf-8")
if not MANIFEST_PATH.is_file():
    raise RuntimeError("Completed run manifest is missing")
print(review.head())
print("Saved manifest and cell-aligned diagnostics for", adata.n_obs, "cells")

The preflight manifest is written and checked before the refusal gate; a refused run records the overlap and reason without calling annotate. On success, preserve the original cell IDs in the review table, both label columns, per-cell probability and decision margins, majority-vote confidence, and full probability/decision matrices. The annotated H5AD retains the input observations and count layer. The manifest records input and model SHA256 checksums, model description and source, timestamp, package versions, preprocessing, declarations, overlap policy and report. These scores are diagnostics, not calibrated biological certainty. A successful run only establishes that the computational checks and outputs completed, not that labels are biologically correct.

Review predictions against independent evidence

For each cluster or cell group under review, make a small decision record. Include the model's label, an appropriate marker panel, the tissue/species/condition covered by the reference, and a reason to accept, revise, or leave the label unresolved. Marker panels are context-dependent; use positive and counter-markers where possible, and examine whether a cluster mixes distinct populations. A label match alone does not validate a novel cell state or biological mechanism.

1. Check the data and model. Record species, tissue, assay, preprocessing, model/reference identifier and software version. Check gene overlap and whether the query has conditions absent from the reference.
2. Inspect ambiguity. Compare available score/probability diagnostics across candidate labels. For SingleR, inspect score patterns and pruned.labels; pruning can leave uncertain assignments unlabelled. A narrow score gap or unresolved output is a reason to review, not a command to force the nearest label.
3. Compare biology. Check known positive and conflicting markers, cluster coherence and tissue context. Investigate discordance instead of choosing whichever tool sounds more advanced.
4. Write the disposition. Save the predicted label, evidence, reference limitation and decision (accept, revise, unresolved). Revisit the call when improved references or orthogonal evidence become available.

Failure cases worth checking

• Unmapped or duplicated genes: inspect feature identifiers and overlap with the model/reference. Unexpectedly low overlap can make predictions unreliable or fail outright.
• Declared context or overlap gate fails: a synthetic low-overlap query should produce a refusal manifest and no predictions. Investigate the ID namespace and selected model; do not lower the cutoff solely to make a run pass.
• Reference mismatch: a human immune model is not suitable simply because the query contains immune cells; species, tissue, disease and assay context affect transfer.
• Strong scores with contradictory markers: investigate preprocessing, doublets, mixed clusters and classes missing from the reference. A classifier still assigns the closest available class in best-match mode.
• Majority-vote labels erase heterogeneity: compare refined labels with per-cell predictions and the clustering used for refinement. The result depends on over-clustering choices.
• Input privacy: use public or synthetic inputs for the examples in this lesson; do not upload patient-linked counts, metadata or cell identifiers to external tools.

References and version checks

The prediction and diagnostic stages completed on 2,638 cells prepared from the official public 10x PBMC 3k counts, using CellTypist 1.7.1, Scanpy 1.12.4 and the official Immune_All_Low.pkl version-2 model on Ubuntu. The query matched 4,177 of the model's 6,639 features; cell-aligned review, probability and decision tables, an annotated H5AD and a completed run manifest were written. This checks software execution, not independent biological label accuracy. The synthetic low-overlap refusal test is retained. The model is loaded by an explicit local path: in CellTypist 1.7.1, a basename can trigger a catalog lookup and download other models even when the named file is present locally. Native Windows/macOS model inference and other datasets remain untested. Record installed versions and recheck APIs, model inventory and defaults before use.

• CellTypist prediction function: accepted inputs, model selection, majority voting and result object.
• CellTypist Model structure: documented features and description properties used by the preflight.
• CellTypist prediction result: predicted labels, decision/probability matrices, and to_adata confidence-score options. Documentation reviewed 2026-10-23; APIs and model metadata can change.
• CellTypist documentation: tool overview and current documentation entry point.
• SingleR Bioconductor vignette: reference-based annotation, score diagnostics and pruning.
• CELLama primary paper: ranked-gene sentence construction, embeddings and study evaluation. Results in that paper do not establish performance on an unrelated dataset.
• Scanpy normalize_total and log1p: normalization calls used in the example.