鱼皮AI导航
鱼皮AI导航
  • 主页
  • 社区
  • AI工具
  • AI知识库
  • AI提示词
  • MCP
  • Skills
  • AI学习资源
  • AI资讯
  • GitHub开源
登录/注册

cellxgene-census

支持对数千万单细胞数据进行大规模查询与分析,可按细胞类型、组织、疾病等条件筛选,获取基因表达矩阵及元数据,并无缝对接 scanpy 和 PyTorch 等工具开展下游建模与可视化。

数据与分析研究
访问 GitHub
文档Skills.MD
name
cellxgene-census
description
Query CZ CELLxGENE Census (61M+ cells). Filter by cell type/tissue/disease, retrieve expression data, integrate with scanpy/PyTorch, for population-scale single-cell analysis.

CZ CELLxGENE Census

Overview

The CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell genomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of millions of cells across thousands of datasets.

The Census includes:

  • 61+ million cells from human and mouse
  • Standardized metadata (cell types, tissues, diseases, donors)
  • Raw gene expression matrices
  • Pre-calculated embeddings and statistics
  • Integration with PyTorch, scanpy, and other analysis tools

When to Use This Skill

This skill should be used when:

  • Querying single-cell expression data by cell type, tissue, or disease
  • Exploring available single-cell datasets and metadata
  • Training machine learning models on single-cell data
  • Performing large-scale cross-dataset analyses
  • Integrating Census data with scanpy or other analysis frameworks
  • Computing statistics across millions of cells
  • Accessing pre-calculated embeddings or model predictions

Installation and Setup

Install the Census API:

▼
bash
复制代码
uv pip install cellxgene-census

For machine learning workflows, install additional dependencies:

▼
bash
复制代码
uv pip install cellxgene-census[experimental]

Core Workflow Patterns

1. Opening the Census

Always use the context manager to ensure proper resource cleanup:

▼
python
复制代码
import cellxgene_census # Open latest stable version with cellxgene_census.open_soma() as census: # Work with census data # Open specific version for reproducibility with cellxgene_census.open_soma(census_version="2023-07-25") as census: # Work with census data

Key points:

  • Use context manager (with statement) for automatic cleanup
  • Specify census_version for reproducible analyses
  • Default opens latest "stable" release

2. Exploring Census Information

Before querying expression data, explore available datasets and metadata.

Access summary information:

▼
python
复制代码
# Get summary statistics summary = census["census_info"]["summary"].read().concat().to_pandas() print(f"Total cells: {summary['total_cell_count'][0]}") # Get all datasets datasets = census["census_info"]["datasets"].read().concat().to_pandas() # Filter datasets by criteria covid_datasets = datasets[datasets["disease"].str.contains("COVID", na=False)]

Query cell metadata to understand available data:

▼
python
复制代码
# Get unique cell types in a tissue cell_metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'brain' and is_primary_data == True", column_names=["cell_type"] ) unique_cell_types = cell_metadata["cell_type"].unique() print(f"Found {len(unique_cell_types)} cell types in brain") # Count cells by tissue tissue_counts = cell_metadata.groupby("tissue_general").size()

Important: Always filter for is_primary_data == True to avoid counting duplicate cells unless specifically analyzing duplicates.

3. Querying Expression Data (Small to Medium Scale)

For queries returning < 100k cells that fit in memory, use get_anndata():

▼
python
复制代码
# Basic query with cell type and tissue filters adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", # or "Mus musculus" obs_value_filter="cell_type == 'B cell' and tissue_general == 'lung' and is_primary_data == True", obs_column_names=["assay", "disease", "sex", "donor_id"], ) # Query specific genes with multiple filters adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", var_value_filter="feature_name in ['CD4', 'CD8A', 'CD19', 'FOXP3']", obs_value_filter="cell_type == 'T cell' and disease == 'COVID-19' and is_primary_data == True", obs_column_names=["cell_type", "tissue_general", "donor_id"], )

Filter syntax:

  • Use obs_value_filter for cell filtering
  • Use var_value_filter for gene filtering
  • Combine conditions with and, or
  • Use in for multiple values: tissue in ['lung', 'liver']
  • Select only needed columns with obs_column_names

Getting metadata separately:

▼
python
复制代码
# Query cell metadata cell_metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="disease == 'COVID-19' and is_primary_data == True", column_names=["cell_type", "tissue_general", "donor_id"] ) # Query gene metadata gene_metadata = cellxgene_census.get_var( census, "homo_sapiens", value_filter="feature_name in ['CD4', 'CD8A']", column_names=["feature_id", "feature_name", "feature_length"] )

4. Large-Scale Queries (Out-of-Core Processing)

For queries exceeding available RAM, use axis_query() with iterative processing:

▼
python
复制代码
import tiledbsoma as soma # Create axis query query = census["census_data"]["homo_sapiens"].axis_query( measurement_name="RNA", obs_query=soma.AxisQuery( value_filter="tissue_general == 'brain' and is_primary_data == True" ), var_query=soma.AxisQuery( value_filter="feature_name in ['FOXP2', 'TBR1', 'SATB2']" ) ) # Iterate through expression matrix in chunks iterator = query.X("raw").tables() for batch in iterator: # batch is a pyarrow.Table with columns: # - soma_data: expression value # - soma_dim_0: cell (obs) coordinate # - soma_dim_1: gene (var) coordinate process_batch(batch)

Computing incremental statistics:

▼
python
复制代码
# Example: Calculate mean expression n_observations = 0 sum_values = 0.0 iterator = query.X("raw").tables() for batch in iterator: values = batch["soma_data"].to_numpy() n_observations += len(values) sum_values += values.sum() mean_expression = sum_values / n_observations

5. Machine Learning with PyTorch

For training models, use the experimental PyTorch integration:

▼
python
复制代码
from cellxgene_census.experimental.ml import experiment_dataloader with cellxgene_census.open_soma() as census: # Create dataloader dataloader = experiment_dataloader( census["census_data"]["homo_sapiens"], measurement_name="RNA", X_name="raw", obs_value_filter="tissue_general == 'liver' and is_primary_data == True", obs_column_names=["cell_type"], batch_size=128, shuffle=True, ) # Training loop for epoch in range(num_epochs): for batch in dataloader: X = batch["X"] # Gene expression tensor labels = batch["obs"]["cell_type"] # Cell type labels # Forward pass outputs = model(X) loss = criterion(outputs, labels) # Backward pass optimizer.zero_grad() loss.backward() optimizer.step()

Train/test splitting:

▼
python
复制代码
from cellxgene_census.experimental.ml import ExperimentDataset # Create dataset from experiment dataset = ExperimentDataset( experiment_axis_query, layer_name="raw", obs_column_names=["cell_type"], batch_size=128, ) # Split into train and test train_dataset, test_dataset = dataset.random_split( split=[0.8, 0.2], seed=42 )

6. Integration with Scanpy

Seamlessly integrate Census data with scanpy workflows:

▼
python
复制代码
import scanpy as sc # Load data from Census adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="cell_type == 'neuron' and tissue_general == 'cortex' and is_primary_data == True", ) # Standard scanpy workflow sc.pp.normalize_total(adata, target_sum=1e4) sc.pp.log1p(adata) sc.pp.highly_variable_genes(adata, n_top_genes=2000) # Dimensionality reduction sc.pp.pca(adata, n_comps=50) sc.pp.neighbors(adata) sc.tl.umap(adata) # Visualization sc.pl.umap(adata, color=["cell_type", "tissue", "disease"])

7. Multi-Dataset Integration

Query and integrate multiple datasets:

▼
python
复制代码
# Strategy 1: Query multiple tissues separately tissues = ["lung", "liver", "kidney"] adatas = [] for tissue in tissues: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter=f"tissue_general == '{tissue}' and is_primary_data == True", ) adata.obs["tissue"] = tissue adatas.append(adata) # Concatenate combined = adatas[0].concatenate(adatas[1:]) # Strategy 2: Query multiple datasets directly adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="tissue_general in ['lung', 'liver', 'kidney'] and is_primary_data == True", )

Key Concepts and Best Practices

Always Filter for Primary Data

Unless analyzing duplicates, always include is_primary_data == True in queries to avoid counting cells multiple times:

▼
python
复制代码
obs_value_filter="cell_type == 'B cell' and is_primary_data == True"

Specify Census Version for Reproducibility

Always specify the Census version in production analyses:

▼
python
复制代码
census = cellxgene_census.open_soma(census_version="2023-07-25")

Estimate Query Size Before Loading

For large queries, first check the number of cells to avoid memory issues:

▼
python
复制代码
# Get cell count metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'brain' and is_primary_data == True", column_names=["soma_joinid"] ) n_cells = len(metadata) print(f"Query will return {n_cells:,} cells") # If too large (>100k), use out-of-core processing

Use tissue_general for Broader Groupings

The tissue_general field provides coarser categories than tissue, useful for cross-tissue analyses:

▼
python
复制代码
# Broader grouping obs_value_filter="tissue_general == 'immune system'" # Specific tissue obs_value_filter="tissue == 'peripheral blood mononuclear cell'"

Select Only Needed Columns

Minimize data transfer by specifying only required metadata columns:

▼
python
复制代码
obs_column_names=["cell_type", "tissue_general", "disease"] # Not all columns

Check Dataset Presence for Gene-Specific Queries

When analyzing specific genes, verify which datasets measured them:

▼
python
复制代码
presence = cellxgene_census.get_presence_matrix( census, "homo_sapiens", var_value_filter="feature_name in ['CD4', 'CD8A']" )

Two-Step Workflow: Explore Then Query

First explore metadata to understand available data, then query expression:

▼
python
复制代码
# Step 1: Explore what's available metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="disease == 'COVID-19' and is_primary_data == True", column_names=["cell_type", "tissue_general"] ) print(metadata.value_counts()) # Step 2: Query based on findings adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True", )

Available Metadata Fields

Cell Metadata (obs)

Key fields for filtering:

  • cell_type, cell_type_ontology_term_id
  • tissue, tissue_general, tissue_ontology_term_id
  • disease, disease_ontology_term_id
  • assay, assay_ontology_term_id
  • donor_id, sex, self_reported_ethnicity
  • development_stage, development_stage_ontology_term_id
  • dataset_id
  • is_primary_data (Boolean: True = unique cell)

Gene Metadata (var)

  • feature_id (Ensembl gene ID, e.g., "ENSG00000161798")
  • feature_name (Gene symbol, e.g., "FOXP2")
  • feature_length (Gene length in base pairs)

Reference Documentation

This skill includes detailed reference documentation:

references/census_schema.md

Comprehensive documentation of:

  • Census data structure and organization
  • All available metadata fields
  • Value filter syntax and operators
  • SOMA object types
  • Data inclusion criteria

When to read: When you need detailed schema information, full list of metadata fields, or complex filter syntax.

references/common_patterns.md

Examples and patterns for:

  • Exploratory queries (metadata only)
  • Small-to-medium queries (AnnData)
  • Large queries (out-of-core processing)
  • PyTorch integration
  • Scanpy integration workflows
  • Multi-dataset integration
  • Best practices and common pitfalls

When to read: When implementing specific query patterns, looking for code examples, or troubleshooting common issues.

Common Use Cases

Use Case 1: Explore Cell Types in a Tissue

▼
python
复制代码
with cellxgene_census.open_soma() as census: cells = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'lung' and is_primary_data == True", column_names=["cell_type"] ) print(cells["cell_type"].value_counts())

Use Case 2: Query Marker Gene Expression

▼
python
复制代码
with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", var_value_filter="feature_name in ['CD4', 'CD8A', 'CD19']", obs_value_filter="cell_type in ['T cell', 'B cell'] and is_primary_data == True", )

Use Case 3: Train Cell Type Classifier

▼
python
复制代码
from cellxgene_census.experimental.ml import experiment_dataloader with cellxgene_census.open_soma() as census: dataloader = experiment_dataloader( census["census_data"]["homo_sapiens"], measurement_name="RNA", X_name="raw", obs_value_filter="is_primary_data == True", obs_column_names=["cell_type"], batch_size=128, shuffle=True, ) # Train model for epoch in range(epochs): for batch in dataloader: # Training logic pass

Use Case 4: Cross-Tissue Analysis

▼
python
复制代码
with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="cell_type == 'macrophage' and tissue_general in ['lung', 'liver', 'brain'] and is_primary_data == True", ) # Analyze macrophage differences across tissues sc.tl.rank_genes_groups(adata, groupby="tissue_general")

Troubleshooting

Query Returns Too Many Cells

  • Add more specific filters to reduce scope
  • Use tissue instead of tissue_general for finer granularity
  • Filter by specific dataset_id if known
  • Switch to out-of-core processing for large queries

Memory Errors

  • Reduce query scope with more restrictive filters
  • Select fewer genes with var_value_filter
  • Use out-of-core processing with axis_query()
  • Process data in batches

Duplicate Cells in Results

  • Always include is_primary_data == True in filters
  • Check if intentionally querying across multiple datasets

Gene Not Found

  • Verify gene name spelling (case-sensitive)
  • Try Ensembl ID with feature_id instead of feature_name
  • Check dataset presence matrix to see if gene was measured
  • Some genes may have been filtered during Census construction

Version Inconsistencies

  • Always specify census_version explicitly
  • Use same version across all analyses
  • Check release notes for version-specific changes
数据与分析
研究

快捷安装

复制到终端去安装 Skill

npxskillsadddavila7/claude-code-templates--skill"cellxgene-census"
站长:程序员鱼皮 老鱼简历 商务合作 增值电信业务经营许可证
2026 鱼皮AI导航 | 沪ICP备19026706号-6
站长:程序员鱼皮 老鱼简历 商务合作 增值电信业务经营许可证
2026 鱼皮AI导航 | 沪ICP备19026706号-6
🔥热门工具
朱雀AI检测朱雀AI检测
朱雀AI检测助手朱雀AI检测助手
LovartLovart
老鱼简历老鱼简历
WritePass官网WritePass官网
面多多面多多
CursorCursor
星月写作星月写作
朱雀AI检测助手朱雀AI检测助手
零感AI零感AI
Skills 推荐
definition-of-done-generator
自动生成项目完成标准的规范定义与配置文件,提供符合企业级工作流要求的最佳实践指导,输出可直接投入生产环境的代码和验证结果,适用于需要标准化流程审批、合规审查及跨系统集成的复杂项目管理场景。
sagemaker-endpoint-deployer
自动化部署机器学习模型至云端服务端点,支持从配置生成到生产验证的全流程,适用于模型上线、服务监控与推理优化等场景,确保部署方案符合工业级标准。
docx
支持专业级 Word 文档的全生命周期处理,包括从零创建结构化文档、精准编辑内容、管理修订痕迹与批注、提取纯文本或原始 XML 数据、转换为 PDF 或图像用于视觉分析,适用于法律、学术及企业场景下的高保真文档协作与自动化处理。
skills-search
提供对 Claude Code 技能生态的全周期管理能力,支持按功能关键词检索技能、查看详细信息、安装或卸载指定技能,并区分用户级与项目级作用域,覆盖发现、评估、部署及维护各环节。
moai-foundation-core
构建高质量、可扩展的AI开发工作流,通过领域驱动设计与模块化架构实现需求规范先行,结合自动化质量门禁、精细化上下文管理与渐进式知识呈现,在保障安全与可维护性的同时提升开发效率与协作清晰度。