Spatial Simulators

Assessing the quality of spatial transcriptomics simulators

Repository: openproblems-bio/task_spatial_simulators

Description

Computational methods for spatially resolved transcriptomics (SRT) are frequently developed and assessed through data simulation. The effectiveness of these evaluations relies on the simulation methods’ ability to accurately reflect experimental data. However, a systematic evaluation framework for spatial simulators is lacking. Here, we present SpatialSimBench, a comprehensive evaluation framework that assesses 13 simulation methods using 10 distinct STR datasets.

The research goal of this benchmark is to systematically evaluate and compare the performance of various simulation methods for spatial transcriptomics (ST) data. It aims to address the lack of a comprehensive evaluation framework for spatial simulators and explore the feasibility of leveraging existing single-cell simulators for ST data. The experimental setup involves collecting public spatial transcriptomics datasets and corresponding scRNA-seq datasets. The spatial and scRNA-seq datasets can originate from different study but should consist of similar cell types from similar tissues.

Authors & contributors

Name	Roles	Github	Orcid
Xiaoqi Liang	author	littlecabiria	0009-0004-9625-1441
Yue Cao	author, maintainer	ycao6928	0000-0002-2356-4031
Jean Yang	author	jeany21	0000-0002-5271-2603
Robrecht Cannoodt	contributor	rcannood	0000-0003-3641-729X
Sai Nirmayi Yasa	contributor	sainirmayi	0009-0003-6319-9803

API

flowchart TB
  comp_process_datasets[/"<a href='https://github.com/openproblems-bio/task_spatial_simulators#component-type-process-dataset'>Process Dataset</a>"/]
  file_dataset_sc("<a href='https://github.com/openproblems-bio/task_spatial_simulators#file-format-single-cell-dataset'>Single-Cell Dataset</a>")
  file_dataset_sp("<a href='https://github.com/openproblems-bio/task_spatial_simulators#file-format-spatial-dataset'>Spatial Dataset</a>")
  comp_metric[/"<a href='https://github.com/openproblems-bio/task_spatial_simulators#component-type-metric'>Metric</a>"/]
  comp_control_method[/"<a href='https://github.com/openproblems-bio/task_spatial_simulators#component-type-control-method'>Control Method</a>"/]
  comp_method[/"<a href='https://github.com/openproblems-bio/task_spatial_simulators#component-type-method'>Method</a>"/]
  file_score("<a href='https://github.com/openproblems-bio/task_spatial_simulators#file-format-score'>Score</a>")
  file_simulated_dataset("<a href='https://github.com/openproblems-bio/task_spatial_simulators#file-format-solution'>Solution</a>")
  comp_process_datasets-->file_dataset_sc
  comp_process_datasets-->file_dataset_sp
  file_dataset_sc---comp_metric
  file_dataset_sp---comp_metric
  file_dataset_sp-.-comp_control_method
  file_dataset_sp-.-comp_method
  comp_metric-->file_score
  comp_control_method-->file_simulated_dataset
  comp_method-->file_simulated_dataset
  file_simulated_dataset---comp_metric

Loading

Component type: Process Dataset

Preprocessing of spatial transcriptomics and single-cell transcriptomics datasets.

Arguments:

Name	Type	Description
--input_sc	file	.
--input_sp	file	.
--output_sc	file	(Output) An unprocessed single-cell dataset as output by a dataset loader. Default: $id/output_sc.h5ad.
--output_sp	file	(Output) An unprocessed spatial dataset as output by a dataset loader. Default: $id/output_sp.h5ad.

File format: Single-Cell Dataset

An unprocessed single-cell dataset as output by a dataset loader.

Example file: resources_test/spatialsimbench_mobnew/dataset_sc.h5ad

Description:

This dataset contains raw counts and metadata as output by a dataset loader.

The format of this file is derived from the CELLxGENE schema v4.0.0.

File format: Spatial Dataset

An unprocessed spatial dataset as output by a dataset loader.

Example file: resources_test/spatialsimbench_mobnew/dataset_sp.h5ad

Description:

This dataset contains raw counts and metadata as output by a dataset loader.

The format of this file is derived from the CELLxGENE schema v4.0.0.

Component type: Metric

A metric.

Arguments:

Name	Type	Description
--input_spatial_dataset	file	An unprocessed spatial dataset as output by a dataset loader.
--input_singlecell_dataset	file	An unprocessed single-cell dataset as output by a dataset loader.
--input_simulated_dataset	file	The solution for the test data.
--output	file	(Output) File indicating the score of a metric.

Component type: Control Method

A control method.

Arguments:

Name	Type	Description
--input	file	(Optional) An unprocessed spatial dataset as output by a dataset loader.
--output	file	(Output) The solution for the test data.

Component type: Method

A method.

Arguments:

Name	Type	Description
--input	file	(Optional) An unprocessed spatial dataset as output by a dataset loader.
--output	file	(Output) The solution for the test data.

File format: Score

File indicating the score of a metric.

Example file: resources_test/spatialsimbench_mobnew/score.h5ad

File format: Solution

The solution for the test data

Example file: resources_test/spatialsimbench_mobnew/simulated_dataset_processed.h5ad