DSA Annotation¶
Digital Slide Archive (DSA) is an open-source web application where users can annotate regional and point annotations on the high power slide viewer. Luna Pathology CLIs pull the different annotation types from DSA, and save the annotations in GeoJSON format along with metadata. In this notebook, we will review:
- Project setup on DSA
- Create annotations on DSA
- Run regional annotation ETL
- Run point annotation ETL
DSA provides an excellent video tutorial that covers platform features. For the first two points on DSA, the information below is an abridged version of the tutorial for your reference.
Project setup on DSA¶
Digital Slide Archive (DSA) is a platform that provides the ability to store, manage, visualize and annotate large imaging data sets. The DSA consists of an interface to visualize slides and manage annotations (HistomicsUI), and a web-server that provides a rich API and data management tools (using Girder). This system can:
- Organize images from a variety of assetstores, such as local files systems and S3.
- Provide user access controls.
- Image annotation and review.
HistomicsUI is a web-based application for examining, annotating, and processing histology images to extract both low and high level features (e.g. cellular structure, feature types). Concepts
- Collections correspond to a project. Collections are at the top level objects in the data organization hierarchy.
- Folders help organize slides under a project. e.g. hne_slides
- Items correspond to a slide. An item can have metadata, annotations and files associated with it.
- Annotation is a single rectangle, point, or polygon
- Annotation Document is a set of annotations, created by the pathologist.
- Annotation Style is a predefined set of labels (morphology like tumor, stroma, necrosis etc) and colors.
Create a collection for your project.
Your images can be organized in a folder.
In this example, we have a pathology-tutorial collection with slides folder where we organized the images.
Create annotations on DSA¶
Please see this video tutorial for creating and viewing annotations. The information below is an abridged version of the tutorial for your reference.
1. To navigate to HistomicsUI, go to the Actions → Open in HistomicsUI on the upper right side. HistomicsUI will open a new tab in your browser.
2. Create an annotation document
- Click on the + New button on the Annotation panel. This will bring up a Create annotation modal.
- Name you annotation document regional or point. These are the two types of annotations we support. The annotation document name will be used in the ETL, it is important to standardize your document names so the ETL can download all documents for the annotation type.
- Optionally add a description, then click save.
3. Create annotations
- Select a label (e.g. regional_tumor)
- Click on Point or Polygon. When an annotation shape is highlighted, then your cursor on the slide area will look like a +
- For Point annotation, zoom to an appropriate magnification and click on the cell. The annotation will appear as a circle.
- For Polygon annotation, click and drag your mouse. As you drag the area will be highlighted. Try to meet the starting point, or double click to close the polygon.
Note: Using standardized annotation styles is recommended. A uniform annotation style json can be created and shared among the pathologists making annotations.
Run regional annotation ETL¶
In [1]:
Copied!
import os
HOME = os.environ['HOME']
import os
HOME = os.environ['HOME']
Once you have created annotations on DSA, we can run the annotation ETL CLI! This ETL will download the annotations, convert them to GeoJSON format, and create a parquet table to make the annotations and metadata queryable.
For details of the data and app configuration, please refer to the example configurations.
First, let's look at the CLI arguments, by running --help
In [2]:
Copied!
!dsa_annotation --help
!dsa_annotation --help
2023-04-04 20:25:04,972 - INFO - root - Initalized logger, log file at: luna.log
Usage: dsa_annotation [OPTIONS] INPUT_DSA_ENDPOINT
A cli tool
Inputs:
input_dsa_endpoint: Path to the DSA endpoint like http://localhost:8080/dsa/api/v1
Outputs:
slide_annotation_dataset
Example:
export DSA_USERNAME=username
export DSA_PASSWORD=password
dsa_annotation_etl http://localhost:8080/dsa/api/v1
--collection-name tcga-data
--annotation-name TumorVsOther
-o /data/annotations/
Options:
-o, --output_dir TEXT path to output directory to save results
-c, --collection-name TEXT name of the collection to pull data from in
DSA
-a, --annotation-name TEXT name of the annotations to pull from DSA (same
annotation name for all slides)
-u, --username TEXT DSA username, can be inferred from
DSA_USERNAME
-p, --password TEXT DSA password, should be inferred from
DSA_PASSWORD
-nc, --num_cores INTEGER Number of cores to use
-m, --method_param_path TEXT path to a metadata json/yaml file with method
parameters to reproduce results
--help Show this message and exit.
In [3]:
Copied!
# ingest annotations
!dsa_annotation http://girder:8080/api/v1 \
--output_dir ../dsa_annotations \
--collection-name 'TCGA collection' \
--annotation-name ov_regional \
--num_cores 1 \
--username admin --password password1
# ingest annotations
!dsa_annotation http://girder:8080/api/v1 \
--output_dir ../dsa_annotations \
--collection-name 'TCGA collection' \
--annotation-name ov_regional \
--num_cores 1 \
--username admin --password password1
2023-04-04 20:25:06,400 - INFO - root - Initalized logger, log file at: luna.log
2023-04-04 20:25:06,403 - INFO - luna.common.utils - Started CLI Runner wtih <function dsa_annotation_etl at 0x7fac766f4f70>
2023-04-04 20:25:06,405 - INFO - luna.common.utils - Validating params...
2023-04-04 20:25:06,408 - INFO - luna.common.utils - -> Set input_dsa_endpoint (<class 'str'>) = http://girder:8080/api/v1
2023-04-04 20:25:06,410 - INFO - luna.common.utils - -> Set collection_name (<class 'str'>) = TCGA collection
2023-04-04 20:25:06,413 - INFO - luna.common.utils - -> Set annotation_name (<class 'str'>) = ov_regional
2023-04-04 20:25:06,415 - INFO - luna.common.utils - -> Set num_cores (<class 'int'>) = 1
2023-04-04 20:25:06,417 - INFO - luna.common.utils - -> Set username (<class 'str'>) = *****
2023-04-04 20:25:06,418 - INFO - luna.common.utils - -> Set password (<class 'str'>) = *****
2023-04-04 20:25:06,420 - INFO - luna.common.utils - -> Set output_dir (<class 'str'>) = ../dsa_annotations
2023-04-04 20:25:06,422 - INFO - luna.common.utils - Expanding inputs...
2023-04-04 20:25:06,424 - INFO - luna.common.utils - Attempting to read metadata at http://girder:8080/api/v1/metadata.yml
2023-04-04 20:25:06,427 - INFO - luna.common.utils - Full segment key set: {}
2023-04-04 20:25:06,430 - INFO - luna.common.utils - ------------------------------------------------------------
2023-04-04 20:25:06,430 - INFO - luna.common.utils - Starting transform::dsa_annotation_etl
2023-04-04 20:25:06,430 - INFO - luna.common.utils - ------------------------------------------------------------
2023-04-04 20:25:06,679 - INFO - luna.pathology.dsa.dsa_api_handler - Successfully connected to DSA
2023-04-04 20:25:06,702 - INFO - luna.pathology.dsa.dsa_api_handler - Found collection id=642b13a033dd668f85bbc1ee for collection=TCGA collection
2023-04-04 20:25:06,721 - INFO - luna.pathology.dsa.dsa_api_handler - Found 5 slides!
2023-04-04 20:25:07,072 - INFO - distributed.http.proxy - To route to workers diagnostics web server please install jupyter-server-proxy: python -m pip install jupyter-server-proxy
2023-04-04 20:25:07,106 - INFO - distributed.scheduler - State start
2023-04-04 20:25:07,113 - INFO - distributed.scheduler - Scheduler at: tcp://172.21.0.5:45157
2023-04-04 20:25:07,115 - INFO - distributed.scheduler - dashboard at: 172.21.0.5:8787
2023-04-04 20:25:07,126 - INFO - distributed.nanny - Start Nanny at: 'tcp://172.21.0.5:32949'
2023-04-04 20:25:08,089 - INFO - root - Initalized logger, log file at: luna.log
2023-04-04 20:25:08,111 - INFO - distributed.worker - Start worker at: tcp://172.21.0.5:43523
2023-04-04 20:25:08,114 - INFO - distributed.worker - Listening to: tcp://172.21.0.5:43523
2023-04-04 20:25:08,115 - INFO - distributed.worker - Worker name: 0
2023-04-04 20:25:08,117 - INFO - distributed.worker - dashboard at: 172.21.0.5:43275
2023-04-04 20:25:08,118 - INFO - distributed.worker - Waiting to connect to: tcp://172.21.0.5:45157
2023-04-04 20:25:08,120 - INFO - distributed.worker - -------------------------------------------------
2023-04-04 20:25:08,121 - INFO - distributed.worker - Threads: 2
2023-04-04 20:25:08,123 - INFO - distributed.worker - Memory: 20.29 GiB
2023-04-04 20:25:08,124 - INFO - distributed.worker - Local Directory: /tmp/dask-worker-space/worker-913nas69
2023-04-04 20:25:08,125 - INFO - distributed.worker - -------------------------------------------------
2023-04-04 20:25:08,139 - INFO - distributed.scheduler - Register worker <WorkerState 'tcp://172.21.0.5:43523', name: 0, status: init, memory: 0, processing: 0>
2023-04-04 20:25:08,152 - INFO - distributed.scheduler - Starting worker compute stream, tcp://172.21.0.5:43523
2023-04-04 20:25:08,152 - INFO - distributed.worker - Registered to: tcp://172.21.0.5:45157
2023-04-04 20:25:08,154 - INFO - distributed.core - Starting established connection to tcp://172.21.0.5:49888
2023-04-04 20:25:08,156 - INFO - distributed.worker - -------------------------------------------------
2023-04-04 20:25:08,159 - INFO - distributed.core - Starting established connection to tcp://172.21.0.5:45157
2023-04-04 20:25:08,166 - INFO - distributed.scheduler - Receive client connection: Client-c9dc76fc-d326-11ed-8322-0242ac150005
2023-04-04 20:25:08,168 - INFO - distributed.core - Starting established connection to tcp://172.21.0.5:49900
2023-04-04 20:25:08,171 - INFO - dsa_annotation_etl - Dashboard: http://172.21.0.5:8787/status
2023-04-04 20:25:08,273 - INFO - dsa_annotation_etl - Trying to process annotation for slide_id=01OV002-bd8cdc70-3d46-40ae-99c4-90ef77, item_id=642b13a033dd668f85bbc1f0
2023-04-04 20:25:08,283 - INFO - dsa_annotation_etl - Trying to process annotation for slide_id=01OV002-ed65cf94-8bc6-492b-9149-adc16f, item_id=642b13a933dd668f85bbc1fc
2023-04-04 20:25:08,296 - INFO - luna.pathology.dsa.dsa_api_handler - Found 6 total annotations: {'bitmask', 'ov_regional', 'purple_score_heatmap'}
2023-04-04 20:25:08,301 - INFO - luna.pathology.dsa.dsa_api_handler - Found 1 total annotations: {'ov_regional'}
2023-04-04 20:25:08,311 - INFO - luna.pathology.dsa.dsa_api_handler - Found an annotation called ov_regional!!!!
2023-04-04 20:25:08,315 - INFO - luna.pathology.dsa.dsa_api_handler - Found an annotation called ov_regional!!!!
2023-04-04 20:25:08,382 - INFO - dsa_annotation_etl - About to turn 6 geometric annotations into a geojson!
2023-04-04 20:25:08,384 - INFO - dsa_annotation_etl - About to turn 5 geometric annotations into a geojson!
2023-04-04 20:25:08,389 - INFO - dsa_annotation_etl - Created geometry POLYGON ((28211 42225, 28328 42546, 2858...
2023-04-04 20:25:08,391 - INFO - dsa_annotation_etl - Created geometry POLYGON ((24143 4477, 23829 4477, 23663 ...
2023-04-04 20:25:08,393 - INFO - dsa_annotation_etl - Created geometry POLYGON ((32252 37097, 32252 37001, 3222...
2023-04-04 20:25:08,396 - INFO - dsa_annotation_etl - Created geometry POLYGON ((19647 5125, 19691 5370, 19735 ...
2023-04-04 20:25:08,398 - INFO - dsa_annotation_etl - Created geometry POLYGON ((24180 18972, 24136 18978, 2406...
2023-04-04 20:25:08,401 - INFO - dsa_annotation_etl - Created geometry POLYGON ((10388 30395, 10374 30328, 1033...
2023-04-04 20:25:08,403 - INFO - dsa_annotation_etl - Created geometry POLYGON ((30133 26411, 30073 26465, 3000...
2023-04-04 20:25:08,405 - INFO - dsa_annotation_etl - Created geometry POLYGON ((15745 10073, 15904 10059, 1606...
2023-04-04 20:25:08,407 - INFO - dsa_annotation_etl - Created geometry POLYGON ((16525 27088, 16597 26992, 1670...
2023-04-04 20:25:08,409 - INFO - dsa_annotation_etl - Created geometry POLYGON ((12928 29472, 12872 29504, 1280...
2023-04-04 20:25:08,412 - INFO - dsa_annotation_etl - Created geometry POLYGON ((23389 18929, 23435 19021, 2346...
2023-04-04 20:25:08,414 - INFO - dsa_annotation_etl - Checking geojson, errors with geojson FeatureCollection: []
2023-04-04 20:25:08,415 - INFO - dsa_annotation_etl - Checking geojson, errors with geojson FeatureCollection: []
2023-04-04 20:25:08,437 - INFO - dsa_annotation_etl - Trying to process annotation for slide_id=01OV008-308ad404-7079-4ff8-8232-12ee2e, item_id=642b13bb33dd668f85bbc212
2023-04-04 20:25:08,450 - INFO - dsa_annotation_etl - Trying to process annotation for slide_id=01OV007-9b90eb78-2f50-4aeb-b010-d642f9, item_id=642b13b233dd668f85bbc206
2023-04-04 20:25:08,464 - INFO - luna.pathology.dsa.dsa_api_handler - Found 1 total annotations: {'ov_regional'}
2023-04-04 20:25:08,468 - INFO - luna.pathology.dsa.dsa_api_handler - Found 1 total annotations: {'ov_regional'}
2023-04-04 20:25:08,472 - INFO - luna.pathology.dsa.dsa_api_handler - Found an annotation called ov_regional!!!!
2023-04-04 20:25:08,476 - INFO - luna.pathology.dsa.dsa_api_handler - Found an annotation called ov_regional!!!!
2023-04-04 20:25:08,591 - INFO - dsa_annotation_etl - About to turn 7 geometric annotations into a geojson!
2023-04-04 20:25:08,591 - INFO - dsa_annotation_etl - About to turn 7 geometric annotations into a geojson!
2023-04-04 20:25:08,595 - INFO - dsa_annotation_etl - Created geometry POLYGON ((26095 7561, 25925 7569, 25819 ...
2023-04-04 20:25:08,599 - INFO - dsa_annotation_etl - Created geometry POLYGON ((13850 41438, 13762 41478, 1368...
2023-04-04 20:25:08,601 - INFO - dsa_annotation_etl - Created geometry POLYGON ((21846 9151, 21863 9224, 21863 ...
2023-04-04 20:25:08,603 - INFO - dsa_annotation_etl - Created geometry POLYGON ((88952 17667, 88912 17647, 8884...
2023-04-04 20:25:08,605 - INFO - dsa_annotation_etl - Created geometry POLYGON ((38411 18675, 38419 18637, 3843...
2023-04-04 20:25:08,607 - INFO - dsa_annotation_etl - Created geometry POLYGON ((86674 27291, 86674 27326, 8666...
2023-04-04 20:25:08,610 - INFO - dsa_annotation_etl - Created geometry POLYGON ((38390 23685, 38362 23701, 3832...
2023-04-04 20:25:08,612 - INFO - dsa_annotation_etl - Created geometry POLYGON ((96999 26604, 97030 26610, 9706...
2023-04-04 20:25:08,614 - INFO - dsa_annotation_etl - Created geometry POLYGON ((29241 24267, 28934 24536, 2850...
2023-04-04 20:25:08,616 - INFO - dsa_annotation_etl - Created geometry POLYGON ((15605 16231, 15605 16285, 1560...
2023-04-04 20:25:08,618 - INFO - dsa_annotation_etl - Created geometry POLYGON ((15784 15684, 15840 15673, 1590...
2023-04-04 20:25:08,620 - INFO - dsa_annotation_etl - Created geometry POLYGON ((12688 16850, 12721 16888, 1276...
2023-04-04 20:25:08,622 - INFO - dsa_annotation_etl - Created geometry POLYGON ((24707 16018, 24700 15990, 2463...
2023-04-04 20:25:08,624 - INFO - dsa_annotation_etl - Created geometry POLYGON ((7039 14211, 7015 14211, 6963 1...
2023-04-04 20:25:08,626 - INFO - dsa_annotation_etl - Checking geojson, errors with geojson FeatureCollection: []
2023-04-04 20:25:08,627 - INFO - dsa_annotation_etl - Checking geojson, errors with geojson FeatureCollection: []
2023-04-04 20:25:08,657 - INFO - dsa_annotation_etl - Trying to process annotation for slide_id=01OV008-7579323e-2fae-43a9-b00f-a15c28, item_id=642b13c233dd668f85bbc21e
2023-04-04 20:25:08,671 - INFO - luna.pathology.dsa.dsa_api_handler - Found 1 total annotations: {'ov_regional'}
2023-04-04 20:25:08,677 - INFO - luna.pathology.dsa.dsa_api_handler - Found an annotation called ov_regional!!!!
2023-04-04 20:25:08,711 - INFO - dsa_annotation_etl - About to turn 7 geometric annotations into a geojson!
2023-04-04 20:25:08,715 - INFO - dsa_annotation_etl - Created geometry POLYGON ((25997 19938, 25965 19923, 2593...
2023-04-04 20:25:08,717 - INFO - dsa_annotation_etl - Created geometry POLYGON ((14583 17451, 14511 17460, 1438...
2023-04-04 20:25:08,719 - INFO - dsa_annotation_etl - Created geometry POLYGON ((20485 14427, 20513 14431, 2054...
2023-04-04 20:25:08,722 - INFO - dsa_annotation_etl - Created geometry POLYGON ((16776 11851, 16766 11812, 1675...
2023-04-04 20:25:08,724 - INFO - dsa_annotation_etl - Created geometry POLYGON ((12443 26686, 12462 26654, 1249...
2023-04-04 20:25:08,726 - INFO - dsa_annotation_etl - Created geometry POLYGON ((22006 13625, 21967 13621, 2191...
2023-04-04 20:25:08,728 - INFO - dsa_annotation_etl - Created geometry POLYGON ((20741 13671, 20725 13651, 2067...
2023-04-04 20:25:08,730 - INFO - dsa_annotation_etl - Checking geojson, errors with geojson FeatureCollection: []
2023-04-04 20:25:08,755 - INFO - distributed.nanny - Closing Nanny at 'tcp://172.21.0.5:32949'. Reason: nanny-close
2023-04-04 20:25:08,757 - INFO - distributed.nanny - Nanny asking worker to close. Reason: nanny-close
2023-04-04 20:25:08,760 - INFO - distributed.worker - Stopping worker at tcp://172.21.0.5:43523. Reason: nanny-close
2023-04-04 20:25:08,763 - INFO - distributed.core - Received 'close-stream' from tcp://172.21.0.5:49888; closing.
2023-04-04 20:25:08,764 - INFO - distributed.core - Connection to tcp://172.21.0.5:45157 has been closed.
2023-04-04 20:25:08,765 - INFO - distributed.scheduler - Remove worker <WorkerState 'tcp://172.21.0.5:43523', name: 0, status: closing, memory: 0, processing: 0>
2023-04-04 20:25:08,768 - INFO - distributed.core - Removing comms to tcp://172.21.0.5:43523
2023-04-04 20:25:08,770 - INFO - distributed.scheduler - Lost all workers
2023-04-04 20:25:08,950 - INFO - distributed.scheduler - Scheduler closing...
2023-04-04 20:25:08,953 - INFO - distributed.scheduler - Scheduler closing all comms
_id ... annotation_name
slide_id ...
01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 642b13a033dd668f85bbc1f0 ... ov_regional
01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 642b13a033dd668f85bbc1f0 ... ov_regional
01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 642b13a033dd668f85bbc1f0 ... ov_regional
01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 642b13a033dd668f85bbc1f0 ... ov_regional
01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 642b13a033dd668f85bbc1f0 ... ov_regional
01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 642b13a033dd668f85bbc1f0 ... ov_regional
01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 642b13a033dd668f85bbc1f0 ... ov_regional
01OV002-ed65cf94-8bc6-492b-9149-adc16f 642b13a933dd668f85bbc1fc ... ov_regional
01OV002-ed65cf94-8bc6-492b-9149-adc16f 642b13a933dd668f85bbc1fc ... ov_regional
01OV002-ed65cf94-8bc6-492b-9149-adc16f 642b13a933dd668f85bbc1fc ... ov_regional
01OV002-ed65cf94-8bc6-492b-9149-adc16f 642b13a933dd668f85bbc1fc ... ov_regional
01OV002-ed65cf94-8bc6-492b-9149-adc16f 642b13a933dd668f85bbc1fc ... ov_regional
01OV002-ed65cf94-8bc6-492b-9149-adc16f 642b13a933dd668f85bbc1fc ... ov_regional
01OV007-9b90eb78-2f50-4aeb-b010-d642f9 642b13b233dd668f85bbc206 ... ov_regional
01OV007-9b90eb78-2f50-4aeb-b010-d642f9 642b13b233dd668f85bbc206 ... ov_regional
01OV007-9b90eb78-2f50-4aeb-b010-d642f9 642b13b233dd668f85bbc206 ... ov_regional
01OV007-9b90eb78-2f50-4aeb-b010-d642f9 642b13b233dd668f85bbc206 ... ov_regional
01OV007-9b90eb78-2f50-4aeb-b010-d642f9 642b13b233dd668f85bbc206 ... ov_regional
01OV007-9b90eb78-2f50-4aeb-b010-d642f9 642b13b233dd668f85bbc206 ... ov_regional
01OV007-9b90eb78-2f50-4aeb-b010-d642f9 642b13b233dd668f85bbc206 ... ov_regional
01OV007-9b90eb78-2f50-4aeb-b010-d642f9 642b13b233dd668f85bbc206 ... ov_regional
01OV008-308ad404-7079-4ff8-8232-12ee2e 642b13bb33dd668f85bbc212 ... ov_regional
01OV008-308ad404-7079-4ff8-8232-12ee2e 642b13bb33dd668f85bbc212 ... ov_regional
01OV008-308ad404-7079-4ff8-8232-12ee2e 642b13bb33dd668f85bbc212 ... ov_regional
01OV008-308ad404-7079-4ff8-8232-12ee2e 642b13bb33dd668f85bbc212 ... ov_regional
01OV008-308ad404-7079-4ff8-8232-12ee2e 642b13bb33dd668f85bbc212 ... ov_regional
01OV008-308ad404-7079-4ff8-8232-12ee2e 642b13bb33dd668f85bbc212 ... ov_regional
01OV008-308ad404-7079-4ff8-8232-12ee2e 642b13bb33dd668f85bbc212 ... ov_regional
01OV008-308ad404-7079-4ff8-8232-12ee2e 642b13bb33dd668f85bbc212 ... ov_regional
01OV008-7579323e-2fae-43a9-b00f-a15c28 642b13c233dd668f85bbc21e ... ov_regional
01OV008-7579323e-2fae-43a9-b00f-a15c28 642b13c233dd668f85bbc21e ... ov_regional
01OV008-7579323e-2fae-43a9-b00f-a15c28 642b13c233dd668f85bbc21e ... ov_regional
01OV008-7579323e-2fae-43a9-b00f-a15c28 642b13c233dd668f85bbc21e ... ov_regional
01OV008-7579323e-2fae-43a9-b00f-a15c28 642b13c233dd668f85bbc21e ... ov_regional
01OV008-7579323e-2fae-43a9-b00f-a15c28 642b13c233dd668f85bbc21e ... ov_regional
01OV008-7579323e-2fae-43a9-b00f-a15c28 642b13c233dd668f85bbc21e ... ov_regional
01OV008-7579323e-2fae-43a9-b00f-a15c28 642b13c233dd668f85bbc21e ... ov_regional
[37 rows x 40 columns]
2023-04-04 20:25:09,036 - INFO - dsa_annotation_etl - Created 5 geojsons, 0 points, and 32 polygons
2023-04-04 20:25:09,220 - INFO - luna.common.utils - Code block 'transform::dsa_annotation_etl' took: 2.787281731958501s
2023-04-04 20:25:09,221 - INFO - luna.common.utils - ------------------------------------------------------------
2023-04-04 20:25:09,221 - INFO - luna.common.utils - Done with transform, running post-transform functions...
2023-04-04 20:25:09,221 - INFO - luna.common.utils - ------------------------------------------------------------
2023-04-04 20:25:09,226 - INFO - luna.common.utils - Done.
In [4]:
Copied!
# metadata, geojson, parquet table output
!ls -lh ../dsa_annotations/
# metadata, geojson, parquet table output
!ls -lh ../dsa_annotations/
total 156K -rw-r--r-- 1 limr limr 14K Apr 4 20:25 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77.annotation.geojson -rw-r--r-- 1 limr limr 11K Apr 4 20:25 01OV002-ed65cf94-8bc6-492b-9149-adc16f.annotation.geojson -rw-r--r-- 1 limr limr 14K Apr 4 20:25 01OV007-9b90eb78-2f50-4aeb-b010-d642f9.annotation.geojson -rw-r--r-- 1 limr limr 15K Apr 4 20:25 01OV008-308ad404-7079-4ff8-8232-12ee2e.annotation.geojson -rw-r--r-- 1 limr limr 17K Apr 4 20:25 01OV008-7579323e-2fae-43a9-b00f-a15c28.annotation.geojson drwxr-xr-x 4 limr limr 128 Apr 4 18:48 bitmask drwxr-xr-x 4 limr limr 128 Apr 4 20:20 heatmap -rw-r--r-- 1 limr limr 320 Apr 4 20:25 metadata.yml drwxr-xr-x 4 limr limr 128 Apr 4 20:01 quppath -rw-r--r-- 1 limr limr 71K Apr 4 20:25 'slide_annotation_dataset_TCGA collection_ov_regional.parquet' drwxr-xr-x 4 limr limr 128 Apr 4 18:48 stardist_cell drwxr-xr-x 4 limr limr 128 Apr 4 18:48 stardist_polygon
Annotations are saved in a parquet format, where 1 row represents an annotation element.
We collect metadata about the annotation such as created timestamp and user. Note that different annotation types (point, regional) can be ingested using the same CLI
In [5]:
Copied!
# check annotation metadata table
import pyarrow.parquet as pq
annotation_table = pq.read_table(r'../dsa_annotations/slide_annotation_dataset_TCGA collection_ov_regional.parquet').to_pandas()
print(annotation_table.columns)
annotation_table
# check annotation metadata table
import pyarrow.parquet as pq
annotation_table = pq.read_table(r'../dsa_annotations/slide_annotation_dataset_TCGA collection_ov_regional.parquet').to_pandas()
print(annotation_table.columns)
annotation_table
Index(['_id', 'baseParentId', 'baseParentType', 'created', 'creatorId',
'description', 'folderId', 'largeImage', 'lowerName', 'name', 'size',
'updated', 'annotation_girder_id', '_modelType', '_version',
'createdannotation', 'creatorIdannotation', 'public',
'updatedannotation', 'updatedId', 'groups', 'element_count',
'element_details', 'annidx', 'elementidx', 'element_girder_id', 'type',
'group_name', 'label', 'color', 'xmin', 'xmax', 'ymin', 'ymax',
'bbox_area', 'x_coords', 'y_coords', 'slide_geojson', 'collection_name',
'annotation_name'],
dtype='object')
Out[5]:
| _id | baseParentId | baseParentType | created | creatorId | description | folderId | largeImage | lowerName | name | ... | xmin | xmax | ymin | ymax | bbox_area | x_coords | y_coords | slide_geojson | collection_name | annotation_name | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| slide_id | |||||||||||||||||||||
| 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 | 642b13a033dd668f85bbc1f0 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:57:52.704000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13a833dd668f85bbc1f2', 'source... | 01ov002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | ... | 25250.0 | 28661.0 | 40529.0 | 44372.0 | 13108473.0 | [28211, 28328, 28587, 28630, 28655, 28661, 286... | [42225, 42546, 43126, 43261, 43379, 43607, 437... | None | TCGA collection | ov_regional | |
| 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 | 642b13a033dd668f85bbc1f0 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:57:52.704000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13a833dd668f85bbc1f2', 'source... | 01ov002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | ... | 31532.0 | 33932.0 | 35713.0 | 39793.0 | 9792000.0 | [32252, 32252, 32220, 32164, 32140, 32108, 320... | [37097, 37001, 36897, 36745, 36713, 36689, 366... | None | TCGA collection | ov_regional | |
| 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 | 642b13a033dd668f85bbc1f0 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:57:52.704000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13a833dd668f85bbc1f2', 'source... | 01ov002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | ... | 23557.0 | 25542.0 | 18922.0 | 21735.0 | 5583805.0 | [24180, 24136, 24063, 23980, 23874, 23813, 237... | [18972, 18978, 19017, 19078, 19161, 19217, 192... | None | TCGA collection | ov_regional | |
| 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 | 642b13a033dd668f85bbc1f0 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:57:52.704000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13a833dd668f85bbc1f2', 'source... | 01ov002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | ... | 26951.0 | 30651.0 | 23202.0 | 26990.0 | 14015600.0 | [30133, 30073, 30005, 29951, 29904, 29857, 298... | [26411, 26465, 26506, 26532, 26553, 26566, 265... | None | TCGA collection | ov_regional | |
| 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 | 642b13a033dd668f85bbc1f0 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:57:52.704000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13a833dd668f85bbc1f2', 'source... | 01ov002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | ... | 13533.0 | 17365.0 | 26776.0 | 29672.0 | 11097472.0 | [16525, 16597, 16701, 16781, 16845, 16901, 169... | [27088, 26992, 26896, 26840, 26816, 26808, 267... | None | TCGA collection | ov_regional | |
| 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 | 642b13a033dd668f85bbc1f0 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:57:52.704000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13a833dd668f85bbc1f2', 'source... | 01ov002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | ... | 21459.0 | 23500.0 | 16457.0 | 19848.0 | 6921031.0 | [23389, 23435, 23463, 23481, 23500, 23500, 234... | [18929, 19021, 19103, 19186, 19250, 19425, 194... | None | TCGA collection | ov_regional | |
| 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77 | 642b13a033dd668f85bbc1f0 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:57:52.704000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13a833dd668f85bbc1f2', 'source... | 01ov002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | 01OV002-bd8cdc70-3d46-40ae-99c4-90ef77.svs | ... | NaN | NaN | NaN | NaN | NaN | None | None | ../dsa_annotations/01OV002-bd8cdc70-3d46-40ae-... | TCGA collection | ov_regional | |
| 01OV002-ed65cf94-8bc6-492b-9149-adc16f | 642b13a933dd668f85bbc1fc | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:01.930000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13b133dd668f85bbc1fe', 'source... | 01ov002-ed65cf94-8bc6-492b-9149-adc16f.svs | 01OV002-ed65cf94-8bc6-492b-9149-adc16f.svs | ... | 20217.0 | 25783.0 | 4477.0 | 8874.0 | 24473702.0 | [24143, 23829, 23663, 23611, 23532, 23436, 233... | [4477, 4477, 4503, 4521, 4556, 4591, 4608, 464... | None | TCGA collection | ov_regional | |
| 01OV002-ed65cf94-8bc6-492b-9149-adc16f | 642b13a933dd668f85bbc1fc | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:01.930000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13b133dd668f85bbc1fe', 'source... | 01ov002-ed65cf94-8bc6-492b-9149-adc16f.svs | 01OV002-ed65cf94-8bc6-492b-9149-adc16f.svs | ... | 14683.0 | 19778.0 | 4925.0 | 8728.0 | 19376285.0 | [19647, 19691, 19735, 19778, 19778, 19770, 197... | [5125, 5370, 5605, 5876, 6076, 6190, 6268, 634... | None | TCGA collection | ov_regional | |
| 01OV002-ed65cf94-8bc6-492b-9149-adc16f | 642b13a933dd668f85bbc1fc | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:01.930000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13b133dd668f85bbc1fe', 'source... | 01ov002-ed65cf94-8bc6-492b-9149-adc16f.svs | 01OV002-ed65cf94-8bc6-492b-9149-adc16f.svs | ... | 10240.0 | 12487.0 | 28794.0 | 34122.0 | 11972016.0 | [10388, 10374, 10334, 10294, 10253, 10240, 102... | [30395, 30328, 30207, 30059, 29938, 29830, 296... | None | TCGA collection | ov_regional | |
| 01OV002-ed65cf94-8bc6-492b-9149-adc16f | 642b13a933dd668f85bbc1fc | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:01.930000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13b133dd668f85bbc1fe', 'source... | 01ov002-ed65cf94-8bc6-492b-9149-adc16f.svs | 01OV002-ed65cf94-8bc6-492b-9149-adc16f.svs | ... | 11578.0 | 17692.0 | 10030.0 | 16216.0 | 37821204.0 | [15745, 15904, 16062, 16235, 16351, 16452, 165... | [10073, 10059, 10030, 10030, 10073, 10117, 101... | None | TCGA collection | ov_regional | |
| 01OV002-ed65cf94-8bc6-492b-9149-adc16f | 642b13a933dd668f85bbc1fc | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:01.930000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13b133dd668f85bbc1fe', 'source... | 01ov002-ed65cf94-8bc6-492b-9149-adc16f.svs | 01OV002-ed65cf94-8bc6-492b-9149-adc16f.svs | ... | 12416.0 | 15544.0 | 29416.0 | 33936.0 | 14138560.0 | [12928, 12872, 12808, 12776, 12728, 12704, 126... | [29472, 29504, 29568, 29608, 29672, 29712, 298... | None | TCGA collection | ov_regional | |
| 01OV002-ed65cf94-8bc6-492b-9149-adc16f | 642b13a933dd668f85bbc1fc | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:01.930000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13b133dd668f85bbc1fe', 'source... | 01ov002-ed65cf94-8bc6-492b-9149-adc16f.svs | 01OV002-ed65cf94-8bc6-492b-9149-adc16f.svs | ... | NaN | NaN | NaN | NaN | NaN | None | None | ../dsa_annotations/01OV002-ed65cf94-8bc6-492b-... | TCGA collection | ov_regional | |
| 01OV007-9b90eb78-2f50-4aeb-b010-d642f9 | 642b13b233dd668f85bbc206 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:10.952000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13ba33dd668f85bbc208', 'source... | 01ov007-9b90eb78-2f50-4aeb-b010-d642f9.svs | 01OV007-9b90eb78-2f50-4aeb-b010-d642f9.svs | ... | 6938.0 | 14698.0 | 38558.0 | 42478.0 | 30419200.0 | [13850, 13762, 13682, 13626, 13570, 13490, 134... | [41438, 41478, 41518, 41542, 41582, 41614, 416... | None | TCGA collection | ov_regional | |
| 01OV007-9b90eb78-2f50-4aeb-b010-d642f9 | 642b13b233dd668f85bbc206 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:10.952000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13ba33dd668f85bbc208', 'source... | 01ov007-9b90eb78-2f50-4aeb-b010-d642f9.svs | 01OV007-9b90eb78-2f50-4aeb-b010-d642f9.svs | ... | 87844.0 | 90028.0 | 17627.0 | 20103.0 | 5407584.0 | [88952, 88912, 88848, 88792, 88752, 88704, 886... | [17667, 17647, 17631, 17627, 17627, 17639, 176... | None | TCGA collection | ov_regional | |
| 01OV007-9b90eb78-2f50-4aeb-b010-d642f9 | 642b13b233dd668f85bbc206 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:10.952000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13ba33dd668f85bbc208', 'source... | 01ov007-9b90eb78-2f50-4aeb-b010-d642f9.svs | 01OV007-9b90eb78-2f50-4aeb-b010-d642f9.svs | ... | 85754.0 | 87377.0 | 27279.0 | 29663.0 | 3869232.0 | [86674, 86674, 86668, 86633, 86621, 86598, 865... | [27291, 27326, 27402, 27724, 27771, 27894, 279... | None | TCGA collection | ov_regional | |
| 01OV007-9b90eb78-2f50-4aeb-b010-d642f9 | 642b13b233dd668f85bbc206 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:10.952000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13ba33dd668f85bbc208', 'source... | 01ov007-9b90eb78-2f50-4aeb-b010-d642f9.svs | 01OV007-9b90eb78-2f50-4aeb-b010-d642f9.svs | ... | 96129.0 | 97400.0 | 26598.0 | 28177.0 | 2006909.0 | [96999, 97030, 97061, 97098, 97141, 97178, 972... | [26604, 26610, 26635, 26659, 26696, 26739, 268... | None | TCGA collection | ov_regional | |
| 01OV007-9b90eb78-2f50-4aeb-b010-d642f9 | 642b13b233dd668f85bbc206 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:10.952000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13ba33dd668f85bbc208', 'source... | 01ov007-9b90eb78-2f50-4aeb-b010-d642f9.svs | 01OV007-9b90eb78-2f50-4aeb-b010-d642f9.svs | ... | 13795.0 | 15718.0 | 14948.0 | 17225.0 | 4378671.0 | [15605, 15605, 15605, 15616, 15627, 15632, 156... | [16231, 16285, 16446, 16575, 16661, 16688, 167... | None | TCGA collection | ov_regional | |
| 01OV007-9b90eb78-2f50-4aeb-b010-d642f9 | 642b13b233dd668f85bbc206 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:10.952000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13ba33dd668f85bbc208', 'source... | 01ov007-9b90eb78-2f50-4aeb-b010-d642f9.svs | 01OV007-9b90eb78-2f50-4aeb-b010-d642f9.svs | ... | 12280.0 | 13059.0 | 16566.0 | 18370.0 | 1405316.0 | [12688, 12721, 12764, 12791, 12817, 12844, 128... | [16850, 16888, 16936, 16974, 17017, 17049, 171... | None | TCGA collection | ov_regional | |
| 01OV007-9b90eb78-2f50-4aeb-b010-d642f9 | 642b13b233dd668f85bbc206 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:10.952000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13ba33dd668f85bbc208', 'source... | 01ov007-9b90eb78-2f50-4aeb-b010-d642f9.svs | 01OV007-9b90eb78-2f50-4aeb-b010-d642f9.svs | ... | 6259.0 | 7571.0 | 14199.0 | 15951.0 | 2298624.0 | [7039, 7015, 6963, 6859, 6739, 6679, 6639, 660... | [14211, 14211, 14231, 14275, 14335, 14371, 144... | None | TCGA collection | ov_regional | |
| 01OV007-9b90eb78-2f50-4aeb-b010-d642f9 | 642b13b233dd668f85bbc206 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:10.952000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13ba33dd668f85bbc208', 'source... | 01ov007-9b90eb78-2f50-4aeb-b010-d642f9.svs | 01OV007-9b90eb78-2f50-4aeb-b010-d642f9.svs | ... | NaN | NaN | NaN | NaN | NaN | None | None | ../dsa_annotations/01OV007-9b90eb78-2f50-4aeb-... | TCGA collection | ov_regional | |
| 01OV008-308ad404-7079-4ff8-8232-12ee2e | 642b13bb33dd668f85bbc212 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:19.510000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c133dd668f85bbc214', 'source... | 01ov008-308ad404-7079-4ff8-8232-12ee2e.svs | 01OV008-308ad404-7079-4ff8-8232-12ee2e.svs | ... | 25697.0 | 26787.0 | 7561.0 | 10402.0 | 3096690.0 | [26095, 25925, 25819, 25786, 25754, 25721, 257... | [7561, 7569, 7634, 7683, 7740, 7813, 7862, 794... | None | TCGA collection | ov_regional | |
| 01OV008-308ad404-7079-4ff8-8232-12ee2e | 642b13bb33dd668f85bbc212 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:19.510000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c133dd668f85bbc214', 'source... | 01ov008-308ad404-7079-4ff8-8232-12ee2e.svs | 01OV008-308ad404-7079-4ff8-8232-12ee2e.svs | ... | 20927.0 | 21863.0 | 8532.0 | 11072.0 | 2377440.0 | [21846, 21863, 21863, 21846, 21822, 21806, 217... | [9151, 9224, 9273, 9371, 9452, 9501, 9542, 959... | None | TCGA collection | ov_regional | |
| 01OV008-308ad404-7079-4ff8-8232-12ee2e | 642b13bb33dd668f85bbc212 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:19.510000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c133dd668f85bbc214', 'source... | 01ov008-308ad404-7079-4ff8-8232-12ee2e.svs | 01OV008-308ad404-7079-4ff8-8232-12ee2e.svs | ... | 38127.0 | 38687.0 | 18301.0 | 19409.0 | 620480.0 | [38411, 38419, 38439, 38459, 38469, 38481, 385... | [18675, 18637, 18573, 18533, 18519, 18507, 184... | None | TCGA collection | ov_regional | |
| 01OV008-308ad404-7079-4ff8-8232-12ee2e | 642b13bb33dd668f85bbc212 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:19.510000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c133dd668f85bbc214', 'source... | 01ov008-308ad404-7079-4ff8-8232-12ee2e.svs | 01OV008-308ad404-7079-4ff8-8232-12ee2e.svs | ... | 35419.0 | 38443.0 | 21701.0 | 23747.0 | 6187104.0 | [38390, 38362, 38326, 38229, 38197, 38145, 379... | [23685, 23701, 23712, 23735, 23739, 23747, 237... | None | TCGA collection | ov_regional | |
| 01OV008-308ad404-7079-4ff8-8232-12ee2e | 642b13bb33dd668f85bbc212 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:19.510000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c133dd668f85bbc214', 'source... | 01ov008-308ad404-7079-4ff8-8232-12ee2e.svs | 01OV008-308ad404-7079-4ff8-8232-12ee2e.svs | ... | 21789.0 | 30451.0 | 14502.0 | 26079.0 | 100279974.0 | [29241, 28934, 28506, 27955, 27741, 27557, 274... | [24267, 24536, 24804, 25071, 25190, 25309, 254... | None | TCGA collection | ov_regional | |
| 01OV008-308ad404-7079-4ff8-8232-12ee2e | 642b13bb33dd668f85bbc212 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:19.510000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c133dd668f85bbc214', 'source... | 01ov008-308ad404-7079-4ff8-8232-12ee2e.svs | 01OV008-308ad404-7079-4ff8-8232-12ee2e.svs | ... | 14974.0 | 16345.0 | 15568.0 | 16623.0 | 1446405.0 | [15784, 15840, 15909, 15949, 15973, 15993, 160... | [15684, 15673, 15662, 15658, 15651, 15647, 156... | None | TCGA collection | ov_regional | |
| 01OV008-308ad404-7079-4ff8-8232-12ee2e | 642b13bb33dd668f85bbc212 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:19.510000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c133dd668f85bbc214', 'source... | 01ov008-308ad404-7079-4ff8-8232-12ee2e.svs | 01OV008-308ad404-7079-4ff8-8232-12ee2e.svs | ... | 23333.0 | 24735.0 | 15599.0 | 17067.0 | 2058136.0 | [24707, 24700, 24634, 24544, 24508, 24485, 244... | [16018, 15990, 15821, 15656, 15620, 15607, 155... | None | TCGA collection | ov_regional | |
| 01OV008-308ad404-7079-4ff8-8232-12ee2e | 642b13bb33dd668f85bbc212 | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:19.510000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c133dd668f85bbc214', 'source... | 01ov008-308ad404-7079-4ff8-8232-12ee2e.svs | 01OV008-308ad404-7079-4ff8-8232-12ee2e.svs | ... | NaN | NaN | NaN | NaN | NaN | None | None | ../dsa_annotations/01OV008-308ad404-7079-4ff8-... | TCGA collection | ov_regional | |
| 01OV008-7579323e-2fae-43a9-b00f-a15c28 | 642b13c233dd668f85bbc21e | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:26.586000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c833dd668f85bbc220', 'source... | 01ov008-7579323e-2fae-43a9-b00f-a15c28.svs | 01OV008-7579323e-2fae-43a9-b00f-a15c28.svs | ... | 24679.0 | 28755.0 | 19860.0 | 21427.0 | 6387092.0 | [25997, 25965, 25932, 25876, 25856, 25836, 256... | [19938, 19923, 19899, 19872, 19864, 19860, 198... | None | TCGA collection | ov_regional | |
| 01OV008-7579323e-2fae-43a9-b00f-a15c28 | 642b13c233dd668f85bbc21e | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:26.586000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c833dd668f85bbc220', 'source... | 01ov008-7579323e-2fae-43a9-b00f-a15c28.svs | 01OV008-7579323e-2fae-43a9-b00f-a15c28.svs | ... | 13479.0 | 15095.0 | 17451.0 | 19701.0 | 3636000.0 | [14583, 14511, 14384, 14256, 14074, 14026, 139... | [17451, 17460, 17501, 17551, 17658, 17690, 177... | None | TCGA collection | ov_regional | |
| 01OV008-7579323e-2fae-43a9-b00f-a15c28 | 642b13c233dd668f85bbc21e | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:26.586000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c833dd668f85bbc220', 'source... | 01ov008-7579323e-2fae-43a9-b00f-a15c28.svs | 01OV008-7579323e-2fae-43a9-b00f-a15c28.svs | ... | 18897.0 | 20765.0 | 13531.0 | 15375.0 | 3444592.0 | [20485, 20513, 20549, 20577, 20601, 20621, 206... | [14427, 14431, 14451, 14471, 14495, 14523, 145... | None | TCGA collection | ov_regional | |
| 01OV008-7579323e-2fae-43a9-b00f-a15c28 | 642b13c233dd668f85bbc21e | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:26.586000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c833dd668f85bbc220', 'source... | 01ov008-7579323e-2fae-43a9-b00f-a15c28.svs | 01OV008-7579323e-2fae-43a9-b00f-a15c28.svs | ... | 15986.0 | 17465.0 | 11547.0 | 13601.0 | 3037866.0 | [16776, 16766, 16756, 16730, 16704, 16678, 166... | [11851, 11812, 11772, 11726, 11695, 11656, 116... | None | TCGA collection | ov_regional | |
| 01OV008-7579323e-2fae-43a9-b00f-a15c28 | 642b13c233dd668f85bbc21e | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:26.586000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c833dd668f85bbc220', 'source... | 01ov008-7579323e-2fae-43a9-b00f-a15c28.svs | 01OV008-7579323e-2fae-43a9-b00f-a15c28.svs | ... | 11463.0 | 12733.0 | 26212.0 | 27987.0 | 2254250.0 | [12443, 12462, 12494, 12514, 12542, 12599, 126... | [26686, 26654, 26642, 26637, 26641, 26668, 267... | None | TCGA collection | ov_regional | |
| 01OV008-7579323e-2fae-43a9-b00f-a15c28 | 642b13c233dd668f85bbc21e | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:26.586000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c833dd668f85bbc220', 'source... | 01ov008-7579323e-2fae-43a9-b00f-a15c28.svs | 01OV008-7579323e-2fae-43a9-b00f-a15c28.svs | ... | 21210.0 | 22424.0 | 12780.0 | 13898.0 | 1357252.0 | [22006, 21967, 21911, 21859, 21826, 21821, 218... | [13625, 13621, 13587, 13539, 13492, 13480, 134... | None | TCGA collection | ov_regional | |
| 01OV008-7579323e-2fae-43a9-b00f-a15c28 | 642b13c233dd668f85bbc21e | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:26.586000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c833dd668f85bbc220', 'source... | 01ov008-7579323e-2fae-43a9-b00f-a15c28.svs | 01OV008-7579323e-2fae-43a9-b00f-a15c28.svs | ... | 20037.0 | 21173.0 | 13423.0 | 14415.0 | 1126912.0 | [20741, 20725, 20677, 20621, 20577, 20561, 205... | [13671, 13651, 13615, 13579, 13555, 13539, 135... | None | TCGA collection | ov_regional | |
| 01OV008-7579323e-2fae-43a9-b00f-a15c28 | 642b13c233dd668f85bbc21e | 642b13a033dd668f85bbc1ee | collection | 2023-04-03T17:58:26.586000+00:00 | 642b139d3e6bab4e4d6a30a1 | 642b13a033dd668f85bbc1ef | {'fileId': '642b13c833dd668f85bbc220', 'source... | 01ov008-7579323e-2fae-43a9-b00f-a15c28.svs | 01OV008-7579323e-2fae-43a9-b00f-a15c28.svs | ... | NaN | NaN | NaN | NaN | NaN | None | None | ../dsa_annotations/01OV008-7579323e-2fae-43a9-... | TCGA collection | ov_regional |
37 rows × 40 columns