Organism#

lamindb provides access to the following public organism ontologies through lnschema-bionty:

Here we show how to access and search organism ontologies to standardize new data.

Setup#

!lamin init --storage ./test-organism --schema bionty

✅ saved: User(uid='DzTjkKse', handle='testuser1', name='Test User1', updated_at=2024-01-08 08:53:41 UTC)
✅ saved: Storage(uid='jPwvqvYo', root='/home/runner/work/lamin-usecases/lamin-usecases/docs/test-organism', type='local', updated_at=2024-01-08 08:53:41 UTC, created_by_id=1)

💡 loaded instance: testuser1/test-organism
💡 did not register local instance on hub

import lnschema_bionty as lb
import pandas as pd

Bionty objects#

Let us create a public knowledge accessor with bionty(), which chooses a default public knowledge source from BiontySource. It’s a Bionty object, which you can think about as a less-capable registry:

organism_bt = lb.Organism.bionty()
organism_bt

💡 loaded instance: testuser1/test-organism

Organism
Organism: vertebrates
Source: ensembl, release-110
#terms: 317

📖 Organism.df(): ontology reference table
🔎 Organism.lookup(): autocompletion of terms
🎯 Organism.search(): free text search of terms
✅ Organism.validate(): strictly validate values
🧐 Organism.inspect(): full inspection of values
👽 Organism.standardize(): convert to standardized names
🪜 Organism.diff(): difference between two versions
🔗 Organism.ontology: Pronto.Ontology object

As for registries, you can export the ontology as a DataFrame:

df = organism_bt.df()
df.head()

	scientific_name	division	ontology_id	assembly	assembly_accession	genebuild	variation	microarray	pan_compara	peptide_compara	genome_alignments	other_alignments	core_db	species_id
name
spiny chromis	acanthochromis_polyacanthus	EnsemblVertebrates	NCBITaxon:80966	ASM210954v1	GCA_002109545.1	2018-05-Ensembl/2020-03	N	N	N	Y	Y	Y	acanthochromis_polyacanthus_core_110_1	1
eurasian sparrowhawk	accipiter_nisus	EnsemblVertebrates	NCBITaxon:211598	Accipiter_nisus_ver1.0	GCA_004320145.1	2019-07-Ensembl/2019-09	N	N	N	N	N	Y	accipiter_nisus_core_110_1	1
giant panda	ailuropoda_melanoleuca	EnsemblVertebrates	NCBITaxon:9646	ASM200744v2	GCA_002007445.2	2020-05-Ensembl/2020-06	N	N	N	Y	Y	Y	ailuropoda_melanoleuca_core_110_2	1
yellow-billed parrot	amazona_collaria	EnsemblVertebrates	NCBITaxon:241587	ASM394721v1	GCA_003947215.1	2019-07-Ensembl/2019-09	N	N	N	N	N	Y	amazona_collaria_core_110_1	1
midas cichlid	amphilophus_citrinellus	EnsemblVertebrates	NCBITaxon:61819	Midas_v5	GCA_000751415.1	2018-05-Ensembl/2018-07	N	N	N	Y	Y	Y	amphilophus_citrinellus_core_110_5	1

Unlike registries, you can also export it as a Pronto object via organism_bt.ontology.

Look up terms#

As for registries, terms can be looked up with auto-complete:

lookup = organism_bt.lookup()

The . accessor provides normalized terms (lower case, only contains alphanumeric characters and underscores):

lookup.giant_panda

Organism(name='giant panda', scientific_name='ailuropoda_melanoleuca', division='EnsemblVertebrates', ontology_id='NCBITaxon:9646', assembly='ASM200744v2', assembly_accession='GCA_002007445.2', genebuild='2020-05-Ensembl/2020-06', variation='N', microarray='N', pan_compara='N', peptide_compara='Y', genome_alignments='Y', other_alignments='Y', core_db='ailuropoda_melanoleuca_core_110_2', species_id=1)

To look up the exact original strings, convert the lookup object to dict and use the [] accessor:

lookup_dict = lookup.dict()
lookup_dict["giant panda"]

Organism(name='giant panda', scientific_name='ailuropoda_melanoleuca', division='EnsemblVertebrates', ontology_id='NCBITaxon:9646', assembly='ASM200744v2', assembly_accession='GCA_002007445.2', genebuild='2020-05-Ensembl/2020-06', variation='N', microarray='N', pan_compara='N', peptide_compara='Y', genome_alignments='Y', other_alignments='Y', core_db='ailuropoda_melanoleuca_core_110_2', species_id=1)

By default, the name field is used to generate lookup keys. You can specify another field to look up:

lookup = organism_bt.lookup(organism_bt.scientific_name)

lookup.ailuropoda_melanoleuca

Organism(name='giant panda', scientific_name='ailuropoda_melanoleuca', division='EnsemblVertebrates', ontology_id='NCBITaxon:9646', assembly='ASM200744v2', assembly_accession='GCA_002007445.2', genebuild='2020-05-Ensembl/2020-06', variation='N', microarray='N', pan_compara='N', peptide_compara='Y', genome_alignments='Y', other_alignments='Y', core_db='ailuropoda_melanoleuca_core_110_2', species_id=1)

Search terms#

Search behaves in the same way as it does for registries:

organism_bt.search("rabbit").head(1)

	scientific_name	division	ontology_id	assembly	assembly_accession	genebuild	variation	microarray	pan_compara	peptide_compara	genome_alignments	other_alignments	core_db	species_id	__ratio__
name
rabbit	oryctolagus_cuniculus	EnsemblVertebrates	NCBITaxon:9986	OryCun2.0	GCA_000003625.1	2009-11-Ensembl/2019-05	N	Y	N	Y	Y	Y	oryctolagus_cuniculus_core_110_2	1	100.0

Search another field (default is .name):

organism_bt.search("sapiens", field=organism_bt.scientific_name).head(2)

	name	division	ontology_id	assembly	assembly_accession	genebuild	variation	microarray	pan_compara	peptide_compara	genome_alignments	other_alignments	core_db	species_id	__ratio__
scientific_name
homo_sapiens	human	EnsemblVertebrates	NCBITaxon:9606	GRCh38.p14	GCA_000001405.29	2014-01-Ensembl/2023-03	Y	Y	Y	Y	Y	Y	homo_sapiens_core_110_38	1	90.000000
procavia_capensis	hyrax	EnsemblVertebrates	NCBITaxon:9813	proCap1	None	2008-11-Ensembl/2013-04	N	N	N	Y	Y	Y	procavia_capensis_core_110_1	1	65.769231

Standardize organism identifiers#

Let us generate a DataFrame that stores a number of organism identifiers, some of which corrupted:

df_orig = pd.DataFrame(
    index=[
        "spiny chromis",
        "silver-eye",
        "platyfish",
        "california sea lion",
        "This organism does not exist",
    ]
)
df_orig


spiny chromis
silver-eye
platyfish
california sea lion
This organism does not exist

We can check whether any of our values are validated against the ontology reference:

validated = organism_bt.validate(df_orig.index, organism_bt.name)
df_orig.index[~validated]

✅ 4 terms (80.00%) are validated

❗ 1 term (20.00%) is not validated: This organism does not exist

Index(['This organism does not exist'], dtype='object')

Ontology source versions#

For any given entity, we can choose from a number of versions:

lb.BiontySource.filter(entity="Organism").df()

	uid	entity	organism	currently_used	source	source_name	version	url	md5	source_website	created_at	updated_at	created_by_id
id
1	zvGR	Organism	vertebrates	True	ensembl	Ensembl	release-110	https://ftp.ensembl.org/pub/release-110/specie...	f3faf95648d3a2b50fd3625456739706	https://www.ensembl.org	2024-01-08 08:53:41.688001+00:00	2024-01-08 08:53:41.688024+00:00	1
2	ClG4	Organism	vertebrates	False	ensembl	Ensembl	release-109	https://ftp.ensembl.org/pub/release-109/specie...	7595bb989f5fec07eaca5e2138f67bd4	https://www.ensembl.org	2024-01-08 08:53:41.688140+00:00	2024-01-08 08:53:41.688152+00:00	1
3	biFw	Organism	vertebrates	False	ensembl	Ensembl	release-108	https://ftp.ensembl.org/pub/release-108/specie...	d97c1ee302e4072f5f5c7850eff0b642	https://www.ensembl.org	2024-01-08 08:53:41.688246+00:00	2024-01-08 08:53:41.688256+00:00	1
4	TE9h	Organism	bacteria	True	ensembl	Ensembl	release-57	https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte...	ee28510ed5586ea7ab4495717c96efc8	https://www.ensembl.org	2024-01-08 08:53:41.688349+00:00	2024-01-08 08:53:41.688358+00:00	1
5	OZIG	Organism	fungi	True	ensembl	Ensembl	release-57	http://ftp.ensemblgenomes.org/pub/fungi/releas...	dbcde58f4396ab8b2480f7fe9f83df8a	https://www.ensembl.org	2024-01-08 08:53:41.688452+00:00	2024-01-08 08:53:41.688462+00:00	1
6	W07m	Organism	metazoa	True	ensembl	Ensembl	release-57	http://ftp.ensemblgenomes.org/pub/metazoa/rele...	424636a574fec078a61cbdddb05f9132	https://www.ensembl.org	2024-01-08 08:53:41.688555+00:00	2024-01-08 08:53:41.688564+00:00	1
7	AVh3	Organism	plants	True	ensembl	Ensembl	release-57	https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant...	eadaa1f3e527e4c3940c90c7fa5c8bf4	https://www.ensembl.org	2024-01-08 08:53:41.688655+00:00	2024-01-08 08:53:41.688664+00:00	1
8	MdBu	Organism	all	True	ncbitaxon	NCBItaxon Ontology	2023-06-20	s3://bionty-assets/df_all__ncbitaxon__2023-06-...	00d97ba65627f1cd65636d2df22ea76c	https://github.com/obophenotype/ncbitaxon	2024-01-08 08:53:41.688754+00:00	2024-01-08 08:53:41.688763+00:00	1

When instantiating a Bionty object, we can choose a source or version:

bionty_source = lb.BiontySource.filter(
    source="ensembl", version="release-110", organism="vertebrates"
).one()
organism_bt = lb.Organism.bionty(bionty_source=bionty_source)
organism_bt

Organism
Organism: vertebrates
Source: ensembl, release-110
#terms: 317

📖 Organism.df(): ontology reference table
🔎 Organism.lookup(): autocompletion of terms
🎯 Organism.search(): free text search of terms
✅ Organism.validate(): strictly validate values
🧐 Organism.inspect(): full inspection of values
👽 Organism.standardize(): convert to standardized names
🪜 Organism.diff(): difference between two versions
🔗 Organism.ontology: Pronto.Ontology object

The currently used ontologies can be displayed using:

lb.BiontySource.filter(currently_used=True).df()

Show code cell output Hide code cell output

	uid	entity	organism	currently_used	source	source_name	version	url	md5	source_website	created_at	updated_at	created_by_id
id
1	zvGR	Organism	vertebrates	True	ensembl	Ensembl	release-110	https://ftp.ensembl.org/pub/release-110/specie...	f3faf95648d3a2b50fd3625456739706	https://www.ensembl.org	2024-01-08 08:53:41.688001+00:00	2024-01-08 08:53:41.688024+00:00	1
4	TE9h	Organism	bacteria	True	ensembl	Ensembl	release-57	https://ftp.ensemblgenomes.ebi.ac.uk/pub/bacte...	ee28510ed5586ea7ab4495717c96efc8	https://www.ensembl.org	2024-01-08 08:53:41.688349+00:00	2024-01-08 08:53:41.688358+00:00	1
5	OZIG	Organism	fungi	True	ensembl	Ensembl	release-57	http://ftp.ensemblgenomes.org/pub/fungi/releas...	dbcde58f4396ab8b2480f7fe9f83df8a	https://www.ensembl.org	2024-01-08 08:53:41.688452+00:00	2024-01-08 08:53:41.688462+00:00	1
6	W07m	Organism	metazoa	True	ensembl	Ensembl	release-57	http://ftp.ensemblgenomes.org/pub/metazoa/rele...	424636a574fec078a61cbdddb05f9132	https://www.ensembl.org	2024-01-08 08:53:41.688555+00:00	2024-01-08 08:53:41.688564+00:00	1
7	AVh3	Organism	plants	True	ensembl	Ensembl	release-57	https://ftp.ensemblgenomes.ebi.ac.uk/pub/plant...	eadaa1f3e527e4c3940c90c7fa5c8bf4	https://www.ensembl.org	2024-01-08 08:53:41.688655+00:00	2024-01-08 08:53:41.688664+00:00	1
8	MdBu	Organism	all	True	ncbitaxon	NCBItaxon Ontology	2023-06-20	s3://bionty-assets/df_all__ncbitaxon__2023-06-...	00d97ba65627f1cd65636d2df22ea76c	https://github.com/obophenotype/ncbitaxon	2024-01-08 08:53:41.688754+00:00	2024-01-08 08:53:41.688763+00:00	1
9	o36k	Gene	human	True	ensembl	Ensembl	release-110	s3://bionty-assets/df_human__ensembl__release-...	832f3947e83664588d419608a469b528	https://www.ensembl.org	2024-01-08 08:53:41.688853+00:00	2024-01-08 08:53:41.688862+00:00	1
11	VTEw	Gene	mouse	True	ensembl	Ensembl	release-110	s3://bionty-assets/df_mouse__ensembl__release-...	fa4ce130f2929aefd7ac3bc8eaf0c4de	https://www.ensembl.org	2024-01-08 08:53:41.689051+00:00	2024-01-08 08:53:41.689060+00:00	1
13	Uhnp	Gene	saccharomyces cerevisiae	True	ensembl	Ensembl	release-110	s3://bionty-assets/df_saccharomyces cerevisiae...	2e59495a3e87ea6575e408697dd73459	https://www.ensembl.org	2024-01-08 08:53:41.689251+00:00	2024-01-08 08:53:41.689261+00:00	1
14	000Q	Protein	human	True	uniprot	Uniprot	2023-03	s3://bionty-assets/df_human__uniprot__2023-03_...	1c46e85c6faf5eff3de5b4e1e4edc4d3	https://www.uniprot.org	2024-01-08 08:53:41.689352+00:00	2024-01-08 08:53:41.689361+00:00	1
16	tD7O	Protein	mouse	True	uniprot	Uniprot	2023-03	s3://bionty-assets/df_mouse__uniprot__2023-03_...	9d5e9a8225011d3218e10f9bbb96a46c	https://www.uniprot.org	2024-01-08 08:53:41.689551+00:00	2024-01-08 08:53:41.689560+00:00	1
18	vqWI	CellMarker	human	True	cellmarker	CellMarker	2.0	s3://bionty-assets/human_cellmarker_2.0_CellMa...	d565d4a542a5c7e7a06255975358e4f4	http://bio-bigdata.hrbmu.edu.cn/CellMarker	2024-01-08 08:53:41.689750+00:00	2024-01-08 08:53:41.689759+00:00	1
19	ypPK	CellMarker	mouse	True	cellmarker	CellMarker	2.0	s3://bionty-assets/mouse_cellmarker_2.0_CellMa...	189586732c63be949e40dfa6a3636105	http://bio-bigdata.hrbmu.edu.cn/CellMarker	2024-01-08 08:53:41.689849+00:00	2024-01-08 08:53:41.689858+00:00	1
20	2Zjk	CellLine	all	True	clo	Cell Line Ontology	2022-03-21	https://data.bioontology.org/ontologies/CLO/su...	ea58a1010b7e745702a8397a526b3a33	https://bioportal.bioontology.org/ontologies/CLO	2024-01-08 08:53:41.689948+00:00	2024-01-08 08:53:41.689957+00:00	1
21	4shh	CellType	all	True	cl	Cell Ontology	2023-08-24	http://purl.obolibrary.org/obo/cl/releases/202...	46e7dd89421f1255cf0191eca1548f73	https://obophenotype.github.io/cell-ontology	2024-01-08 08:53:41.690048+00:00	2024-01-08 08:53:41.690056+00:00	1
25	LmWQ	Tissue	all	True	uberon	Uberon multi-species anatomy ontology	2023-09-05	http://purl.obolibrary.org/obo/uberon/releases...	abcee3ede566d1311d758b853ccdf5aa	http://obophenotype.github.io/uberon	2024-01-08 08:53:41.690449+00:00	2024-01-08 08:53:41.690458+00:00	1
29	zMWv	Disease	all	True	mondo	Mondo Disease Ontology	2023-08-02	http://purl.obolibrary.org/obo/mondo/releases/...	7f33767422042eec29f08b501fc851db	https://mondo.monarchinitiative.org	2024-01-08 08:53:41.690850+00:00	2024-01-08 08:53:41.690859+00:00	1
33	cxPr	Disease	human	True	doid	Human Disease Ontology	2023-03-31	http://purl.obolibrary.org/obo/doid/releases/2...	64f083a1e47867c307c8eae308afc3bb	https://disease-ontology.org	2024-01-08 08:53:41.691246+00:00	2024-01-08 08:53:41.691255+00:00	1
35	2wto	ExperimentalFactor	all	True	efo	The Experimental Factor Ontology	3.57.0	http://www.ebi.ac.uk/efo/releases/v3.57.0/efo.owl	2ecafc69b3aba7bdb31ad99438505c05	https://bioportal.bioontology.org/ontologies/EFO	2024-01-08 08:53:41.691444+00:00	2024-01-08 08:53:41.691453+00:00	1
37	3SSF	Phenotype	human	True	hp	Human Phenotype Ontology	2023-06-17	https://github.com/obophenotype/human-phenotyp...	65e8d96bc81deb893163927063b10c06	https://hpo.jax.org	2024-01-08 08:53:41.691640+00:00	2024-01-08 08:53:41.691649+00:00	1
40	nwdt	Phenotype	mammalian	True	mp	Mammalian Phenotype Ontology	2023-05-31	https://github.com/mgijax/mammalian-phenotype-...	be89052cf6d9c0b6197038fe347ef293	https://github.com/mgijax/mammalian-phenotype-...	2024-01-08 08:53:41.691951+00:00	2024-01-08 08:53:41.691963+00:00	1
41	zAfB	Phenotype	zebrafish	True	zp	Zebrafish Phenotype Ontology	2022-12-17	https://github.com/obophenotype/zebrafish-phen...	03430b567bf153216c0fa4c3440b3b24	https://github.com/obophenotype/zebrafish-phen...	2024-01-08 08:53:41.692055+00:00	2024-01-08 08:53:41.692065+00:00	1
43	p1co	Phenotype	all	True	pato	Phenotype And Trait Ontology	2023-05-18	http://purl.obolibrary.org/obo/pato/releases/2...	bd472f4971492109493d4ad8a779a8dd	https://github.com/pato-ontology/pato	2024-01-08 08:53:41.692254+00:00	2024-01-08 08:53:41.692263+00:00	1
44	h0rU	Pathway	all	True	go	Gene Ontology	2023-05-10	https://data.bioontology.org/ontologies/GO/sub...	e9845499eadaef2418f464cd7e9ac92e	http://geneontology.org	2024-01-08 08:53:41.692353+00:00	2024-01-08 08:53:41.692362+00:00	1
46	fxHJ	BFXPipeline	all	True	lamin	Bioinformatics Pipeline	1.0.0	s3://bionty-assets/bfxpipelines.json	a7eff57a256994692fba46e0199ffc94	https://lamin.ai	2024-01-08 08:53:41.692555+00:00	2024-01-08 08:53:41.692564+00:00	1
47	chfO	Drug	all	True	dron	Drug Ontology	2023-03-10	https://data.bioontology.org/ontologies/DRON/s...	75e86011158fae76bb46d96662a33ba3	https://bioportal.bioontology.org/ontologies/DRON	2024-01-08 08:53:41.692653+00:00	2024-01-08 08:53:41.692662+00:00	1
48	7JhT	DevelopmentalStage	human	True	hsapdv	Human Developmental Stages	2020-03-10	http://aber-owl.net/media/ontologies/HSAPDV/11...	52181d59df84578ed69214a5cb614036	https://github.com/obophenotype/developmental-...	2024-01-08 08:53:41.692751+00:00	2024-01-08 08:53:41.692760+00:00	1
49	JIKv	DevelopmentalStage	mouse	True	mmusdv	Mouse Developmental Stages	2020-03-10	http://aber-owl.net/media/ontologies/MMUSDV/9/...	5bef72395d853c7f65450e6c2a1fc653	https://github.com/obophenotype/developmental-...	2024-01-08 08:53:41.692850+00:00	2024-01-08 08:53:41.692859+00:00	1
50	clid	Ethnicity	human	True	hancestro	Human Ancestry Ontology	3.0	https://github.com/EBISPOT/hancestro/raw/3.0/h...	76dd9efda9c2abd4bc32fc57c0b755dd	https://github.com/EBISPOT/hancestro	2024-01-08 08:53:41.692948+00:00	2024-01-08 08:53:41.692957+00:00	1
51	rsbG	BioSample	all	True	ncbi	NCBI BioSample attributes	2023-09	s3://bionty-assets/df_all__ncbi__2023-09__BioS...	918db9bd1734b97c596c67d9654a4126	https://www.ncbi.nlm.nih.gov/biosample/docs/at...	2024-01-08 08:53:41.693047+00:00	2024-01-08 08:53:41.693056+00:00	1