1.5 Frequently Asked Questions
1.5.0.1 General
- I have data and want to publish to OBIS - what do I do?
- Why is it important to share and format data?
- How do I handle sensitive data?
- I am having trouble understanding how Core and Extension tables relate to each other
- How does the OBIS format avoid redundancy in data
- Where can I learn about “Darwin Core”?
- How are extension tables (e.g. eMOF, occurrence) linked with the core table?
- What is the difference between Occurence Core and Event Core?
- What are the responsibilities of node managers?
- Where can I find marine datasets linked to the OBIS network by the GBIF registry, that now require endorising?
- Where can I make suggestions for improvements on this Manual?
- Where can I find OBIS related training videos?
1.5.0.2 Formatting Data
- Is there a checklist of all required Darwin Core fields for OBIS?
- How does data flow in OBIS?
- What should I do if I do not have the data for required fields by OBIS?
- How do I construct an eventID?
- How do I construct occurrenceID?
- What data goes into Occurrence core (or extension) and how do I set up this file?
- How do I set up an Event core table?
-
Do I have to provide decimalLatitude and decimalLongitude for the Event and Occurrence tables?
The answer may depend on your dataset structure, but generally, no. If you have Event core, then you do not need to repeat location information in the Occurrence table (but you can if you’d like). If you are using Occurrence core, then location information must be provided in the Occurrence table. - What data goes into extendedMeasurementOrFact and how do I set it up?
- How do I format dates?
- How do I handle historical data?
- How do I convert coordinates to decimal degrees?
- How do I convert different geographical formats to WGS84?
- How do I compile acoustic, imaging, or other multimedia data for OBIS?
- How do I compile habitat data for OBIS?
- How do I compile tracking data for OBIS?
- How do I compile DNA and genetic data for OBIS?
1.5.0.3 Vocabulary
- How do I map Measurement or Fact terms in OBIS with preferred BODC vocabulary?
- I can’t find a suitable vocabulary, what do I do? How do I request a new vocabulary term?
-
Should I use taxon-specific P01 codes to populate for measurementTypeID? e.g. http://vocab.nerc.ac.uk/collection/P01/current/A15985A1
No. You should never use taxon-specific P01 codes. This is because the taxa are already identified in the Occurrence table, in the fieldsscientificName
andscientificNameID
. - How should I match raw data fields with Darwin Core terminology?
1.5.0.4 Tools
- How do I use the WoRMS taxon match tool?
- Can I fetch a full classification for a list of species from WoRMS?
- What do I do if my scientificName does not return a match from WoRMS?
- Where can I find DNA sequences published in OBIS?
-
Is there a template generator I can use to help create my Event, Occurrence, and eMoF tables?
Yes. There is an Excel template generator developed by Luke Marsden & Olaf Schneider as part of the Nansen Legacy project. Note this template generator is aimed at GBIF users, so make to account for required OBIS terms.
There is also this Excel to Darwin Core macro tool you can use to help generate templates.
- How do I georeference locations, including text-based descriptions?
1.5.0.5 Quality Control
- How do I do data quality control?
- What are the OBIS quality control flags?
- Why are certain records dropped in OBIS?
- What do I do when I am uncertain about the:
- What do I do with freshwater species that are part of my marine dataset?
1.5.0.6 Publishing
- How do I add my data to the OBIS database?
- What metadata do I have to provide? Where? How?
- How do you know which license to choose?
- How do I access the IPT?
- How do I use the IPT?
- Are there instructions for IPT administrators?
- How do I add DOI to my dataset?
- How do I publish to both GBIF and OBIS?
- How do I update my already published dataset?
1.5.0.7 Accessing data
- How do I download data from OBIS?
-
How do I load the full (.csv) export of OBIS data?
Loading the entire OBIS dataset uses a lot of memory and is probably not feasible on most desktop computers. You have a few potential options depending on the use case: i) process the data in smaller batches, or ii) load the dataset into a local database such as SQLite and use SQL queries to analyze the data
Otherwise, we recommend you use the parquet download which is available here, instead of the CSV. Then in R, you can use thearrow
package to work with parquet files. We also have a short tutorial on working with parquet files in R here, with an example application of this approach here (see first code block). - How can I use R to access OBIS data?
- How do I use the OBIS API to fetch and filter data?
- How do I contact the data provider?
- How can I cite OBIS datasets and downloads?
- What are the definitions of the field names in the downloads generated by OBIS?
-
How do I obtain a taxon checklist for an area?
There are a few possible ways to obtain a taxon checklist for a given area. We will obtain a checklist of species in the Albain EEZ as an example. To do this we will create a bounding box around our area of interest, and then apply filters to simplify the geometry.
library(mregions) library(dplyr) library(robis) library(sf) #obtain Albanian EEZ as sf <- mr_shp(key = "MarineRegions:eez", filter = "Albanian Exclusive Economic Zone", maxFeatures = NULL) geom #get WKT for the bounding box <- st_as_text(st_as_sfc(st_bbox(geom)), digits = 6) wkt #fetch occurrences for bounding box <- occurrence(geometry = wkt) %>% occ st_as_sf(coords = c("decimalLongitude", "decimalLatitude"), crs = 4326) #filter using geometry <- occ %>% occ_filtered filter(st_intersects(geometry, geom, sparse = FALSE)) %>% as_tibble() %>% select(-geometry) #get taxa <- occ_filtered %>% alb_taxa group_by(phylum, class, order, family, genus, species, scientificName) %>% summarize(records = n())
-
How do I convert or obtain separate elements from dates in the data download file (e.g.
date_start
field)?The values in
date_start
,date_mid
, anddate_end
are unix timestamps which have been calculated from the ISO date in theeventDate
column. We can convert these numerical values to dates using the formula below.=(E2/86400000)+DATE(1970,1,1)
If, when you apply this formula, you still see numbers, you will need to set the cell formatting to Date. Once you have dates, you can obtain, e.g. months for seasonal analyses using:
=MONTH(H2)
-
How do I filter by or obtain trait information for OBIS data (e.g. all benthic organisms)?
Currently, it is not possible to filter OBIS data by trait. To do this, we recommend using the traits database of the World Register of Marine Species. For example, searching by “functional group”, you can specify benthos, plankton, nekton, etc.