MetaStore
Introduction
MetaStore is a metadata repository for managing millions of metadata documents. MetaStore supports communities with their specific schemas (XSD, JSON Schema).
Datasheet
Status: 14.06.2022
| Homepage | --- |
| Description | https://github.com/kit-data-manager/metastore2#readme |
| Code | https://github.com/kit-data-manager/metastore2 |
| Communities | --- |
| Version | 1.0.0 (released on 29.03.2022) |
Features
Status: 17.05.2021
| Supported Schema(s) | --- | All schemas with supported format |
| Supported Format(s) | XSD, JSON Schema | |
| Interface(s) | REST, GUI | GUI (coming soon) |
| Open Source | yes | gitHub |
| License | Apache 2.0 | |
| Versioning | yes | Only simple versioning. Works like a history. |
| AAI | yes | Authorization for documents based on roles. (JWT) |
| External Storage | no | Linking via URLs could be possible but not implemented yet. |
| Referencable | yes | Status: Referencable via PID(coming soon) or persistent URL. |
Description
- Register Schema:
- Support for arbitrary schemas of a specific format (e.g. JSON Schema, XSD)
- The schema should at least be referencable by a unique identifier.
- Update Schema:
- Possibility to
- work on different versions of a schema
- adapt schemas over time
- Possibility to
- Validate Schema:
- Check schema for correct syntax
- Ingest Metadata:
- Store metadata (document) in repository
- Ideally with previous validation
- Store metadata (document) in repository
- Update Metadata:
- Possibility to update already ingested metadata (documents).
- Validate Metadata:
- Possibility to validate documents on the basis of registered schemas.
- Search by Administrative MD:
- Search documents by their metadata (e.g. ingest date, ingester, ...)
- Search by Content:
- Search documents by their content
- Persistent Identifier:
- Support for Persistent Identifiers (e.g. DOI, Handle)
Additional Features
Status: 17.05.2021
MetaStore consists of two parts:
- Metadata Repository for managing metadata documents
- Schema Registry for registering schema documents
Metadata Repository
- Up to million of documents (tested with 10 M metadata documents)
- Automatic validation during ingest
- High throughput, high performance (ingest & access)
- Live interaction / online handling and manipulation of metadata documents for data of ongoing research
Schema Registry
- Registering schema documents (XSD, JSON Schema)
- Supports arbitrary number of user-defined metadata schemas
- Supports different versions of a metadata schema
Functionality
Status: 17.05.2021
| Function | Supported | Remarks |
|---|---|---|
| Register Schema | + | JSON Schema, XSD |
| Update Schema | + | |
| Validate Schema | + | |
| Ingest Metadata | + | |
| Update Metadata | + | |
| Validate Metadata | + | |
| Search by ... | ||
| ... Administrative MD | + | |
| ... Content | (+) | coming soon via enhanced service |
| Persistent Identifier | + | via external service |
Remarks
Enhanced services
- Metadata Standards Catalog
- Human readable description for finding appropriate schema
- Direct access to schema via MetaStore.
- Metadata Explorer (Outlook)
- Exploring metadata by content
- Metadata Editor (coming soon)
- Update schema/documents online
- Authentication Server
- If a separate authentication service is to be set up.