« API / Fast Dataset

@graphy/memory.dataset.fast

Primer

• FastDataset is used throughout examples in this document’s code sections to refer to the module’s export. This module was previously called DatasetTree.

Contents

• Memory and Performance – what to be aware of when using this package.
• Construction – how to create an instance of a FastDataset
• Properties
  • .size – number of quads in the dataset.
• Prototype Methods –
  • Iterators
    • * [Symbol.iterator](...) – instances are iterable
  • Canonicalization
    • .canonicalize(...)
  • Set Mutators
    • .add(...)
    • .addAll(...)
    • .addQuads(...)
    • .delete(...)
    • .deleteQuads(...)
    • .clear(...) – remove all quads from the dataset
  • Set Analogues
    • .has(...) – test if the dataset has a given quad
  • Set Algebra Booleans
    • .equals(...) – A = B
    • .contains(...) – (A ∩ B) = B
    • .disjoint(...) – (A ∩ B) = Ø
  • Set Algebra Primitives
    • .union(...) – A ∪ B
    • .intersection(...) – A ∩ B
  • Set Algebra Derivatives
    • .minus(...) – A - (A ∩ B)
    • .difference(...) – (A - (A ∩ B)) ∪ (B - (A ∩ B))
  • Selection
    • .match(...)

---

Memory and Performance

This data structure is best suited for storing quads in memory when the objective is to perform set operations quickly (e.g., union, difference, intersection, etc.) on relatively small datasets (still much better storage density than the alternatives – see memory usage comparison in the performance document).

Future releases of graphy plan to include other data structures, such as DenseDataset, FastDatacache, and DenseDatacache, for meeting the various trade-offs between storage density and insertion/deletion time. In this implementation, certain set operations may reuse pointers to existing object trees in order to save the time it takes to copy subtrees and in order to reduce the overall memory footprint. This has no effect on user functionality since object reuse is handled internally and all methods ensure that stale objects are released to GC.

---

Construction

The require’d return value is a wrapper function that constructs an instance of Dataset. The new keyword is optional.

dataset([config: DatasetConfig])

• Create a new dataset.
• returns a new Duplex<Quad, Quad> (accepts Quad objects on its writable side, pushes Quad objects on its readable side)

Usage examples

The following example is shown for usage in Node.js, however the same async/await mechanisms can be used in the browser with polyfills for piping fetch and awaiting stream events.

Read from a Turtle file in Node.js:

const fs = require('fs');
const { once } = require('events');
const ttl_read = require('@graphy/content.ttl.read');
const dataset = require('@graphy/memory.dataset.fast');

// load 'input-a.ttl' into a new Dataset
let y_input_a = dataset();

fs.createReadStream('input-a.ttl')
    .pipe(ttl_read())
    .pipe(y_input_a);


// load 'input-b.ttl' into a new Dataset
let y_input_b = dataset();

fs.createReadStream('input-b.ttl')
    .pipe(ttl_read())
    .pipe(y_input_b);


// wait for both datasets to finish loading using async/await
(async() => {
    // Dataset extends Node.js' Duplex, so simply listen for the 'finish' event
    // to be notified once the dataset has finished loading
    await Promise.all([
        once(y_input_a, 'finish'),
        once(y_input_b, 'finish'),
    ]);

    // compute the union of the two datasets
    let y_union = y_input_a.union(y_input_b);

    // do something with union...
})();

---

Properties

.size

• get the number of quads in the dataset.
• returns a #number/integer
• examples:

    let y_dataset = dataset();
    let h_prefixes = {
       dbr: 'http://dbpedia.org/resource/',
       rdfs: 'http://www.w3.org/2000/01/rdf-schema#',
       graph: 'http://ex.org/graph#',
    };
  
    y_dataset.size;  // 0
  
    y_dataset.addQuads(factory.c4({
        'graph:example': {
            'dbr:Banana': {
                'rdfs:label': [
                    '@en"Banana',
                    '@fr"Banane',
                ],
            },
        },
    }, h_prefixes));
       
    y_dataset.size;  // 2
  

---

Methods

* [Symbol.iterator]() per @RDFJS/DatasetCore

• create an iterator to traverse each quad in this.
• yields Quads.

.canonicalize()

• create a new FastDataset by applying the RDF Dataset Normalization Algorithm (URDNA2015). If you want isomorphism to hold under the usual FastDataset methods, you should use this method on both dataset instances prior to testing .equals(), .contains(), .disjoint(), and prior to using .union(), .intersection(), .minus(), and .difference().
• returns a new FastDataset.

.add(quad: AnyQuad) implements @RDFJS/DatasetCore.add

• add a single quad to the dataset; will only succeed if the quad is not already present.
• returns this.

.addAll(quads: @RDFJS/Dataset | sequence<AnyQuad>) implements @RDFJS/Dataset.addAll

• add quads to the dataset; will only add each quad that is not already present.
• returns this

.addQuads(quads: Iterable<Quad>)

• add quads to the dataset; will only add each quad that is not already present. Bypass the internal overhead of checking and needlessly converting each quad to graphy-safe objects. Notice that quads must be an Iterable sequence of graphy Quads.
• returns a #number/integer indicating how many quads were successfully added to the dataset.

.delete(quad: AnyQuad) implements @RDFJS/DatasetCore.delete

• delete the given quad from the dataset if it exists.
• returns this

.deleteQuads(quads: list<Quad>)

• delete the given quads from the dataset if they exist.
• returns a #number/integer indicating how many quads were successfully deleted from the dataset.

.clear()

• remove all quads from the dataset.
• returns undefined.

.has(quad: AnyQuad) implements @RDFJS/DatasetCore.has

• tests if this contains the given quad.
• returns a boolean.

.equals(other: FastDataset) implements @RDFJS/Dataset.equals

• A = B
• tests if this and other are strictly equal graphs.
• returns a boolean

.contains(other: FastDataset)

• (A ∩ B) = B
• tests if this contains all quads in other.
• returns a boolean.

.disjoint(other: FastDataset)

• (A ∩ B) = Ø
• tests if this is disjoint with other.
• returns a boolean.

.union(other: FastDataset) implements @RDFJS/Dataset.union

• create a new FastDataset by combining the quads from both this and other.
• returns a new FastDataset.

.intersection(other: FastDataset) implements @RDFJS/Dataset.intersection

• A ∩ B
• create a new FastDataset by intersecting the quads between this and other.
• returns a new FastDataset.

.minus(other: FastDataset)

• (A - (A ∩ B))
• create a new FastDataset by subtracting the quads in other from this.
• returns a new FastDataset.

.difference(other: FastDataset) implements @RDFJS/Dataset.difference

• (A - (A ∩ B)) ∪ (B - (A ∩ B))
• create a new FastDataset by taking the difference between this and other.
• returns a new FastDataset.

.match([subject: null | AnyTerm[, predicate: null | AnyTerm[, object: null | AnyTerm[, graph: null | AnyTerm]]]]) implements @RDFJS/DatasetCore.match

• create a new FastDataset by matching the specified subject, predicate, object, and/or graph, or any quads if null is given for any role.
• returns a new FastDataset.