Importing RDF Data

The table below describes the available procedures and functions:

Once a basic GraphConfig has been created, we can call the RDF import procedures. Let’s look at the first of them: In its most basic form the n10s.rdf.import.fetch method takes a URL string to access the RDF data and the serialisation format. Let’s say you’re trying to load the following RDF document into Neo4j.

You can save it to your local drive or access them directly here. All you’ll need to provide to n10s is the location (file:// or http://) and the serialisation used, Turtle in this case.

The following imports the nsmntx.ttl file hosted on GitHub

n10s will import the RDF data and persist it into your Neo4j graph as the following structure

The first thing we notice is that dataType properties in your RDF have been converted into node properties and object properties are now relationships connecting nodes. Every node represents a resource and has a property with its uri. Similarly, rdf:type statements are transformed into node labels. That’s pretty much it but if you are interested, there is a complete description of the way triple data is transformed into Property Graph data for storage in Neo4j in this post. You will also notice a terminology/vocabulary transformation applied by default. The URIs identifying the elments in the RDF data (resources, properties, etc) have their namespace part shortened to make them more human readable and easier to query with Cypher.

In our example, http://neo4j.org/vocab/sw#name has been shortened to ns0_\_name (notice the double underscore separator used between the prefix and the local name in the URI). Similarly, http://www.w3.org/1999/02/22-rdf-syntax-ns#type would be shortened to rdf\_\_type and so on…​

Prefixes for custom namespaces are assigned dynamically in sequence (ns0, ns1, etc) as they appear in the imported RDF. This is the default behavior but we’ll see later on that it is possible to control that, and use custom prefixes. More details in section Defining custom prefixes for namespaces.

Keeping namespaces can be important if you care about being able to regenerate the imported RDF as we will see in section Exporting RDF data. However, if you don’t care about that, you can tell Neo4j to ignore the namespaces by setting the handleVocabUris parameter to 'IGNORE' in the GraphConfig.

After applying this setting, namespaces will be discarded on all subsequent imports. So if you run the import, only the local names of URIs will be kept. The importRDF method call is identical: here’s what that would look like:

In this case, the imported graph will look something like this, in which the names for labels, properties and relationships are more of the kind you’re use to work with in Neo4j:

Here’s an example that showcases the difference: Let’s say you want to produce a list of plugins that run on Neo4j and what’s the latest versions of each.

If your RDF data is stored in a triple store, you would need to use the SPARQL query on the left to answer the question. To the right you can see the same thing expressed with Cypher in Neo4j.

We’ve seen how to shorten RDF uris into more readable names using namespace prefixes, and we’ve also seen how to ignore them completely. There is a third option: You can keep the complete uris in property names, labels and relationships in the graph by setting the handleVocabUris property to "KEEP". The result will not be pretty and your cypher queries on the imported model will look pretty horrible, but hey, the option is there.

The following deletes existing Resource nodes and updates the GraphConfig

And we can re-import with the usual command:

The imported graph in this case has the same structure, but uses full URIs as labels, relationships, and property names.

Something you may need to do when importing RDF data into Neo4j is exclude certain triples so that they are not persisted in your Neo4j graph. This is useful when only a portion of the RDF data is relevant to your project. The exclusion is done by predicate type "I don’t need to load the version property, or the release date", all you’ll need to do is provide the list of URIs of the predicates you want excluded in parameter predicateExclList. Note that the list needs to contain full URIs.

The following imports data from the nsmntx.ttl Turtle file excluding triples with version or releaseDate as predicates:

In RDF multiple values for the same property are just multiple triples. For example, you can have multiple alternative names for an individual like in the next RDF fragment:

n10s default behavior is to keep only one value for literal properties and it will be the last one read in the triples parsed. So if you run a straight import on that data like this:

The following deletes existing Resource nodes and updates the GraphConfig

The following imports data from the multivalued1.nt N-Triples file

Only the last value for the multivalued altName property will be kept.

This makes things simple and will be perfect if your dataset does not have multivalued properties. It can also be fine if keeping only one value is acceptable, either because the property is not critical or because one value is enough. There will be other cases though, where we do need to keep all the values, and here’s where the config parameter handleMultival will help. Here’s how:

The following deletes existing Resource nodes and updates the GraphConfig

The following imports data from the multivalued1.nt N-Triples file

Now all properties are stored as arrays in Neo4j. Even the ones that have one value only! But we can do better than that, let’s have a look at another example.

The following Turtle RDF fragment with the description of a news article. The article has a number of keywords associated with it.

We want to make sure we keep all values for the nyt:keyword property. The natural way to do this in Neo4j is storing them in an array, so we’ll instruct Neosemantics to do that by setting in the GraphConfig the parameters handleMultival to 'ARRAY' and multivalPropList to the list of property types that are multivalued and we want stored as arrays of values. In the previous example the list will only contain 'http://nyt.com/voc/keyword'.

Here’s the config setting and import commands that we need. Note that I’m combining the multivalued property config setting with the handleVocabUris set to false (the interested reader can try to drop this config and get URIS shortened with prefixes instead):

The following deletes existing Resource nodes and updates the GraphConfig

The following imports data from the multivalued2.ttl Turtle file

And here’s what the result of the import would look like:

When we analyse the result in the Neo4j browser we realise that there’s only one node for the nine triples imported! Yes, keep in mind that all triples in our RDF fragment are datatype properties, or in other words, properties with literal values, which are stored in Neo4j as node properties. All the statements are there, no data is lost, it’s just stored as the internal structure of the node. We can see all properties on the table view on the left hand side of the image.

Note that this time only the properties enumerated in the multivalPropList config parameter are stored as arrays, the rest are kept as atomic values.

Here’s an example of how to query the multiple values of the keyword property: Give me articles tagged with the "Global Warming" keyword.

Literal values in RDF can be tagged with language information. This can be used in any context but it’s common to find it used in combination with multivalued properties to create multilingual descriptions for items in a dataset. In the following example we have a description of a TV series with a multivalued property show:localName where each of the values is annotated with the language.

By default, Neosemantics will strip out the language tags but if you want to keep them you’ll need to set the keepLangTag to true. If we uset it in combination with the setting required to keep all values of a property stored in an array, the import invocation would look something like this:

The following deletes existing Resource nodes and updates the GraphConfig

The following imports data from the multilang.ttl Turtle file

When you import literal values keeping the language annotation, you’ll see that string values have a suffix like @fr for French language, @zh-cmn-Hant for Mandarin Chinese traditional, and so on. The function getLangValue can be used to get the value for a particular language tag. It returns null when there is not a value for the selected language tag. The following Cypher fragment returns the french version of a property and when not found, defaults to the english version.

There are two additional functions for handling language-tagged property values: n10s.rdf.hasLangTag and n10s.rdf.getLangTag.

Here is an example of both functions:

First we create a node with a multivalued property where each value is language-tagged:

We can use n10s.rdf.getLangTag to get the language tag for any specific value

or for all of them:

And similarly, we can use n10s.rdf.hasLangTag to check whether the first value matches a given language tag:

Language tags can also be used as a filter criteria. If we are only interested in a particular language when loading a multilingual dataset, we can set a filter so only literal values with a given language tag (or untagged ones) are imported into Neo4j. The configuration parameter that does it is languageFilter and you’ll need to set it to the relevant tag, for instance 'es' for literals in Spanish language.

Note that this config is request specific, as we may want to import certain language tags from a file, and others from a different one. That’s why instead of being in the GraphConfig it’s in the import method param section. Here’s what it would look like:

In RDF custom data types are annotated to literals after the seperator ^^ in form of an IRI. For example, you can have a custom data type for a currency like in the following Turtle RDF fragment:

The default behavior of neosemantics is to not keep custom data types for properties. So if you run a straight import on that data like this:

Only the value for the properties will be kept.

This makes things simple and will be perfect if your dataset does not have properties with custom data types. But if you need to keep the custom data types, you’ll want to set the config parameter keepCustomDataTypes in your GraphConfig. Here’s how:

The following deletes existing Resource nodes and updates the GraphConfig

The following imports data from the customDataTypes.ttl Turtle file

Now all properties that have a custom data type are saved as strings with their respective custom data type IRIs in Neo4j.

But we can do better than that, let’s have a look at another example. We will use the same Turtle file from above for this example.

If we want to keep the custom data type for only some of the properties then we can instruct neosemantics to do that by setting keepCustomDataTypes to true and customDataTypePropList to the list of property types whose custom data types we want to keep. In the example the list will only contain 'http://example.com/power'.

Here is the import command that we need:

And here’s what the result of the cypher query above would look like after this import:

Note that this time only the custom data types of the properties listed in the customDataTypedPropList are kept, the rest will only have the literal value.

When you import literal values keeping the custom data types, you’ll see that string values have a IRI suffix separated by ^^ from the raw value. For instance "10000^^ns0__EUR" from the example above. The function getDataType can be used to get the data type for a particular property. It returns null when there is no custom data type for the given property.

The following Cypher fragment returns the data type of power.

The function getValue can be used to get the raw value of a particular property without custom data types or language tags.

The following Cypher fragment returns the raw value of power.

The user functions mentioned above can be combined with other user functions like uriFromShort or getIRILocalName etc.

The rdf:type statements in RDF (triples) are transformed into labels by default when we import them into Neo4j. While this is a reasonable approach it may not be your preferred option, especially if you want to load an ontology too and link it to your instance data. In that case you’ll probably want to represent the types as nodes and have 'the magic' of uris have them linked.

By setting the handleRDFTypes parameter in the graph config to "LABELS_AND_NODES" we can achieve exactly that behavior. Actually, as the name suggests we get a double effect and rdf:type statements (triples) are persisted both as labels and as rdf_type relationships connecting the individual to a node representing the class.

Here’s an example: Let’s say we want to load an ontology (notice that it’s actually a small fragment of several ontologies, but it will work for our example). For what it’s worth, it’s an RDF file, so we load it the usual way, with all default settings

We can inspect the result of the import to see that the ontology contains the five class definitions linked in a hierarchy as in the following image. (there are additional rdf_type relationships not shown in this image, linking each class node to an rdfs:Class node).

Next we want to import the instance data, and we want it to link to the ontology graph rather than build a disconnected graph (this is what would happen if rdf:type statements were imported only as labels). With the current "LABELS_AND_NODES" setting, we get the types as label but also as relationships connecting the instances to the previously loaded ontology.

The resulting graph connects the instance data to the ontology elements. This is the magic of unique identifiers (uris), tere’s nothing you need to do for the linkage to happen, if your RDF is well formed and uris are used consistently in it, then it will happen automatically.

There is a third option for the handleRDFTypes configuration parameter, it is "NODES" and as its name suggests persists rdf:type statements exclusively as relationships.

More on the usefulness of representing the ontology in the neo4j graph in section Inference/Reasoning.

Sometimes the RDF data will be a static file, and other times it’ll be dynamically generated in response to an HTTP request (GET or POST) possibly containg parameters, even a SPARQL query. The following two parameters will help in these situations: payload : Takes a String as value and sends the specified data in a POST HTTP request to the the url passed as first parameter of the Stored Procedure. Useful typicaloy for SPARQL endpoints where we want to submit a query to produce the actual RDF. headerParams : Takes a map of property-values and adds each of them as an extra header in the HTTP request. Useful for sending credentials to services requiring authentication (using Authorization header) or to specify the required format (using Accept header).

Here is an example of how to send a request to a SPARQL endpoint and ingest the results directly in Neo4j. The service in question is the Linked Open Data service of the British Library. You can test it here. The service is not authenticated, so no need to use the Authorization header but we want to select the RDF serialisation produced by our request, which we do by setting Accept: "application/turtle". Finally, we pass the SPARQL query as the value of the payload parameter, prefixed with query=.

We obviously need a query producing RDF so we can import it into Neo4j. I’m using a SPARQL DESCRIBE query in the following example but a SPARQL CONSTRUCT query could be used too. If you want to import all the details available in the British Library about 'The world of yesterday' by Stefan Zweig’s, which by the way, if you haven’t read, you should really take a break after this section and go read.

The following deletes all Resource nodes and sets config to ignore vocab URIs

Notice that the British Library service requires you to encode the SPARQL query. We do this with APOC’s apoc.text.urlencode function. After running this you get a pretty poor graph, because the DESCRIBE query only returns the statements having 'The world of yesterday' (http://bnb.data.bl.uk/id/resource/018212405) as subject or object. But we can enrich it a bit by re-running it for a all of the URIs connected to our book as follows:

Which returns:

And produces this graph:

Of course you could do achieve this -or something similar- in different ways, in this case we are using a SPARQL CONSTRUCT query in order to be able to limit the number of triples returned for each resource as some of them are pretty dense.

When applying url shortening on RDF ingestion (either explicitly or implicitly), we have the option of letting neosemantics automatically assign prefixes to namespaces as they appear in the imported RDF. But before doing that, a few popular ones will be set with familiar prefixes. These include "http://www.w3.org/1999/02/22-rdf-syntax-ns#" prefixed as rdf and "http://www.w3.org/2004/02/skos/core#" prefixed as skos.

At any point you can check the prefixes in use by running the n10s.nsprefixes.list procedure.

Before running your first import this method should return no results but after your first run, it should return a list containing at least the following entries.

Let’s say the RDF dataset that you are going to import uses the namespace http://neo4j.org/voc/sw# and you want it to be prefixed as neo instead of ns0 (or ns7) as would happen if the prefix was assigned automatically by neosemantics. You can do this by calling the addNamespacePrefix procedure as follows:

And then when the namespace is detected during the ingestion of the RDF data, the neo prefix will be used.

Make sure you know what you’re doing if you manipulate the prefix definition, especially after loading RDF data as you can overwrite namespaces in use, which would affect the possibility of regenerating the imported RDF.

Sometimes we have an RDF header with a bunch of prefix definitions. This can come from a SPARQL query, from a Turtle document header or from an RDF/XML document root. You can grab this fragment from your RDF document and pass it to the (experimental) addNamespacePrefixesFromText method and it will try to extract the namespace definitions and add each of them individually by invoking n10s.nsprefixes.addFromText. Here’s an example of how it works: