mappingbased_properties_en.nt: (some line)
<http://dbpedia.org/resource/12_Monkeys>
<http://dbpedia.org/ontology/editing>
<http://dbpedia.org/resource/Mick_Audsley> .       
stands for:  “12 Monkeys” has a “editor” “Mick Audsley”.

In other files there is additional information available, e.g. that

• “12 Monkeys” is a film
• “Mick Audsley” is a person
• … probably more information about “12 Monkeys” and “Mick Audsley”

What we want to do in sones GraphDB is to create a VERTEX for the film “12 Monkeys”. This includes

• type information – 12 Monkeys is a film
• a set of properties – e.g. its budget
• EDGES to related information, e.g. the editor Mick Audsley.

There is a single point of information (The VERTEX “12 Monkeys”) that holds all information and relation in a single instance. To import the VERTEX “12 monkeys”, we had to write a parser over all available triple files that gives us all related information from DBPedia data set.
At this point we’ve had two options implementing this parser. The first one was to read all triple files in a dedicated order to ensure data validity (we need to know that “12 Monkeys” is a movie, to be able to assign the predicate “editor” unambiguous) or do an intermediate step by creating a temporary file that collects all data without validation and to do the import afterwards.
Our decision was to do the intermediate step, because of that it allows some synchronization during reading the triple files and avoids creating invalid data since exported data can be cross-checked easily.
This step is represented by project “2_ParseAndConvertTripleDataFiles” in solution GraphDBPedia available at http://github.com/sones/sones-dbpedia.  The parser reads only a subset of offered data-files to show functionality and focus on the added values.
The result of the export for “Apollo 8” looks like this:

1  VertexID=-9223372036854775808
2  http://dbpedia.org/resource/Apollo_8=http://dbpedia.org/ontology/SpaceMission
3  LongAbstract_de=viel text
4  LongAbstract_en=a lot of text
5  http://dbpedia.org/ontology/commandModule_en=CM-103
6  http://dbpedia.org/ontology/missionDuration_en=529242.0
7  http://dbpedia.org/ontology/lunarOrbitTime_en=72613.0
8  http://dbpedia.org/ontology/crewSize_en=3
9  http://dbpedia.org/ontology/lunarModule_en=Ballast: Lunar Test Article B
10  http://dbpedia.org/ontology/serviceModule_en=SM-103
11  http://dbpedia.org/ontology/nextMission_en=http://dbpedia.org/resource/Apollo-9-patch.png
12   http://dbpedia.org/ontology/booster_en=http://dbpedia.org/resource/Saturn_V
13   http://dbpedia.org/ontology/previousMissions_en= http://dbpedia.org/resource/AP7lucky7.png
14   http://dbpedia.org/ontology/launchPad_en=http://dbpedia.org/resource/Kennedy_Space_Center_Launch_Complex_39
15   ShortAbstract_en=some text
16   Name_en=http://dbpedia.org/resource/Apollo_8
17    http://dbpedia.org/ontology/SpaceMission/lunarOrbitTime_en=20.170277777777777
18   http://dbpedia.org/ontology/SpaceMission/missionDuration_en=6.125486111111111

Apart from one property, all data had been exported from the triple files. During importing the ontology information (line2 in this example), we’ve also created a VertexID – unique for the corresponding VERTEX TYPE. This allows us to do a unique and performant linking during data import (happens later) by referring to this ID.

After this intermediate step, the real import step can be done. Sones Graph DB offers GraphQL as simple and intuitive language. Based on the data-structure we’ve prepared above, with GQL two steps have to be done. At first, create all VERTICES including all properties and afterwards do the linking between all VERTICES.
Therefore, for the example above, two statements would be created:

 INSERT INTO httpwwwdbpediaorgontologySpaceMisson VALUES (
   VertexID=-9223372036854775808,
   LongAbstract_de=’viel text’,
   LongAbstract_en=’a lot of text’,
   Name_en=’ http://dbpedia.org/resource/Apollo_8’,
   httpdbpediaorgontologycommandModule_en=’CM-103’,
   httpdbpediaorgontologymissionDuration_en=529242.0,
   httpdbpediaorgontologylunarOrbitTime_en=72613.0,
   httpdbpediaorgontologycrewSize_en=3,
   httpdbpediaorgontologylunarModule_en=’Ballast: Lunar Test Article B’,
   httpdbpediaorgontologyserviceModule_en=’SM-103’,
   ShortAbstract_en=’some text’,
   httpdbpediaorgontologySpaceMissionlunarOrbitTime_en=20.170277777777777,  
   httpdbpediaorgontologySpaceMissionmissionDuration_en=6.125486111111111

UPDATE httpwwwdbpediaorgontologySpaceMisson SET
(
   httpdbpediaorgontologynextMission_en=SETOF(Name_en=’http://dbpedia.org/resource/Apollo-9’)
   httpdbpediaorgontologybooster_en=SETOF(Name_en=’http://dbpedia.org/resource/Saturn_V’)
httpdbpediaorgontologypreviousMission_en=SETOF( Name_en=’http://dbpedia.org/resource/AP7’)
httpdbpediaorgontologylaunchPad_en=SETOF(Name_en=’http://dbpedia.org/resource/Kennedy_Space_Center_Launch_Complex_39’)
)
WHERE VertexID=-9223372036854775808

The problem of this approach is, that EDGES are set via a WHERE condition that maybe is not unique or the attribute is not set at all at the target VERTEX. An option to solve this, is to verify the ID of the target vertex and do the linking via this condition.

Sones GraphDB also offers another option to do the importing, XmlBulkImport. It has the advantage that it is faster than GraphQL (due to the fact it uses Graph-filesystem interfaces) and also organizes INSERTING and LINKING of data itself. Instead of creating GraphQL, a proprietary XML structure has to be created and the import is done via a single IMPORT GQL statement.
A description of this format and its usage can be found at: http://developers.sones.de/wiki/doku.php?id=importexport:xmlbulkimport

This XmlBulkImport data file is created by project “3_ParseAndConvertTripleDataFiles” in solution GraphDBPedia available at http://github.com/sones/sones-dbpedia”.

OWL-Class:

<owl:Class rdf:about="http://dbpedia.org/ontology/Island">
   <rdfs:label xml:lang="en">island</rdfs:label>
    <rdfs:label xml:lang="el">νησί</rdfs:label>
    <rdfs:label xml:lang="fr">île</rdfs:label>
    <rdfs:subClassOf
           rdf:resource="http://dbpedia.org/ontology/PopulatedPlace">
    </rdfs:subClassOf>
</owl:Class>

- represents an Island (based on a PopulatedPlace)

OWL-DatatypeProperty:

<owl:DatatypeProperty rdf:about="http://dbpedia.org/ontology/numberOfIslands">
   <rdfs:label xml:lang="en">number of islands</rdfs:label>
    <rdfs:domain rdf:resource="http://dbpedia.org/ontology/Island"></rdfs:domain>
    <rdfs:range rdf:resource="http://www.w3.org/2001/XMLSchema#nonNegativeInteger"></rdfs:range>
</owl:DatatypeProperty>

- describes the non-negative Integer attribute “number of islands” for the class Island.

OWL-ObjectProperty:

 <owl:ObjectProperty rdf:about="http://dbpedia.org/ontology/highestState">
    <rdfs:label xml:lang="en">highest state</rdfs:label>
    <rdfs:domain rdf:resource="http://dbpedia.org/ontology/Island"></rdfs:domain>
    <rdfs:range rdf:resource="http://dbpedia.org/ontology/PopulatedPlace"></rdfs:range>
</owl:ObjectProperty>

represents the highest “PopulatedPlace” on an island.

The conversion creates a

• VERTEX TYPE  – one for each class,
• having multiple PROPERTIES – from datatype properties
• and multiple EDGES – from object-properties

Within the data schema, there is a big amount of multi-lateral dependencies. The CREATE VERTEX TYPES statement solves all of them and creates a valid data schema.

Additionally to the ontology from the OWL file, we’ve added some vertex types to fix some problems we’ve run at and to enhance the functionality a little bit:

• At first, the VERTEX TYPE Thing was not described in the Ontology. It is the base class in the ontology that all other VERTEX TYPES are base upon.
• To reflect disambiguation, we’ve created a VERTEX TYPE Instance with an EDGE to a SET of Thing. In case there is a disambiguation, an Instance refers to the corresponding NODEs in the GraphDB.
• Within the RDF-files, labels are saved in dedicated triples. We’ve added a dedicated VERTEX TYPE also, to avoid a mix-up in case one label refers to multiple Instances.

Currently, the GraphDB has some limitations regarding the allowed characters within VERTEX TYPES, its ATTRIBUTES and EDGES. The OWL and RDF format is generally based on URLs as data-definition. GraphDB has limitations working with colons, dots  and slashes (both slash and backslash). Our simple workaround was to keep the URL and remove all occurrences of these characters. This leads us from http://dbpedia.org/ontology/Island to httpdbpediaorgontologyIsland.

Another challenge is the type-mapping between OWL and GraphDB. GraphDB supports c# simple data types, in the DBPedia OWL we are facing a list of 9 datatypes from an XML schema,  DBPedia area units, speed units, density units, time units, volume units, distance units and several others. This led us to a huge switch that does the mapping – all properties could be reflected with the C# data types without data loss.

Wikipedia is available in multiple languages. DBPedia export currently is provided in 99 of them.
Some time later (during the next steps) we’ve found out that data in several languages differs a little bit sometimes, since there are different authors. For the data schema, this is relevant, because there are options how to handle this behavior.

One option is to let the data importer application logic decide how to handle this. We’ve decided to make the data schema language specific and provide a separate – language specific – attributes. This grows up the data schema a little bit, but does not lead to any data loss. Additionally, some application logic can be implemented later on, to check data quality for each node.

The command-line  tool “1_CreateGqlSchemaFromOntology” ‚available at GitHub (https://github.com/sones/sones-dbpedia) VisualStudion solution creates the CREATE VERTEX TYPES statements as described above, based on the ontology of DBPedia 3.6. – later versions currently have not yet been tested.
The command line executable has to be started with 2 parameters:

• .owl filename (the filename has either to be an absolute path or located within the executables directory.
• result .gql file – name of the file, where all queries will be inserted in.

During runtime, the user will be requested for all languages that have to be reflected in schema. Our suggestion is to use 2-letter county-codes like “_en” or “_de”. An empty string exits the iteration.
After the execution the result .gql file easily can be imported via IMPORT GQL statement.

from www.dbpedia.org/About:

“DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. DBpedia allows you to ask sophisticated queries against Wikipedia, and to link other data sets on the Web to Wikipedia data. We hope this will make it easier for the amazing amount of information in Wikipedia to be used in new and interesting ways, and that it might inspire new mechanisms for navigating, linking and improving the encyclopaedia itself.”

We’ve decided: Yes, it is!.

 from www.dbpedia.org/Datasets:

DBpedia uses the Resource Description Framework (RDF) as a flexible data model for representing extracted information and for publishing it on the Web. We use the SPARQL query language to query this data. Please refer to the Developers Guide to Semantic Web Toolkits to find a development toolkit in your preferred programming language to process DBpedia data.

The DBpedia knowledge base currently describes more than 3.64 million things, out of which 1.83 million are classified in a consistent Ontology, including 416,000 persons, 526,000 places (including 360,000 populated places), 106,000 music albums, 60,000 films, 17,500 video games, 169,000 organizations (including 40,000 companies and 38,000 educational institutions), 183,000 species and 5,400 diseases.

At this time we’ve not yet had too much experiences with the Semantic Web, therefore there was probably some work to do.

The following blog articles will describe our work and refer to the source-code available under www.github.com/sones/sones-dbpedia

The interfaces are explained in our developer wiki, the tutorial can also be found there. The source code regarding the tutorial is located at github.

Source 1: Index Interfaces at developer wiki
Source 2: Integration tutorial at developer wiki
Source 3: Source code at github

One big player in the ESB environment is the Mule Open Source ESB:

“Mule is a lightweight enterprise service bus (ESB) and integration framework. It can handle services and applications using disparate transport and messaging technologies. The platform is Java-based, but can broker interactions between other platforms such as .NET using web services or sockets.

The architecture is a scalable, highly-distributable object broker that can seamlessly handle interactions across legacy systems, in-house applications and almost all modern transports and protocols.”

In order to show how a GraphDB integrates into those typical ESB environments we created a small example.

The architecture of this example is like this:

The idea behind this is that an example Message-WebApp is posting a message to the Mule ESB and then this message gets transformed and in the last consequence consumed by a sones RESTful webservice hosted by a GraphDB.

You can read more in this tutorial here and you can download the sourcecode here.

Source 1: http://www.mulesoft.org/
Source 2: https://github.com/sones/sones-mule
Source 3: http://developers.sones.de/wiki/doku.php?id=tutorials:muleexampleapp

In this Property Hypergraph model there are some standard edge types:

• single-edge: an edge between two vertices
• multi-edge: an edge splitting up into single-edges of the same edge type pointing towards the same vertex type.
• hyper-edge: an edge to a subgraph made up by all possible types of vertices

In order to create different vertex types since version 1.0 of sones GraphDB the GraphQL command “CREATE VERTEX TYPE” is available to users. In version 2.0 we introduced edge types but those were only useable if you would use the API to interact with the GraphDB instance.

In version 2.1 we’ve now added a full edge type management to be used either through GraphDBs new edge-type API and of course through the new GraphQL extensions which add edge-type handling.

So for example let’s say you want to create an edge type “User” and insert a bunch of those. And you want those users to be connected by a specific edge type which comes with it’s own attributes – something like this:

The GraphQL Queries to create the above scheme would be these:

CREATE EDGE TYPE UserLink ATTRIBUTES (Double weight, String priority, LIST<String> tags) COMMENT = ‘This is my edge type named UserLink.’
CREATE VERTEX TYPE User ATTRIBUTES (String name, SET<User(UserLink)> friends)
INSERT INTO User VALUES (name = ‘UserA’)
INSERT INTO User VALUES (name = ‘UserB’, friends = SETOF(name = ‘UserA’ : (weight = 15.5, priority = ‘high’, tags = LISTOF(‘best friend’, ‘mate’))))

As you can see it’s easy to actually create edge types and add attributes to the relationships these edges represent. It even gets better: you can also use the inheritance mechanisms you are already used to on vertex types as well as undefined (schemeless) attributes.

If you want to dive deeper into edge types you won’t have to wait until the release of GraphDB 2.1 at the end of this year. You can just grab the current source code on github and get started.

Additional documentation and examples are available in our documentation wiki. Here are some places you will find more information:

• Edges in GraphDB
• GraphQL data definition language (DDL)

And you can download it now with our source-code package from our GitHub repository.

The best way to show-off the new functionality is by literally showing it:

A good start for the documentation of the new visualization options is our ever growing wiki.

To demonstrate how sones GraphDB performs at given use-cases we created a benchmark framework and tool which basically divides benchmarking into two steps:

1. Generate and/or Import use-case specific data and measure the performance

2. Execute use-case specific algorithms on the graph and measure the performance

Because there are many different use-cases these both steps are made up by plug-ins which can be adressed using the commandline which is integrated into the benchmark tool.

The framework, tool and plug-ins are released as AGPLv3 licensed OpenSource software and can be downloaded here.

We distribute the source code mainly because it’s the best way for you to reproduce the results and take a look at what actually is being tested, the other main cause is that we want everybody to be able to benchmark and test their own algorithms on GraphDB.

Source 1: https://github.com/sones/benchmark
Source 2: http://developers.sones.de/wiki/doku.php?id=benchmarks

forum.sones.de

It’s a way to communicate with GraphDB users and developers, either those working at sones or those out in the community.

jira.sones.de

In order to make a wish, to report bugs, to track the progress of the GraphDB development the bugtracking tool Jira (made by Atlassian) is used.

You can always go to those new tools and just read – but we encourage you to jump in and register yourself an account to write what you think about GraphDB, what your experience was while using it and what cool projects you are working on currently.

Source 1: http://github.com/sones/sones
Source 2: http://forum.sones.de
Source 3: http://jira.sones.de
Source 4: http://forum.sones.de/register