Kategorie: GraphDB
Today we added a plug-in to the 2.1 version of the community edition of GraphDB which is called “FastImport”. It’s basically a bulk import plug-in which takes a proprietary XML format as input and imports vertices and edges into a running GraphDB instance.
In order to use this new plug-in and import feature you need to know that an import basically splits into a two-stage process:
- scheme setup
- fast-import
So first you’ll have to define which vertex and edge types get imported by the following step 2 – you normally do this using the GraphQL and specifying several vertex types. For demonstration purposes we take a small social network with only one vertex type:
CREATE VERTEX TYPE User ATTRIBUTES (String Name, Int64 Age, Set<User> Friends)
After having set-up the scheme the only thing left is to actually call the import plug-in using another short GraphQL query:
IMPORT FROM ‘file:\\100k_import.xml’ FORMAT FastImport
This will, for example, take the 100.000 user dataset and import it into the current GraphDB instance. Of course we did that already for you so here are the comparison results between a GraphQL and FastImport and the persistent and In-Memory version of GraphDB:
Of course you can also download the data-sets used in this small benchmark here:
10.000 Users / 592.374 edges : GraphQL Import , FastImport
100.000 Users / 5.944.332 edges : GraphQL Import, FastImport
In the course of our work at sones GraphDB 2.1 we refactored our index interfaces to make them more suitable for our needs. Furthermore, we wanted to make it easier for the community to implement custom index structures for their special needs. For the latter reason we set up a tutorial and published a sample implementation on github.
The interfaces are explained in our developer wiki, the tutorial can also be found there. The source code regarding the tutorial is located at github.
Source 1: Index Interfaces at developer wiki
Source 2: Integration tutorial at developer wiki
Source 3: Source code at github
We always think about new ways to integrate GraphDB into existing environments. And one of those environments our users are working with right now are the several Enterprise Service Busses which are available right now.
One big player in the ESB environment is the Mule Open Source ESB:
“Mule is a lightweight enterprise service bus (ESB) and integration framework. It can handle services and applications using disparate transport and messaging technologies. The platform is Java-based, but can broker interactions between other platforms such as .NET using web services or sockets.
The architecture is a scalable, highly-distributable object broker that can seamlessly handle interactions across legacy systems, in-house applications and almost all modern transports and protocols.”
In order to show how a GraphDB integrates into those typical ESB environments we created a small example.
The architecture of this example is like this:
The idea behind this is that an example Message-WebApp is posting a message to the Mule ESB and then this message gets transformed and in the last consequence consumed by a sones RESTful webservice hosted by a GraphDB.
You can read more in this tutorial here and you can download the sourcecode here.
Source 1: http://www.mulesoft.org/
Source 2: https://github.com/sones/sones-mule
Source 3: http://developers.sones.de/wiki/doku.php?id=tutorials:muleexampleapp
With version 2.0 of sones GraphDB we introduced an enhanced graph model we call Property Hypergraph.
In this Property Hypergraph model there are some standard edge types:
- single-edge: an edge between two vertices
- multi-edge: an edge splitting up into single-edges of the same edge type pointing towards the same vertex type.
- hyper-edge: an edge to a subgraph made up by all possible types of vertices
In order to create different vertex types since version 1.0 of sones GraphDB the GraphQL command “CREATE VERTEX TYPE” is available to users. In version 2.0 we introduced edge types but those were only useable if you would use the API to interact with the GraphDB instance.
In version 2.1 we’ve now added a full edge type management to be used either through GraphDBs new edge-type API and of course through the new GraphQL extensions which add edge-type handling.
So for example let’s say you want to create an edge type “User” and insert a bunch of those. And you want those users to be connected by a specific edge type which comes with it’s own attributes – something like this:
The GraphQL Queries to create the above scheme would be these:
CREATE EDGE TYPE UserLink ATTRIBUTES (Double weight, String priority, LIST<String> tags) COMMENT = ‘This is my edge type named UserLink.’
CREATE VERTEX TYPE User ATTRIBUTES (String name, SET<User(UserLink)> friends)
INSERT INTO User VALUES (name = ‘UserA’)
INSERT INTO User VALUES (name = ‘UserB’, friends = SETOF(name = ‘UserA’ : (weight = 15.5, priority = ‘high’, tags = LISTOF(‘best friend’, ‘mate’))))
As you can see it’s easy to actually create edge types and add attributes to the relationships these edges represent. It even gets better: you can also use the inheritance mechanisms you are already used to on vertex types as well as undefined (schemeless) attributes.
If you want to dive deeper into edge types you won’t have to wait until the release of GraphDB 2.1 at the end of this year. You can just grab the current source code on github and get started.
Additional documentation and examples are available in our documentation wiki. Here are some places you will find more information:
As you might have noticed by the check-ins on the source code repository we are well on the way towards version 2.1 of sones GraphDB. Besides the many new features and fixes we just merged a feature branch into the main repository which contains the first steps of a simple but already powerful visualization.
For some time now we are looking into ways to visualize the data stored in GraphDB and since todays web browsers come with HTML5 features the idea was born to integrate a future visualization into the existing integrated WebShell and the future web administration tool.
The WebShell is an integrated module of GraphDB which allows the user to access sones GraphDB by just logging in using a web browser.
With the ability to run queries and use plug-ins to determine how the output will look like the WebShell is a perfect place to enhance user experience. Since there are several output plug-ins available with version 2.0 already (JSON, XML, Text, HTML,…) we thought it would be a great idea to have a simple visualization implemented just by adding a new output plug-in to GraphDB.
And that’s what we did in the first step: We added an additional output plug-in called “barchart”. This first new output plug-in uses the great D3.js library to draw nice charts, plots and graphs in HTML5 and the GraphQL ALIAS feature to map x and y coordinate-axes of those visualizations.
A bit tricky is how multi-level mappings of x– and y-axes are handled, we are still working on that. But currently if your data is in the same level you can already output nice graphs like this one:
Just by switching to the barchart format using the FORMAT WebShell command and then changing a query like this:
FROM SimpsonCharacter SELECT Friend.Name, Friend.Friend.Count() WHERE Name = “Homer Simpson”
into
FROM SimpsonCharacter SELECT Friend.Name AS x, Friend.Friend.Count() AS y WHERE Name = “Homer Simpson”
Since the current state is only the beginning we will add more visualization options and features for the release of GraphDB version 2.1 at the end of this year.
For many scenarios it’s important to know how a database performs. Especially these days when the number of databases seem to grow by the day and a choice is hard to make.
To demonstrate how sones GraphDB performs at given use-cases we created a benchmark framework and tool which basically divides benchmarking into two steps:
-
Generate and/or Import use-case specific data and measure the performance
-
Execute use-case specific algorithms on the graph and measure the performance
Because there are many different use-cases these both steps are made up by plug-ins which can be adressed using the commandline which is integrated into the benchmark tool.
The framework, tool and plug-ins are released as AGPLv3 licensed OpenSource software and can be downloaded here.
We distribute the source code mainly because it’s the best way for you to reproduce the results and take a look at what actually is being tested, the other main cause is that we want everybody to be able to benchmark and test their own algorithms on GraphDB.
Source 1: https://github.com/sones/benchmark
Source 2: http://developers.sones.de/wiki/doku.php?id=benchmarks
We want to build the GraphDB community a home – a home they meet to discuss, track, code and have all sorts of graphy fun. This is why we just launched our forum and our bugtracking tool. Those both tools complement the already existing GitHub presence by adding the functionality to discuss and get/give support.
forum.sones.de
It’s a way to communicate with GraphDB users and developers, either those working at sones or those out in the community.
jira.sones.de
In order to make a wish, to report bugs, to track the progress of the GraphDB development the bugtracking tool Jira (made by Atlassian) is used.
You can always go to those new tools and just read – but we encourage you to jump in and register yourself an account to write what you think about GraphDB, what your experience was while using it and what cool projects you are working on currently.
Source 1: http://github.com/sones/sones
Source 2: http://forum.sones.de
Source 3: http://jira.sones.de
Source 4: http://forum.sones.de/register
There is a new example available for all the developers out there who want to start using GraphDB.
This tutorial describes and shows the simplicity of setting up a GraphDB by using the sones GraphDB CommunityEdition. It shows how to create your own database by using the sones GraphDB C# API and how to use GraphDB to create your own graph scheme (in not one but two different approaches).
-
by using GraphDB API Requests
-
by using GraphQL queries
Beside the tutorial itself there is source code available here.
If you are using the GraphQL please read our GraphQL Cheatsheet there you can find the description of all available statements and some additional examples.
The tutorial itself will show you how to:
-
create user defined types, add structured properties, add unknown/schemeless properties
-
create outgoing and incoming edges on a vertex type
-
create an index on a specified property in 3 different ways
-
set constraints on properties (like “unique” and “mandatory”)
-
set up queries and analyze them
-
use function and aggregates in a query
Source 1: C# TagExample Tutorial
Source 2: C# TagExample Source Code
As you already know the Community Edition of sones GraphDB was always released under the GNU Affero General Public License version 3. All Language Bindings and Client Libraries we (sones) released so far were also released under that certain AGPLv3 license.
We got quite a lot interesting feedback on that matter and decided that from now on the GraphDB Language Bindings, Client Libraries and Talend OpenStudio component will be released under the GNU Lesser General Public License version 3.
From now on you can link to the library in your project, no matter what license your project uses, your code will not become GPL’d. That’s basically the only reason for that change.
Currently the following libraries are available:
Besides that the software itself can also be licensed using a proprietary commercial license. Just ask for it.
Link 1: http://www.gnu.org/licenses/agpl.html
Link 2: http://www.gnu.org/licenses/lgpl.html
Link 3: http://www.sones.com/kontakt
The final version of the technical datasheet for the newly released GraphDB 2.0 can be downloaded now:
