Can import be called via a REST API?

Support Feature Request
1

This post is also related to [How to bulk load data into stardog using python?]

So I know I can import triples in bulk at the command line like Command Line Interface | Stardog Documentation Latest

  • Can I do the same via an HTTP REST API?

In my case I have new data coming in bulk in JSON format. So I will need to transform and bulk load into Stardog.

The other thought I have is to get a virtual graph out of a mongodb where JSON can be bulk re-loaded more easily...

Regards,
Radu

2

Hi Radu,
We don't yet provide a facility to map JSON data to RDF. Using MongoDB is a good intermediate solution as the mappings will be very similar.

The virtual import command is a porcelain command which does not have a single REST API counterpart. You can accomplish the same task by creating a temporary virtual graph using the REST API and then sending the SPARQL command COPY <virtual://your-dataset> TO <a-local-named-graph>.

Jess

3

Jess,

Thanks for your advice. So I am going to create a virtual graph from a mongodb. I found this tutorial handy https://www.stardog.com/blog/stardog--mongodb/ or this one based on GraphQL SMS2 format Native MongoDB Support is Here! | Stardog

  • Where can I find the source code for the above mongodb tutorials?
  • Which should I use old mapping or new SMS2 GraphQL based one?

Thanks,
Radu

4

We haven't yet published a virtual graph tutorial using MongoDB. Both the JSON and GraphQL syntax mappings can be used with MongoDB. In addition to the blog posts, you can find detailed documentation here. Please note that older blog entitled "STARDOG & MONGODB" includes examples based on the older syntax. The newer blog post and the documentation both give examples of the new syntax which starts with the MAPPING keyword. Let me know if you have any questions about it.

Jess

5

Jess,

A formal tutorial on MongoDB virtual graph is so much needed. I wish we had one.

So instead let me try to piece together from the blog and manual a solution. So far I have the follwoing:

  1. MongoDB 3.6.3 and 10 documents in a collection in a database
  2. mongo-java-driver-3.11.0-rc0.jar uploaded to /server/dbms directory and server restarted
  3. Writing my first sms2 mapping - see below
  4. What is the command for creating mongodb virtual graph?
PREFIX : <http://stardog.com/movies/>
MAPPING <urn:movies>
FROM GraphQL {
  movie {
    movieId: _id
    name
  }
}
TO {
  ?movie a :Movie ;
    :name ?name .
}
WHERE {
  BIND (template("http://stardog.com/movies/Title_{movieId}") AS ?movie)
}
6

Hi Radu,

It looks like your movie entity in the source GraphQL is missing a terminating curly brace.

To create the virtual graph, you will need to create a properties file setting the mongodb.uri property containing a URL to connect to your MongoDB instance. You can then use the virtual add command with the --format sms2 argument to support your GraphQL mapping.

Jess

7

Jess,

Yes that is right - i have fixed it above - thank you.

Now can you please provide a sample properties file for this reason?

Thank you,
Radu

8

The properties file needs to contain the mongodb.uri property. This is the same URL that you would use when connecting with the MongoDB driver, e.g.:

mongodb.uri=mongodb://username:password@host/db
9

Ok that makes sense one liner.

Then I should be able to do this, right?

$ stardog-admin virtual add mongodb.properties movies.sms
10

Please include the --format sms2 argument.

11 
$ stardog-admin virtual add --format SMS2 mongodb.properties movies.sms
12

Please execute this command and let us know the result.

13

getting a strange error:

state should be: databaseName does not contain '/'

my working pymongo code:

client = MongoClient('mongodb://u:p@host:port/?authSource=some')
db = client['mydb']

and the contents of mongodb.properties is:

mongodb.uri=mongodb://u:p@host:port/mydb/?authSource=some
14

Try to remove the last slash in your MongoDB URI.

15

getting this now:

There was an unexpected error on the server.

16

Please check the server log for the error.

17

exception deleted to not be annoying

18

Can you share the exact contents of the mapping file? It looks like there is a syntax error.

19

Successfully added virtual graph mongodb

How do I run queries? Which database?

20

So the manual Command Line Interface | Stardog Documentation Latest says:

By default, '*' will be used which means the virtual graph can be accessed from any database.

So I have issued this query but no results come back. I know I have one row in mongodb

SELECT * {
   GRAPH <virtual://contracts> {
      ?s ?p ?o
   }
}