Load json into a materialized graph via HTTP POST - possible?

Support Feature Request
1

For now I have my extracted information loaded into a mongodb and then virtualized using an SMS2 GraphQL mapping. So far so good - it works. Once written data will not changing over time but it needs to be mapped to some other materialized (or virtual) graphs.

So I would like to bypass mongodb altogether and leverage the SMS2 GaraphQL mapping to load JSON documents into a materialized (i.e. default) graph.

  • Can I do that from my Python code via an HTTP POST api?

Regards,
Radu

2

From the release notes it looks like Stardog 7.0.1 added support for importing JSON files directly.

VG JSON file import (#6217)

I'm not sure if there's anything in the docs about it but I'd imaging it works similarly to importing CSV...

...I haven't found the docs yet but it's in the stardog-admin help page

stardog-admin help virtual import
  • Import the contents of a JSON file with the given mappings
    $ stardog-admin virtual import --format sms2 myDB mappings.rq input.json
3

Radu,
For the time being, the HTTP API to load JSON files is not stabilized. You should call out to stardog-admin virtual import from your Python script.

4

Jess,

Does it mean that i have to write JSON to a file first? That would be counterproductive.

I am all for testing the new HTTP API - let me know when that can be done.

Regards,
Radu

5

Thanks Radu. We will let you know when the new API is ready.

6

Jess,

Looking at:

Import the contents of a JSON file with the given mappings

    $ stardog-admin virtual import --format sms2 myDB mappings.rq input.json
  • Can I have multiple JSON documents in input.json file?
7

Yes, you can. If the template in the mapping is for a single document, the file should contain multiple documents, eg { "doc":1 } { "doc":2 } ...

8

Jess,

Can you please provide an example of mapping.rq file. The one that I have is SMS2 based and it is named mapping.sms and I get this error:

  • 'sms' is not a known RDF format file extension
9

What command are you running?

10

I am running the above command - but I believe it is not supported in v6 ... :frowning:

Upgrade to v7 a bit painful...

11

That's correct. JSON import was introduced in Stardog 7.0.1.

12

Hi Jess,

  • Can use the HTTP API to load JSON data with SMS2 mapping now? Is it stable? Last time you mentioned it wasn't stabilized yet...

If so where can I find doc and example?

I found this apiary documentation but the example uses R2RML and no input JSON. Can you please provide an example or better yet a tutorial?

Thanks,
Radu

13

Hi Radu,

We are planning to release a stable HTTP API to import JSON in the Feb (or late Jan) release of Stardog. If you need to do this from your Python code, the best approach at the moment is to shell out to stardog-admin virtual import. You can do this using the os.system() method in Python.

Jess

14

Jess,

Thank you for your response. Looking forward to use the new HTTP API.

  • Would it be possible to better understand the SMS2 roadmap?

Previously I had questions and suggestions such as IF/ELSE conditions support in SMS2. They may or have not been implemented yet. However if they are implemented - that will decrease my python code by a factor.

Regards,
Radu

15

Hi Radu,

I don't believe we much in the way of a SMS2 roadmap. Do you have any particular questions?

We maintain a table that shows what each is supported in the different mapping syntaxes. For JSON, we have full support of SPARQL functions in bind expressions, so you can use IF().

A full list of the functions we support is here.

Thank you,
Kate

16

Hi Jess,

Just checking on this:

Like kids in the car asking:

  • Are we there yet?

Thanks,
Radu

17

Radu,
This is supported in Stardog 7.1.2. You can now import JSON and CSV files using SMS2 mappings with the HTTP API. Here is how it might be used from the command line with cURL:

curl -u admin:admin -F "database=foo" \
  -F "mappings=<mappings.sms" \
  -F "input_file_type=JSON" \
  -F named_graph=urn:ng \
  -F "input_file=<data_file.json" \
  http://localhost:5820/admin/virtual_graphs/import
18

Thanks Jess - I will try that with python api