# Federated recursive query

**URL:** <https://community.stardog.com/t/federated-recursive-query/2105>\
**Category:** Feature Request\
**Created:** [November 20, 2019, 10:38am UTC](https://community.stardog.com/t/federated-recursive-query/2105 "2019-11-20T10:38:23Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![lavhal](https://avatars.discourse-cdn.com/v4/letter/l/f9ae1b/32.png) [@lavhal](https://community.stardog.com/u/lavhal)\
**Post date:** [November 20, 2019, 10:38am UTC](https://community.stardog.com/t/federated-recursive-query/2105/1 "2019-11-20T10:38:23Z")

</div>

Hi,

I am trying to run the following query for combining data from Stardog with data from another SPARQL Endpoint:

```
Prefix ex: <https://example.org/> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
 
 SELECT * WHERE {
             ?subj ex:category "StatusCat";
                 rdfs:label ?status.
 
 SERVICE <http://endpoint1:3030/sparql> {		
        	?uri rdfs:label ?status.
       }
 }}

```

Basically, this SPARQL Endpoint is built on top of a normal JSON Web API where conversion of the JSON response to RDF triples is done on the fly. So, this service accepts as a parameter the variable ?status, therefore the query should be executed in an iterative way, i.e. for each value of the ?status variable there should be initiated a call to `<http://endpoint1:3030/sparql>`.

When I run directly:

```
 Prefix ex: <https://example.org/> 
 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
 PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
  
  SELECT * WHERE {
        SERVICE <http://endpoint1:3030/sparql> {		
         	?uri rdfs:label "status1".
         }
  }

```

it works as expected, but this is only for one value.

I also tried without success the following ways:

```
Prefix ex: <https://example.org/> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
    SELECT * WHERE {
        SERVICE <http://endpoint1:3030/sparql> {		
        	?uri rdfs:label ?status.
             {
                 select ?status where{ 
                   ?subj ex:category "StatusCat";
                    rdfs:label ?status.
                 }
             }
        } 
 }

```

and

```
Prefix ex: <https://example.org/> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

SELECT * WHERE {

      ?subj ex:category "StatusCat".

       SERVICE <http://endpoint1:3030/sparql> {		
        	?uri rdfs:label ?status.

             {
                   ?subj rdfs:label ?status.
             }
        }
     
 }

```

The issue seems to be the same or similar to the one posted here: [https://community.stardog.com/t/federated-query-with-inline-data-returns-empty-result/487/2](https://community.stardog.com/t/federated-query-with-inline-data-returns-empty-result/487/2)

Maybe I am doing something wrong.

Best,  
Lav

---

<div class="post-metadata">

**Author:** ![jess](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/jess/32/10_2.png) [@jess](https://community.stardog.com/u/jess)\
**Post date:** [November 20, 2019, 1:54pm UTC](https://community.stardog.com/t/federated-recursive-query/2105/2 "2019-11-20T13:54:10Z")

</div>

Hey Lav,  
Are you implementing Stardog's Java-based Service API? There's a method on the query called `getRequiredInputBindings` which lets you specify which input bindings are required. In this case you should return the `?status` variable.  
Jess

---

<div class="post-metadata">

**Author:** ![lavhal](https://avatars.discourse-cdn.com/v4/letter/l/f9ae1b/32.png) [@lavhal](https://community.stardog.com/u/lavhal)\
**Post date:** [November 20, 2019, 2:25pm UTC](https://community.stardog.com/t/federated-recursive-query/2105/3 "2019-11-20T14:25:53Z")

</div>

Hi Jess,

thanks for your reply.

No, I am using another solution, which basically uses RDF to describe the WEB API Service and map their required parameter(s) directly to desired RDF properties, in this case "rdfs:label". So whatever value is next to rdfs:label (as a literal value) it is passed to the WEB API service, which after the invocation returns a JSON response. Next, this response is converted to triples using a SPARQL construct query, which allows defining a customized RDF graph in the way that I need/want.

---

<div class="post-metadata">

**Author:** ![jess](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/jess/32/10_2.png) [@jess](https://community.stardog.com/u/jess)\
**Post date:** [November 21, 2019, 8:25pm UTC](https://community.stardog.com/t/federated-recursive-query/2105/4 "2019-11-21T20:25:52Z")

</div>

Lav,

We do this type of optimization in certain situations but we would send a batch of `?status` bindings to minimize the number of queries. The batch would be sent as a `VALUES` list which I'm guessing might not play well with your SPARQL interpreter on the remote end.

Jess

---

<div class="post-metadata">

**Author:** ![lavhal](https://avatars.discourse-cdn.com/v4/letter/l/f9ae1b/32.png) [@lavhal](https://community.stardog.com/u/lavhal)\
**Post date:** [November 22, 2019, 9:21am UTC](https://community.stardog.com/t/federated-recursive-query/2105/5 "2019-11-22T09:21:48Z")

</div>

Jess,

I already tried with `VALUES` list, but unfortunately (as you also said) that doesn't work for this scenario, since the WEB API accepts only one value per time. There is a possibility to write code some work on top of the the SPARQL interpreter but that wouldn't be a clean solution that we are looking for.

Anyway, could you post here the optimization that you mentioned using VALUES list?  
-\> I would like to compare with the way how I am doing it, maybe there is something that can help temporarily.

---

<div class="post-metadata">

**Author:** ![jess](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/jess/32/10_2.png) [@jess](https://community.stardog.com/u/jess)\
**Post date:** [November 25, 2019, 5:52pm UTC](https://community.stardog.com/t/federated-recursive-query/2105/6 "2019-11-25T17:52:25Z")

</div>

There's no straightforward way to force this at the moment. It's possible we'll add a hint for this in the not too distant future. Given a query that includes a SERVICE pattern and another arbitrary solution-producing pattern, we would implement it as follows.

Query:

```auto
{
  ?x a :LocalThing
  SERVICE <...> {
    ?x :someAttribute ?y
  }
}

```

The naive approach here would send `select ?x ?y where { ?x :someAttribute ?y }` to the remote endpoint and join the results with the solutions produced by the local pattern.

When the above mentioned optimization is applied, we send bindings of `?x` to the remote endpoint to avoid returning all possible bindings of `?x`. If the local database contains `:X1 a :LocalThing` and `:X2 a :LocalThing`, we might send the following query:

```auto
select ?x ?y {
  ?x :someAttribute ?y
  VALUES (?x) { (:X1) (:X2) }
}

```

If there were only one binding of `?x` we could _potentially_ replace instances of the variable with the constant. It's conceivable that the hint would allow specifying the batch size which could be set to 1 in your case leading to the behavior you need.

Jess

---

<div class="post-metadata">

**Author:** ![lavhal](https://avatars.discourse-cdn.com/v4/letter/l/f9ae1b/32.png) [@lavhal](https://community.stardog.com/u/lavhal)\
**Post date:** [November 26, 2019, 7:32am UTC](https://community.stardog.com/t/federated-recursive-query/2105/7 "2019-11-26T07:32:37Z")

</div>

Hi Jess,

thank you for your explanation. I see and can imagine that this kind of issue is a bit out of traditional query-matching-binding way of the SPARQL and a bit tricky to implement. In the meantime, I will have to think for a workaround solution.

Best,  
Lav

---

<div class="post-metadata">

**Author:** ![zachary.whitley](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/zachary.whitley/32/94_2.png) [@zachary.whitley](https://community.stardog.com/u/zachary.whitley)\
**Post date:** [November 26, 2019, 3:48pm UTC](https://community.stardog.com/t/federated-recursive-query/2105/8 "2019-11-26T15:48:17Z")

</div>

Can you modify your service to extract values expression and iterate on the service side? It would be a lot more performant than a network round trip for each iteration.

---

<div class="post-metadata">

**Author:** ![lavhal](https://avatars.discourse-cdn.com/v4/letter/l/f9ae1b/32.png) [@lavhal](https://community.stardog.com/u/lavhal)\
**Post date:** [November 27, 2019, 7:56am UTC](https://community.stardog.com/t/federated-recursive-query/2105/9 "2019-11-27T07:56:13Z")

</div>

Yes, this was one idea that I already wrote here: [Federated recursive query - #5 by lavhal](https://community.stardog.com/t/federated-recursive-query/2105/5) . So, in principle it is possible, but that would be not a clean solution and very constrained to each use case that we would have.

---

<div class="post-metadata">

**Author:** ![zachary.whitley](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/zachary.whitley/32/94_2.png) [@zachary.whitley](https://community.stardog.com/u/zachary.whitley)\
**Post date:** [November 27, 2019, 1:49pm UTC](https://community.stardog.com/t/federated-recursive-query/2105/10 "2019-11-27T13:49:36Z")

</div>

You can use the example [here](https://github.com/stardog-union/stardog-examples/tree/develop/examples/service) to get started on your masterpiece.

![image](https://canada1.discourse-cdn.com/flex030/uploads/stardog/original/1X/e5f5ffaac2558b91b23b9d8279d2ac221c8829a9.jpeg)

---

<div class="post-metadata">

**Author:** ![lavhal](https://avatars.discourse-cdn.com/v4/letter/l/f9ae1b/32.png) [@lavhal](https://community.stardog.com/u/lavhal)\
**Post date:** [November 28, 2019, 3:48pm UTC](https://community.stardog.com/t/federated-recursive-query/2105/11 "2019-11-28T15:48:04Z")

</div>

Thank you, I will have a look.
