# Query response time very high

**URL:** <https://community.stardog.com/t/query-response-time-very-high/2042>\
**Category:** Support\
**Created:** [October 26, 2019, 7:28am UTC](https://community.stardog.com/t/query-response-time-very-high/2042 "2019-10-26T07:28:17Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 26, 2019, 7:28am UTC](https://community.stardog.com/t/query-response-time-very-high/2042/1 "2019-10-26T07:28:17Z")

</div>

Hi I'm running following query on dataset of count 44,000 (mongodb)

SELECT ?seqId {  
GRAPH virtual://aus {  
?images :existence "0707" ;  
:seqId ?seqId .  
}  
}  
LIMIT 50

Its taking almost 2 minutes to get the response. Is this the usual behavior or is there something i have to do to make it run faster...?

---

<div class="post-metadata">

**Author:** ![zachary.whitley](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/zachary.whitley/32/94_2.png) [@zachary.whitley](https://community.stardog.com/u/zachary.whitley)\
**Post date:** [October 26, 2019, 1:13pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/2 "2019-10-26T13:13:27Z")

</div>

What version of stardog are you running and can share the query plan?

Stardog query explain

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 26, 2019, 6:22pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/3 "2019-10-26T18:22:40Z")

</div>

I'm using 7.0.1 @zachary.whitley

```
Plan is
Slice(offset=0, limit=50) [#50]
`─ Projection(?seqId) [#18K]
   `─ Projection(?images, ?seqId) [#18K]
      `─ ServiceJoin [#18K]
         +─ VirtualGraphMongoDB<virtual://austria> [#36212] {
         │ +─ Query=
         │ +─ { $match : {$and : [{"frameId" : {$exists : true, $ne : null}}, {"seqId" : {$exists : true, $ne : null}}]} },
         │ +─ {$project: {frameId: '$frameId', seqId: '$seqId'} }
         │ +─ Vars=
         │ +─ ?seqId <- COLUMN($1)^^xsd:string
         │ +─ ?var100 <- COLUMN($0)^^xsd:string
         │ }
         `─ VirtualGraphMongoDB<virtual://austria> [#9053] {
            +─ Query=
            +─ {$unwind: "$existence"},
            +─ { $match : {$and : [{"existence" : {$eq : "0707"}}, {"frameId" : {$exists : true, $ne : null}}]} },
            +─ {$project: {frameId: '$frameId'} }
            +─ Vars=
            +─ ?images <- TEMPLATE(http://austria.com/images/{frameId/0})
            +─ ?var100 <- COLUMN($0)^^xsd:string
            }
```

---

<div class="post-metadata">

**Author:** ![zachary.whitley](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/zachary.whitley/32/94_2.png) [@zachary.whitley](https://community.stardog.com/u/zachary.whitley)\
**Post date:** [October 26, 2019, 6:35pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/4 "2019-10-26T18:35:45Z")

</div>

It looks like it's breaking it into two separate queries and doing a ServiceJoin which probably isn't very performant. I'm not sure why. You might want to include you mapping if you can so you can avoid a round trip when someone more knowledgable about this takes a look at this. You might also want to add some details about your mongodb setup, schema, indexes, etc

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 26, 2019, 9:06pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/5 "2019-10-26T21:06:02Z")

</div>

Okay.. I have shared mapping file and schema.. let me know if anything else is required

```
Mapping
    FROM JSON {
      "aus":{
        "_id": "?imageId",
        "frameId": "?frameId",
        "frameNo": "?frameNo",
        "seqId": "?seqId", 
        "existence": ["?existence"],
        "edge": ["?edge"],
        "population": ["?population"],
        "nearEdge": ["?nearEdge"],
        "farEdge": ["?farEdge"],
        "centerDirection": ["?centerDirection"],
        "leftDirection":["?leftDirection"],
        "rightDirection": ["?rightDirection"], 
        "dualTruncation":["?dualTruncation"],
        "leftTruncation": ["?leftTruncation"],
        "rightTruncation": ["?rightTruncation"],
        "entropy": "?entropy",
        "roadEntropy": "?roadEntropy"
      }
    }
    TO {
    ?image a :Images;
            :_id ?imageId;
            :frameId ?frameId;
            :frameNo "?frameNo"^^xsd:integer;
            :seqId ?seqId;
            :existence ?existence;
            :edge ?edge;
            :population ?population;
            :nearEdge ?nearEdge;
            :farEdge ?farEdge;
            :centerDirection ?centerDirection;
            :leftDirection ?leftDirection;
            :rightDirection ?rightDirection;
            :dualTruncation ?dualTruncation;
            :leftTruncation ?leftTruncation;
            :rightTruncation ?rightTruncation;
            :entropy "?entropy"^^xsd:integer;
            :roadEntropy "?roadEntropy"^^xsd:integer;
        }
    WHERE {
      BIND (template("http://austria.com/images/{frameId}") AS ?image)
    }

```

Have attached screen shot of schema

 ![schema](https://canada1.discourse-cdn.com/flex030/uploads/stardog/original/1X/595c86a68064761ce9110fdd2244b4a7e5aff0d7.png)

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 28, 2019, 9:39am UTC](https://community.stardog.com/t/query-response-time-very-high/2042/6 "2019-10-28T09:39:03Z")

</div>

Any help on this?  
cc @zachary.whitley

---

<div class="post-metadata">

**Author:** ![PaulJackson](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/pauljackson/32/243_2.png) [@PaulJackson](https://community.stardog.com/u/PaulJackson)\
**Post date:** [October 28, 2019, 1:19pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/7 "2019-10-28T13:19:42Z")

</div>

Hi Fasal,

This happens because `frameId` is not known to be a unique field. `frameId` is the field that's used to create the IRI for `?image`, which in turn is the variable we're joining on in the query.

If `frameId` is not a unique field in the `aus` collection, then Stardog needs this join to create the cross product of `existence` and `seqId` fields per `frameId`.

If this is not what's intended, you have a couple options, depending on your data, what it means, and how you wish to model it.

If the combination of the `_id` and `frameId` fields represent some concept (an image element, say) you could change your model such that each `aus` document represents an "ImageFrame" with an IRI who's template includes both the `_id` and `frameId` fields. See this blog post for a detailed explanation: [Mapping Denormalized Data | Stardog](https://www.stardog.com/blog/mapping-denormalized-data/)

If there is a 1 to 1 relationship between frames and documents (between the `frameId` field and the `_id` field), then you have a couple choices.

1. You could use the `_id` field in place of the `frameId` field in the `image` template.
2. You could indicate to Stardog that `frameId` is a unique field in `aus` collection using the `unique.key.sets` virtual graph property.

In this example, the property would be set to:  
`unique.key.sets=(aus.[].frameId)`

See [Home | Stardog Documentation Latest](https://www.stardog.com/docs/#_setting_unique_keys_manually) for details.

-Paul

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 29, 2019, 3:13pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/8 "2019-10-29T15:13:17Z")

</div>

Hi @PaulJackson,

Thanks for the response..  
frameId is unique in my dataset but as you suggested I tried following things:

1. I did set unique.key.sets=(aus.[].frameId) in my stardog studio virtual graph configuration options under other options as property & value keys..  
Output:- I couldn't find any improvement in the response time of the query.

2. Then i tried using \_id in place of frameId field to create IRI  
`WHERE { BIND (template("http://austria.com/images/{_id}") AS ?image) }`

In this case simple query like

`SELECT * {GRAPH <virtual://austria> { ?image a :Images ; :frameId ?frameId ;}}`

fetches around 44k records in just a second which is what i'm looking for.. But now when i add one more condition to this query I don't get any output.. This happens when I try to add a condition from any of the array fields (in this query existence is an array object)..

`SELECT * {GRAPH <virtual://austria> {?image a :Images ; :frameId ?frameId ; :existence "0707" .}}`

I just see header fields as image and frameId without any data.

Observation: In 2nd case queries are working fine if I query only on int and string fields but when I'm including array objects not able to see any result..

1. Other option you suggested was combination of frameId and \_id for IRI, again with this change query is behaving like 2nd case..

Not sure where i'm going wrong!!!

---

<div class="post-metadata">

**Author:** ![jess](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/jess/32/10_2.png) [@jess](https://community.stardog.com/u/jess)\
**Post date:** [October 29, 2019, 3:16pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/9 "2019-10-29T15:16:20Z")

</div>

Hi Fasal,

The inclusion of `[]` in the `unique.key.sets` property indicates an array.

What was the result of using `_id` in the subject? Can you share the query plan?

---

<div class="post-metadata">

**Author:** ![jess](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/jess/32/10_2.png) [@jess](https://community.stardog.com/u/jess)\
**Post date:** [October 29, 2019, 3:18pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/10 "2019-10-29T15:18:28Z")

</div>

I was wrong about the `[]`. Can you include the query plan for both changes you tried?

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 29, 2019, 3:36pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/11 "2019-10-29T15:36:29Z")

</div>

I have updated my response above @jess please have a re-look

---

<div class="post-metadata">

**Author:** ![jess](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/jess/32/10_2.png) [@jess](https://community.stardog.com/u/jess)\
**Post date:** [October 29, 2019, 3:41pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/12 "2019-10-29T15:41:03Z")

</div>

Hi Fasal,

It sounds like you're describing a case where you query for images where one of the values of the `existence` field is `"0707"`. You used this query as an example:

```auto
SELECT * {GRAPH <virtual://austria> {
?image a :Images ; :frameId ?frameId ; :existence "0707" .}}

```

Can you share the query plan for this query?

What result do you get if you replace the constant value for `existence` with a variable, eg:

```auto
SELECT * {GRAPH <virtual://austria> {
?image a :Images ; :frameId ?frameId ; :existence ?existence .}}

```

What values are bound to the `?existence` variable?

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 29, 2019, 3:53pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/13 "2019-10-29T15:53:26Z")

</div>

@jess query plan

```
   ` Projection(?image, ?frameId) [#9.1K]
    `─ Projection(?image, ?frameId) [#9.1K]
       `─ ServiceJoin [#9.1K]
          +─ VirtualGraphMongoDB<virtual://austria> [#36212] {
          │ +─ Query=
          │ +─ { $match : {$and : [{"_id" : {$exists : true, $ne : null}}, {"frameId" : {$exists : true, $ne : null}}]} },
          │ +─ {$project: {_id: '$_id', frameId: '$frameId'} }
          │ +─ Vars=
          │ +─ ?image <- TEMPLATE(http://austria.com/images/{_id/0})
          │ +─ ?frameId <- COLUMN($1)^^xsd:string
          │ +─ ?var100 <- COLUMN($0)^^xsd:string
          │ }
          `─ VirtualGraphMongoDB<virtual://austria> [#9053] {
             +─ Query=
             +─ {$unwind: "$existence"},
             +─ { $match : {$and : [{"existence" : {$eq : "0707"}}, {"_id" : {$exists : true, $ne : null}}]} },
             +─ {$project: {_id: '$_id'} }
             +─ Vars=
             +─ ?var100 <- COLUMN($0)^^xsd:string
             } `

```

for this query  
SELECT \* {GRAPH virtual://austria {  
?image a :Images ; :frameId ?frameId ; :existence ?existence .}}  
Nothing gets displayed

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 29, 2019, 3:58pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/14 "2019-10-29T15:58:15Z")

</div>

_Deleted!!!!!!!!!!!!!!!!!!!!!!_

---

<div class="post-metadata">

**Author:** ![jess](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/jess/32/10_2.png) [@jess](https://community.stardog.com/u/jess)\
**Post date:** [October 29, 2019, 4:19pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/15 "2019-10-29T16:19:25Z")

</div>

Do you get results for this query:

```auto
?image :existence ?existence

```

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 29, 2019, 4:25pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/16 "2019-10-29T16:25:38Z")

</div>

@jess No results... nothing gets displayed .

query plan follows:

> Projection(?image, ?frameId, ?existence) [#72K]  
> `─ Projection(?image, ?frameId, ?existence) [#72K] `─ ServiceJoin [#72K]  
> +─ VirtualGraphMongoDBvirtual://austria [#54319] {  
> │ +─ Query=  
> │ +─ {$unwind: "$existence"},  
> │ +─ { $match : {$and : [{"\_id" : {$exists : true, $ne : null}}, {"existence" : {$exists : true, $ne : null}}]} }  
> │ +─ Vars=  
> │ +─ ?existence \<- COLUMN($1)^^xsd:string  
> │ +─ ?var100 \<- COLUMN($0)^^xsd:string  
> │ }  
> `─ VirtualGraphMongoDBvirtual://austria [#36212] {  
> +─ Query=  
> +─ { $match : {$and : [{"\_id" : {$exists : true, $ne : null}}, {"frameId" : {$exists : true, $ne : null}}]} },  
> +─ {$project: {\_id: '$\_id', frameId: '$frameId'} }  
> +─ Vars=  
> +─ ?image \<- TEMPLATE([http://austria.com/images/{\_id/0}](http://austria.com/images/%7B_id/0%7D))  
> +─ ?frameId \<- COLUMN($1)^^xsd:string  
> +─ ?var100 \<- COLUMN($0)^^xsd:string  
> }

---

<div class="post-metadata">

**Author:** ![jess](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/jess/32/10_2.png) [@jess](https://community.stardog.com/u/jess)\
**Post date:** [October 29, 2019, 4:27pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/17 "2019-10-29T16:27:40Z")

</div>

```auto
SELECT * {GRAPH <virtual://austria> {
?image :existence ?existence .}}

```

Please use this _exact_ query. Can you share the query plan?

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 29, 2019, 4:31pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/18 "2019-10-29T16:31:43Z")

</div>

Oh okay!! yeah i got the results for

> SELECT \* {GRAPH virtual://austria {  
> ?image :existence ?existence .}}

plan is

> Projection(?image, ?existence) [#54K]  
> `─ VirtualGraphMongoDBvirtual://austria [#54319] {  
> +─ Query=  
> +─ {$unwind: "$existence"},  
> +─ { $match : {$and : [{"\_id" : {$exists : true, $ne : null}}, {"existence" : {$exists : true, $ne : null}}]} }  
> +─ Vars=  
> +─ ?image \<- TEMPLATE([http://austria.com/images/{\_id/0}](http://austria.com/images/%7B_id/0%7D))  
> +─ ?existence \<- COLUMN($1)^^xsd:string  
> }

---

<div class="post-metadata">

**Author:** ![jess](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/jess/32/10_2.png) [@jess](https://community.stardog.com/u/jess)\
**Post date:** [October 29, 2019, 4:34pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/19 "2019-10-29T16:34:57Z")

</div>

Great, and what about if you include the constant:

```auto
SELECT * {GRAPH <virtual://austria> {
?image :existence "0707" .}}

```

---

<div class="post-metadata">

**Author:** ![fasal](https://avatars.discourse-cdn.com/v4/letter/f/b4bc9f/32.png) [@fasal](https://community.stardog.com/u/fasal)\
**Post date:** [October 29, 2019, 4:36pm UTC](https://community.stardog.com/t/query-response-time-very-high/2042/20 "2019-10-29T16:36:53Z")

</div>

Yes records are displayed following is the query plan

> Projection(?image) [#9.1K]  
> `─ VirtualGraphMongoDBvirtual://austria [#9053] {  
> +─ Query=  
> +─ {$unwind: "$existence"},  
> +─ { $match : {$and : [{"existence" : {$eq : "0707"}}, {"\_id" : {$exists : true, $ne : null}}]} },  
> +─ {$project: {\_id: '$\_id'} }  
> +─ Vars=  
> +─ ?image \<- TEMPLATE([http://austria.com/images/{\_id/0}](http://austria.com/images/%7B_id/0%7D))  
> }

[Next page](https://community.stardog.com/t/query-response-time-very-high/2042.md?page=2)