Query with wrong results using rules

Victor_Julio_Ramirez · January 13, 2020, 1:21pm

I have the following rule to detect an increment in the value of the data:

RULE :T4C39B_Increment
IF{
demo:T4C39B sosa:madeObservation ?obs.
?obs a demo:ObservationWithAverage.
?obs sosa:resultTime ?actDate.
?obs demo:hasAVGResult ?actVal.
BIND (?actDate + "-PT15M"^^xsd:duration AS ?pasDate).
demo:T4C39B sosa:madeObservation ?obs1.
?obs1 sosa:resultTime ?pasDate.
?obs1 demo:hasAVGResult ?pasVal.
BIND (?pasVal + 500 AS ?pasVal2).
FILTER(?actVal > ?pasVal2) .
}
THEN {
demo:T4C39B demo:valueIncrement ?actDate.
}

if I run the following query:

SELECT ?actDate
WHERE {
    demo:T4C39B demo:valueIncrement ?actDate
}

The result is:
Query returned 2,922 results in 00:02:41.675

But, if I run a direct query (using the content of the rule):

SELECT ?actDate
WHERE {
    demo:T4C39B sosa:madeObservation ?obs.
	?obs a demo:ObservationWithAverage.
	?obs sosa:resultTime ?actDate.
	?obs demo:hasAVGResult ?actVal.
	BIND (?actDate + "-PT15M"^^xsd:duration AS ?pasDate).
	demo:T4C39B sosa:madeObservation ?obs1.
	?obs1 sosa:resultTime ?pasDate.
	?obs1 demo:hasAVGResult ?pasVal.
	BIND (?pasVal + 500 AS ?pasVal2).
	FILTER(?actVal > ?pasVal2) .
}

the result is:
Query returned 220 results in 00:00:00.490

The correct answer for the query is the last one: 220 results.

I don't know why using a rule there is a huge variation in the results.

I'm attaching the query plans.

Thanks in advance.qp.zip (1.2 KB)

zachary.whitley · January 13, 2020, 2:42pm

Did you enable reasoning when running the direct query?

Victor_Julio_Ramirez · January 13, 2020, 2:57pm

Yes, I try the direct query with and without reasoning, the result is correct. But the excessive amount of records is shown with the simple query:

SELECT ?actDate
WHERE {
    demo:T4C39B demo:valueIncrement ?actDate
}

demo:valueIncrement was defined by the rule.

zachary.whitley · January 13, 2020, 5:10pm

Something definitely looks to be a bit off comparing the query plans. Someone from Stardog would have to say why it's generating the query plans that it is. I'd be curious to know what would happen if you replaced the '^^' in the BINDS with the use of STRDT.

zachary.whitley · January 13, 2020, 7:52pm

Follow on question to the Stardog folks about reading the query plan. How do you read the underscores in the query plan? Blank nodes?

Distinct [#1.6K]
`─ Projection(?actDate) [#1.6K]
   `─ Filter(?kwgetazz > ?gesehxsw) [#1.6K]
      `─ Bind((?wgiisylm + "500"^^xsd:integer) AS ?gesehxsw) ((?actDate + "-PT15M"^^xsd:duration) AS ?htdzrepq) [#3.2K]
         `─ NestedLoopJoin(_) [#3.2K]
            +─ MergeJoin(?xgblycpy) [#1]
            │  +─ MergeJoin(?xgblycpy) [#3.0K]
            │  │  +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?xgblycpy) [#2.7M]
            │  │  `─ Scan[PSOC](?xgblycpy, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?kwgetazz) [#36K]
            │  `─ MergeJoin(?xgblycpy) [#12K]
            │     +─ Scan[POSC](?xgblycpy, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
            │     `─ Scan[PSOC](?xgblycpy, <http://www.w3.org/ns/sosa/resultTime>, ?actDate) [#32.1M]
            `─ MergeJoin(?nasjdpze) [#3.2K]
               +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?nasjdpze) [#2.7M]
               `─ MergeJoin(?nasjdpze) [#36K]
                  +─ Scan[PSC](?nasjdpze, <http://www.w3.org/ns/sosa/resultTime>, _) [#27.3M]
                  `─ Scan[PSOC](?nasjdpze, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?wgiisylm) [#36K]

Specifically the following two lines

              +─ Scan[PSC](?nasjdpze, <http://www.w3.org/ns/sosa/resultTime>, _) [#27.3M]

and

     `─ NestedLoopJoin(_) [#3.2K]

Is it always unambiguous to just be an underscore and not have a more specific identifier?

Victor_Julio_Ramirez · January 16, 2020, 3:31pm

The visible difference in the execution plans is:

Using rule:

  `─ Bind((?wgiisylm + "500"^^xsd:integer) AS ?gesehxsw) ((?actDate + "-PT15M"^^xsd:duration) AS ?htdzrepq) [#3.2K]
     `─ NestedLoopJoin(_) [#3.2K]

*that kind of BIND is unknown for me.

With the query:

   `─ Bind((?pasVal + "500"^^xsd:integer) AS ?pasVal2) [#3.0K]
      `─ BindJoin(?pasDate) [#3.0K]
         +─ Bind((?actDate + "-PT15M"^^xsd:duration) AS ?pasDate) [#1]

As Zachary Whitley pointed out, there is a strange (or unknown) behaviour in the rule query plan.

Is there any Stardog engineer that can help me with this?

Thanks.

zachary.whitley · January 16, 2020, 4:12pm

I've included the query plans inline and replaced the variable names in the rule to match the query so it's easier to compare.

Query

 Distinct [#1.6K]
`─ Projection(?actDate) [#1.6K]
   `─ Filter(?actVal > ?pasVal2) [#1.6K]
      `─ Bind((?pasVal + "500"^^xsd:integer) AS ?pasVal2) ((?actDate + "-PT15M"^^xsd:duration) AS ?pasDate) [#3.2K]
         `─ NestedLoopJoin(_) [#3.2K]
            +─ MergeJoin(?obs) [#1]
            │  +─ MergeJoin(?obs) [#3.0K]
            │  │  +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs) [#2.7M]
            │  │  `─ Scan[PSOC](?obs, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?actVal) [#36K]
            │  `─ MergeJoin(?obs) [#12K]
            │     +─ Scan[POSC](?obs, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
            │     `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?actDate) [#32.1M]
            `─ MergeJoin(?obs1) [#3.2K]
               +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs1) [#2.7M]
               `─ MergeJoin(?obs1) [#36K]
                  +─ Scan[PSC](?obs1, <http://www.w3.org/ns/sosa/resultTime>, _) [#27.3M]
                  `─ Scan[PSOC](?obs1, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?pasVal) [#36K]

Rule

Projection(?actDate) [#1.5K]
`─ Filter(?actVal > ?pasVal2) [#1.5K]
   `─ Bind((?pasVal + "500"^^xsd:integer) AS ?pasVal2) [#3.0K]
      `─ BindJoin(?pasDate) [#3.0K]
         +─ Bind((?actDate + "-PT15M"^^xsd:duration) AS ?pasDate) [#1]
         │  `─ MergeJoin(?obs) [#1]
         │     +─ MergeJoin(?obs) [#3.0K]
         │     │  +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs) [#2.7M]
         │     │  `─ Scan[PSOC](?obs, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?actVal) [#36K]
         │     `─ MergeJoin(?obs) [#12K]
         │        +─ Scan[POSC](?obs, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
         │        `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?actDate) [#32.1M]
         `─ MergeJoin(?obs1) [#3.0K]
            +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs1) [#2.7M]
            `─ MergeJoin(?obs1) [#36K]
               +─ Scan[PSOC](?obs1, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?pasVal) [#36K]
               `─ Scan[PSOC](?obs1, <http://www.w3.org/ns/sosa/resultTime>, ?pasDate) [#32.1M]

zachary.whitley · January 16, 2020, 4:17pm

Is that computing the cross join and then computing the bindings from the result? I'm not familiar with the bind join. Can you provide some info on that?

@Victor_Julio_Ramirez if you don't mine I'm using this as an opportunity to get a better understanding of reading Stardog query plans.

Victor_Julio_Ramirez · January 16, 2020, 4:34pm

Thanks @zachary.whitley for your interest, I don't care if this is an opportunity to get an answer that satisfies both of us.

pavel · January 16, 2020, 4:35pm

I haven't yet looked into the rest of the issue but here's a quick note re: underscores in the plans:
The _ in NestedLoopJoin(_) means that there's no join key, i.e. it's the full Cartesian product. The _ in Scan[PSC](?nasjdpze, <http://www.w3.org/ns/sosa/resultTime>, _) means that the variable in the object position is not used anywhere else in the query so the engine does not have to compute its values (and it may even aggregate partials results which only differ in values of that variable).

We'll look at the rest shortly,
Pavel

pavel · January 16, 2020, 5:31pm

Hi Victor,

How many result will you get for the following small variation of your query (no reasoning):

SELECT ?actDate ?pasDate
WHERE {
    demo:T4C39B sosa:madeObservation ?obs.
	?obs a demo:ObservationWithAverage.
	?obs sosa:resultTime ?actDate.
	?obs demo:hasAVGResult ?actVal.
	BIND (?actDate + "-PT15M"^^xsd:duration AS ?pasDate).
	demo:T4C39B sosa:madeObservation ?obs1.
	?obs1 sosa:resultTime ?pasDate.
	?obs1 demo:hasAVGResult ?pasVal.
	BIND (?pasVal + 500 AS ?pasVal2).
	FILTER(?actVal > ?pasVal2) .
}

Please show the plan, too.

Thanks,
Pavel

pavel · January 16, 2020, 7:41pm

OK, I think we know the issue here. I got confused initially because the plans are swapped in the Zach's message above. The direct query plan has BindJoin(?pasDate) and the rule plan has NestedLoopJoin(_), not the other way around.

The former is correct and the latter is wrong. There's a known issue that Stardog assumes that variables introduced in BINDs in the body of a rule are only used in the head. That essentially means that the BIND in the body floats to the bottom of the body, which is not semantics-preserving. You can see it in the rule plan: the BIND is at the very top.

We'll try to fix this in one of the next releases but for now it might be possible to split the rule on two s.t. the BIND would only be used in the head of the auxiliary rule.

Best,
Pavel

Victor_Julio_Ramirez · January 17, 2020, 8:41am

Thanks @pavel, I'm glad knowing that it is not an issue in the rule formulation. Is there any way that you can inform us in which upcoming Stardog release the issue has been fixed?

pavel · January 17, 2020, 8:43am

The Stardog issue number is #8407. If you see that in the release notes of a future release, then it should be fixed.

Cheers,
Pavel

Victor_Julio_Ramirez · January 17, 2020, 11:22am

@pavel, I tried your approach : split the rule in two. The auxiliary rule is:

RULE :obs_pasDate
IF{
?obs a demo:ObservationWithAverage;
	sosa:resultTime ?actDate;
	demo:hasAVGResult ?actVal.
BIND (?actDate + "-PT15M"^^xsd:duration AS ?pasDate).
}
THEN {
?obs demo:pastResultTime ?pasDate.
}

But when testing the auxiliary rule, I saw another problem. I tried two queries that, I think, have the same result, but they don't.

First query:

select *
where {
demo:T4C39B sosa:madeObservation ?obs.
?obs a demo:ObservationWithAverage;
	sosa:resultTime ?actDate;
	demo:hasAVGResult ?actVal;
	demo:pastResultTime ?pasDate.
demo:T4C39B sosa:madeObservation ?obs1.
?obs1 a demo:ObservationWithAverage;
	sosa:resultTime ?pasDate;
	demo:hasAVGResult ?pasVal.
FILTER(?actVal > ?pasVal+500) .
}
LIMIT 1

First query plan:

Slice(offset=0, limit=1) [#1]
`─ Distinct [#1]
   `─ Projection(?obs, ?actDate, ?actVal, ?pasDate, ?obs1, ?pasVal) [#1]
      `─ Bind((?bxzisfru + "-PT15M"^^xsd:duration) AS ?pasDate) [#1]
         `─ MergeJoin(?obs1) [#1]
            +─ Scan[PSOC](?obs1, <http://www.w3.org/ns/sosa/resultTime>, ?pasDate) [#32.1M]
            `─ Filter(?actVal > (?pasVal + "+500"^^xsd:integer)) [#1]
               `─ Sort(?obs1) [#1]
                  `─ NestedLoopJoin(_) [#1]
                     +─ MergeJoin(?obs) [#1]
                     │  +─ MergeJoin(?obs) [#12K]
                     │  │  +─ Scan[POSC](?obs, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
                     │  │  `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?actDate) [#32.1M]
                     │  `─ MergeJoin(?obs) [#3.0K]
                     │     +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs) [#2.7M]
                     │     `─ MergeJoin(?obs) [#36K]
                     │        +─ Scan[PSOC](?obs, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?actVal) [#36K]
                     │        `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?bxzisfru) [#32.1M]
                     `─ MergeJoin(?obs1) [#1]
                        +─ Scan[POSC](?obs1, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
                        `─ MergeJoin(?obs1) [#3.0K]
                           +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs1) [#2.7M]
                           `─ Scan[PSOC](?obs1, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?pasVal) [#36K]

Second query:

select *
where {
demo:T4C39B sosa:madeObservation ?obs.
?obs a demo:ObservationWithAverage;
	sosa:resultTime ?actDate;
	demo:hasAVGResult ?actVal;
	demo:pastResultTime ?pasDate.
demo:T4C39B sosa:madeObservation ?obs1.
?obs1 a demo:ObservationWithAverage;
	sosa:resultTime ?pasDate1;
	demo:hasAVGResult ?pasVal.
FILTER(?actVal > ?pasVal+500 && ?pasDate = ?pasDate1) .
}
LIMIT 1

Second query plan:

Slice(offset=0, limit=1) [#1]
`─ Distinct [#1]
   `─ Projection(?obs, ?actDate, ?actVal, ?pasDate, ?obs1, ?pasDate1, ?pasVal) [#1]
      `─ Filter((?pasDate = ?pasDate1 && ?actVal > (?pasVal + "+500"^^xsd:integer))) [#1]
         `─ NestedLoopJoin(_) [#1]
            +─ Bind((?shypgopb + "-PT15M"^^xsd:duration) AS ?pasDate) [#1]
            │  `─ MergeJoin(?obs) [#1]
            │     +─ MergeJoin(?obs) [#12K]
            │     │  +─ Scan[POSC](?obs, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
            │     │  `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?actDate) [#32.1M]
            │     `─ MergeJoin(?obs) [#3.0K]
            │        +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs) [#2.7M]
            │        `─ MergeJoin(?obs) [#36K]
            │           +─ Scan[PSOC](?obs, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?actVal) [#36K]
            │           `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?shypgopb) [#32.1M]
            `─ MergeJoin(?obs1) [#1]
               +─ MergeJoin(?obs1) [#3.0K]
               │  +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs1) [#2.7M]
               │  `─ Scan[PSOC](?obs1, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?pasVal) [#36K]
               `─ MergeJoin(?obs1) [#12K]
                  +─ Scan[POSC](?obs1, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
                  `─ Scan[PSOC](?obs1, <http://www.w3.org/ns/sosa/resultTime>, ?pasDate1) [#32.1M]

As you can see, in the first query I use the same variable (?pasDate) to make the relation between the two observations. In the second I use two different variables (?pasDate and ?pasDate1) but they are compared in the filter (FILTER(?actVal > ?pasVal+500 && ?pasDate = ?pasDate1) .). The right answer is provided by the latter query.

Is this another issue or is a query definition problem?

Thanks and sorry for the huge post.

pavel · January 17, 2020, 11:54am

Not necessarily. These queries are not equivalent because in SPARQL term equality isn't the same as term identity. In other words, when you use the same variable twice in two patterns, like in the first query, both patterns must bind it to exactly the same RDF node. But when you use two different variables ?pasDate and ?pasDate1 and say FILTER(?pasDate = ?pasDate1), that means that the variables can be bound to different RDF nodes (literals) but which are considered equal. The classical example would be something like "1"^^xsd:int and "1"^^xsd:integer.

So based on your example I'd hypothesize that the 2nd query returns more results exactly because equality is more relaxed than identity. It's of course slower to evaluate though.

If you print both ?pasDate and ?pasDate1 you might be able to see non-identical values.

Best,
Pavel

Victor_Julio_Ramirez · January 17, 2020, 1:37pm

Hi @pavel, that is not the behaviour of the queries, I'm attaching the results with limit 10:

Query 1:

Query 2:

Query 2 provides the right answer (220 records without limit), but query 1 is matching wrongly the resultTime due that it is not possible that the same observation (demo:T4C39Bdate20190103obs1) has 10 different resultTimes resulting in more than 200.000 records without limit.

pavel · January 17, 2020, 3:44pm

Yes, I see what you mean and you're right -- this is not the issue I described (though what I said is still correct re: filter equality vs same variable).

Looks like the BIND in rules issue still creates troubles. What happens if you run the following:

select *
where {
demo:T4C39B sosa:madeObservation ?obs.
?obs a demo:ObservationWithAverage;
	sosa:resultTime ?actDate;
	demo:hasAVGResult ?actVal .

{ select * { ?obs demo:pastResultTime ?pasDate }}

demo:T4C39B sosa:madeObservation ?obs1.
?obs1 a demo:ObservationWithAverage;
	sosa:resultTime ?pasDate;
	demo:hasAVGResult ?pasVal.
FILTER(?actVal > ?pasVal+500) .
}

Could you show the plan too?

Thanks,
Pavel

Victor_Julio_Ramirez · January 17, 2020, 4:07pm

@pavel, your query returns the right results and is extremely fast!

Query 3:
query3

I'm attaching the 3 query execution plans:

Query 1 exec plan (wrong results):

Distinct [#1]
`─ Projection(?obs, ?actDate, ?actVal, ?pasDate, ?obs1, ?pasVal) [#1]
   `─ Bind((?riuscvic + "-PT15M"^^xsd:duration) AS ?pasDate) [#1]
      `─ MergeJoin(?obs1) [#1]
         +─ Scan[PSOC](?obs1, <http://www.w3.org/ns/sosa/resultTime>, ?pasDate) [#32.1M]
         `─ Filter(?actVal > (?pasVal + "+500"^^xsd:integer)) [#1]
            `─ Sort(?obs1) [#1]
               `─ NestedLoopJoin(_) [#1]
                  +─ MergeJoin(?obs) [#1]
                  │  +─ MergeJoin(?obs) [#12K]
                  │  │  +─ Scan[POSC](?obs, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
                  │  │  `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?actDate) [#32.1M]
                  │  `─ MergeJoin(?obs) [#3.0K]
                  │     +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs) [#2.7M]
                  │     `─ MergeJoin(?obs) [#36K]
                  │        +─ Scan[PSOC](?obs, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?actVal) [#36K]
                  │        `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?riuscvic) [#32.1M]
                  `─ MergeJoin(?obs1) [#1]
                     +─ Scan[POSC](?obs1, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
                     `─ MergeJoin(?obs1) [#3.0K]
                        +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs1) [#2.7M]
                        `─ Scan[PSOC](?obs1, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?pasVal) [#36K]

Query 2 exec plan (right results but slower):

Distinct [#1]
`─ Projection(?obs, ?actDate, ?actVal, ?pasDate, ?obs1, ?pasDate1, ?pasVal) [#1]
   `─ Filter((?pasDate = ?pasDate1 && ?actVal > (?pasVal + "+500"^^xsd:integer))) [#1]
      `─ NestedLoopJoin(_) [#1]
         +─ Bind((?uaddryvr + "-PT15M"^^xsd:duration) AS ?pasDate) [#1]
         │  `─ MergeJoin(?obs) [#1]
         │     +─ MergeJoin(?obs) [#12K]
         │     │  +─ Scan[POSC](?obs, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
         │     │  `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?actDate) [#32.1M]
         │     `─ MergeJoin(?obs) [#3.0K]
         │        +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs) [#2.7M]
         │        `─ MergeJoin(?obs) [#36K]
         │           +─ Scan[PSOC](?obs, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?actVal) [#36K]
         │           `─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?uaddryvr) [#32.1M]
         `─ MergeJoin(?obs1) [#1]
            +─ MergeJoin(?obs1) [#3.0K]
            │  +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs1) [#2.7M]
            │  `─ Scan[PSOC](?obs1, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?pasVal) [#36K]
            `─ MergeJoin(?obs1) [#12K]
               +─ Scan[POSC](?obs1, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
               `─ Scan[PSOC](?obs1, <http://www.w3.org/ns/sosa/resultTime>, ?pasDate1) [#32.1M]

Query 3 exec plan (right results and the fastest):

Distinct [#1]
`─ Projection(?obs, ?actDate, ?actVal, ?pasDate, ?obs1, ?pasVal) [#1]
   `─ Filter(?actVal > (?pasVal + "+500"^^xsd:integer)) [#1]
      `─ HashJoin(?pasDate) [#1]
         +─ MergeJoin(?obs1) [#1]
         │  +─ MergeJoin(?obs1) [#12K]
         │  │  +─ Scan[POSC](?obs1, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
         │  │  `─ Scan[PSOC](?obs1, <http://www.w3.org/ns/sosa/resultTime>, ?pasDate) [#32.1M]
         │  `─ MergeJoin(?obs1) [#3.0K]
         │     +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs1) [#2.7M]
         │     `─ Scan[PSOC](?obs1, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?pasVal) [#36K]
         `─ MergeJoin(?obs) [#1]
            +─ Scan[PSOC](?obs, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, ?actVal) [#36K]
            `─ MergeJoin(?obs) [#1]
               +─ Scan[POSC](?obs, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
               `─ MergeJoin(?obs) [#1]
                  +─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?actDate) [#32.1M]
                  `─ MergeJoin(?obs) [#1]
                     +─ Scan[SPOC](<http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#T4C39B>, <http://www.w3.org/ns/sosa/madeObservation>, ?obs) [#2.7M]
                     `─ Projection(?obs, ?pasDate) [#12K]
                        `─ Bind((?npmdbixb + "-PT15M"^^xsd:duration) AS ?pasDate) [#12K]
                           `─ MergeJoin(?obs) [#12K]
                              +─ Scan[PSOC](?obs, <http://www.w3.org/ns/sosa/resultTime>, ?npmdbixb) [#32.1M]
                              `─ MergeJoin(?obs) [#36K]
                                 +─ Scan[POSC](?obs, rdf:type, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#ObservationWithAverage>) [#36K]
                                 `─ Scan[PSC](?obs, <http://bdi.si.ehu.es/bdi/ontologies/ExtruOnt/demo#hasAVGResult>, _) [#30K]

pavel · January 17, 2020, 4:15pm

OK, I think this confirms what we thought the issue is. BINDs in rules sometimes slip to the bottom of its scope in the query and that breaks the semantics. By introducing a subquery (i.e. an extra scope) I forced the BIND to stay where it needs to be -- on top of the patterns in the rule's body.

We'll try to improve this as soon as feasible.

Pavel

Topic		Replies	Views
Rule latest timestamp Support	6	388	July 3, 2018
Odd filter behavior Support	5	370	June 21, 2018
Reasoning query returns unexpected results Support	4	457	May 16, 2018
Unexpected results with SERVICE clause Bug	10	456	March 6, 2019
Query time blows out > 1000x moving from 5.0.3 => 5.0.4 Bug	7	601	November 8, 2017

Query with wrong results using rules

Related Topics