Cannot get geof:distance calculated

[dlbis (Daniel Lavoie)]

Hi,

I have some geo:Geometry resources with wgs:lat and wgs:long. And I am trying to get the distance between them. However, nothing is returned. Any clue why?

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX wgs: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
PREFIX unit: <http://qudt.org/vocab/unit#>

select ?prop1 ?type ?prop1Lat ?prop1Long (datatype(?prop1Lat) as ?lat1Type) (datatype(?prop1Long) as ?long1Type)
?prop2 ?prop2Lat ?prop2Long (datatype(?prop2Lat) as ?lat2Type) (datatype(?prop2Long) as ?long2Type)
?distance
from <http://cdpq.com/rcagraph_geo>
where {
{ select * where
{?prop1 rdf:type ?type; wgs:lat ?prop1Lat; wgs:long ?prop1Long . } limit 1
}
{ select * where
{?prop2 rdf:type geo:Geometry; wgs:lat ?prop2Lat; wgs:long ?prop2Long . } limit 1 offset 10}
optional { bind(geof:distance(?prop1, ?prop2, unit:Meter) as ?distance) }
}

[stephen]

Hi Daniel,

First off, have you taken a look at our Geospatial primer? You probably have, but I would be remiss not to have pointed it out to you if you haven't.

That said, can you share the data you are using with which you see this issue?

[dlbis (Daniel Lavoie)]

Hi Stephen,

Yes, I have read multiple times the primer and the geospatial section in the manual. For the data I am not sure if I can share it with you. I will verify.

In the meantime, here is a sample showing one Geometry instance (which is also a Property) with its lat and long wgs properties.

Thanks
Daniel

[dlbis (Daniel Lavoie)]

Another view of the same sample:

[stephen]

Daniel,

I threw together an incredibly small dataset in an attempt to reproduce your problem, and I wasn't able to:

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX wgs: <http://www.w3.org/2003/01/geo/wgs84_pos#>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
PREFIX unit: <http://qudt.org/vocab/unit#>
PREFIX : <https://community.stardog.com/t/cannot-get-geof-distance-calculated/1576#>

insert data {
    graph <http://cdpq.com/rcagraph_geo> {
        :geom1 a geo:Geometry ;
            wgs:lat 45.4587 ;
            wgs:long -73.8646 .
        :geom2 a geo:Geometry ;
            wgs:lat 45.463497 ;
            wgs:long -73.70412 .
        }
}

select ?distance 
from <http://cdpq.com/rcagraph_geo>
where {
    BIND(geof:distance(:geom1, :geom2, unit:Meter) as ?distance)
}

Result: ?distance = 12527.43347840403

Do you by any chance not have the spatial indexing enabled on your database? Studio will show that right at the top of the database page. If it's not enabled, you can offline the db, enable it, and online the db again, at which point the geo functions should start working.

If that's not the case then perhaps something in your query is using the wrong named graph(s) or introducing some other level of misdirection.

Edit: I rewrote your query to be as simple as possible and was able to select the appropriate values:

[dlbis (Daniel Lavoie)]

Thanks again Stephen. Yes, the spatial indexing is ON.

[dlbis (Daniel Lavoie)]

Hi Stephen,

To verify my setup, I have loaded your test data and executed your query. And the result was fine. Conclusion: my database and my ontologies are ok. But something else is wrong.

The source of my problem seems to be the data itself. In fact, my wgs source coordinates are in string format. Before loading them in the wgs property triples, I have converted them in xsd:float like in the primer example in the Representing Single Point section. But as I have mentionned, I was not able to get the distance calculated that way.

After having successfully executed your query with your data, I have noticed that you have used xsd:decimal instead of xsd:float. So I have reloaded my data in decimal instead. But that has not worked neither.

Then, I have noticed that my string coordinated were having a lot of decimals. Is there a limit of decimal positions supported for the wgs lat and long? Should I round them?

In the following example, the coordinates have up to 15 decimal positions. Is that too much and causing the problem?

[stephen]

The decimal positions don't seem to matter, nor does the conversion from string to decimal. At least not in my simple data set:

insert data {
    graph <http://cdpq.com/rcagraph_geo2> {
        :geom1 a geo:Geometry ;
            :lat "45.463497600000004" ;
            :lon "-73.70411709999999" .
        :geom2 a geo:Geometry ;
            :lat "45.476427600300004" ;
            :lon "-73.71412709959929" .
        }
    };

insert {
    graph <http://cdpq.com/rcagraph_geo2> {
        ?geom wgs:lat ?xlat;
            wgs:long ?xlong .
    }
}
where {
    graph <http://cdpq.com/rcagraph_geo2> {
        ?geom :lat ?lat ; :lon ?long .
        BIND(xsd:decimal(?lat) as ?xlat) .
        BIND(xsd:decimal(?long) as ?xlong) .
    }
}

[dlbis (Daniel Lavoie)]

Thanks again for your help Stephen,

I did a few other tests and got something new. I have used a combination of your test dataset and mine. And I have tried to calculate the distance 3 different ways:
(1) Using two variables ?geom1 and ?geom2
(2) Directly referring to your dataset IRI short names using the prefix
(3) And doing the same with the IRI of my dataset

The results were as following:
(1) No distance obtained, as before for my dataset but even for yours.
(2) Distance obtained as before for your dataset.
(3) I had to put that line in comment because it was crashing with QueryEval: -1 and the following message in the log.

ERROR 2019-04-08 20:04:05,947 [stardog-user-21] com.complexible.stardog.protocols.http.server.StardogHttpServiceLoader:accept(235): An unexpected exception was handled by the server
com.stardog.stark.query.QueryExecutionFailure: -1
at com.complexible.common.rdf.query.IteratorAsTupleQueryResult.hasNext(IteratorAsTupleQueryResult.java:82) ~[stardog-utils-rdf-6.1.2.jar:?]
at com.stardog.stark.query.ClosingSpliterator.forEachRemaining(ClosingSpliterator.java:37) ~[stardog-stark-query-api-6.1.2.jar:?]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_201]
at com.stardog.stark.query.io.QueryResultWriters.write(QueryResultWriters.java:142) ~[stardog-stark-query-io-6.1.2.jar:?]
at com.stardog.stark.query.io.QueryResultWriters.write(QueryResultWriters.java:127) ~[stardog-stark-query-io-6.1.2.jar:?]
at com.complexible.stardog.protocols.http.server.ProtocolUtils.writeTupleResponse(ProtocolUtils.java:579) ~[stardog-protocols-http-server-6.1.2.jar:?]
at com.complexible.stardog.protocols.http.server.ProtocolUtils.executeReadQuery(ProtocolUtils.java:489) ~[stardog-protocols-http-server-6.1.2.jar:?]
at com.complexible.stardog.protocols.http.server.ProtocolUtils.executeReadQuery(ProtocolUtils.java:474) ~[stardog-protocols-http-server-6.1.2.jar:?]
at com.complexible.stardog.protocols.http.server.SPARQLProtocol.executeQuery(SPARQLProtocol.java:127) ~[stardog-protocols-http-server-6.1.2.jar:?]
at com.complexible.stardog.protocols.http.server.SPARQLProtocol.post(SPARQLProtocol.java:91) ~[stardog-protocols-http-server-6.1.2.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-6.1.2.jar:?]
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.2.3.jar:1.2.3]
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.2.3.jar:1.2.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1

Conclusions:
Can we use or not variables as argument in the distance function or do we have to use IRI?
What IRI formats are supported?

PREFIX owl: <http://www.w3.org/2002/07/owl#&gt;

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#&gt;

PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#&gt;

PREFIX xsd: <XML Schema;

PREFIX onto: <http://www.cdpq.com/ontology#&gt;

PREFIX wgs: <http://www.w3.org/2003/01/geo/wgs84_pos#&gt;

PREFIX geo: <http://www.opengis.net/ont/geosparql#&gt;

PREFIX geof: <http://www.opengis.net/def/function/geosparql/&gt;

PREFIX unit: <http://qudt.org/vocab/unit#&gt;

PREFIX : <Cannot get geof:distance calculated;

PREFIX geom: <http://www.cdpq.com/PropertyGeom/&gt;

select ?geom1 ?lat1 ?long1 ?geom2 ?lat2 ?long2

?distance ?distanceSDdataset ?distanceCDPQdataset

from <http://cdpq.com/rcagraph_geo&gt;

from <http://cdpq.com/rcagraph_geo_sd&gt;

where {

{ { select * where

{?geom1 wgs:lat ?lat1; wgs:long ?long1 .

} limit 1 offset 10 }

{ select * where

{?geom2 wgs:lat ?lat2; wgs:long ?long2 .

} limit 1 offset 20 } }

union

{

{ values(?geom1) {(:geom1)}

?geom1 wgs:lat ?lat1; wgs:long ?long1 }

{ values(?geom2) {(:geom2)}

?geom2 wgs:lat ?lat2; wgs:long ?long2 }

}

optional { bind(geof:distance(?geom1, ?geom2, unit:Meter) as ?distance) }

optional { bind(geof:distance(:geom1, :geom2, unit:Meter) as ?distanceSDdataset) }

#optional { bind(geof:distance(geom:6b8ebc9f90e85ffce1d722702a132a75, geom:d2026ee617b6ca0ba19430e3eea7bee9, unit:Meter) as ?distanceCDPQdataset) }

[dlbis (Daniel Lavoie)]

Hi Stephen,

I have done more tests this morning.

(1) On the first execution, I have reproduced your other test with your small dataset. And the result was fine.
(2) On the second execution, I have modified the previous query in (1) and added an optional clause for the bind statement in which the distance is calculated. The distance was not calculated that way however.
(3) On the third execution, I have modified the previous query in (1) and replaced your small dataset graph with my data in which I have about 250 geo:Geometry instances. Nothing was returned.

(4) On the fourth execution, I have modified the previous query in (3) and added an optional clause for the bind statement in which the distance is calculated. The distance was not calculated that way.

In the execution (4) results, you can see that my Geometry instances and wgs coordinates seem to be ok. And in one of your previous comments, you have suggested that something in my query was misdirecting SD but now I am using the same query as you did. So, I am left to think that something in my data is not correct.

Any clue what else I could check?

Also, why isn't the distance calculated in execution (2)?

Thanks
Daniel

[lorenz_b]

What is the reason of putting it inside an OPTIONAL? An OPTIONAL clause is evaluated separately and then via a left-outer join put together with the other BGP, so the BIND inside the OPTIONAL is not aware of any binding of the variables at all.

[dlbis (Daniel Lavoie)]

I am using the same query with and without optional. First without, to try to obtain the distance calculated. And when I am not getting it, I am adding optional to check my input. That's why.

Thanks

[stephen]

lorenz_b is correct here; adding the OPTIONAL to a bind is completely unneeded and in fact could just cause issues. The bind will either bind, or it won't; the OPTIONAL doesn't really change anything.

I am still unable to reproduce this issue, as I'm getting distances calculated for all the points. Could you share your Stardog version, stardog.properties file, and stardog-admin metadata get myDb output for the database?

[dlbis (Daniel Lavoie)]

I have solved my problem but it was hard to do. In fact, I was using an INSERT syntax that does not seem to work for geospatial even if it is working for the standard query. I cannot explain why but if someone can explain, I would appreciate.

The triples produced with both insert statements are exactly the same. I am sure that the two inserts are not exactly doing the same thing however. Internally, there must be a diiference but what exactly? I have included that comparison at the end of this text.

(1) How I have first inserted my geospatial triples (this is for Geometry but I was doing lat and long the same way). With those triples I was not able to calculate the distance.

with <http://cdpq.com/rcagraph_geo_c2&gt;
insert
{?s ?p ?o}
using <http://cdpq.com/rcagraph&gt;
where {
values(?city) {("Montreal")}
values(?p) {(rdf:type)}
values(?o) {(geo:Geometry)}
?prop onto:Property-cityName ?city .
?prop onto:Property-id ?id .
bind(iri(concat("http://www.cdpq.com/PropertyGeom/", str(?id))) as ?s)

}

(2) How I have finally inserted my geospatial triples and got the distance calculated.

insert { graph http://cdpq.com/rcagraph_geo_c4 { ?s ?p ?o} }
where {
graph http://cdpq.com/rcagraph
{
values(?city) {("Montreal")}
values(?p) {(rdf:type)}
values(?o) {(geo:Geometry)}
?prop onto:Property-cityName ?city .
?prop onto:Property-id ?id .
bind(iri(concat("http://www.cdpq.com/PropertyGeom/", str(?id))) as ?s)
}
}

(3) The tiples comparison. For graph c2, I was not able to get the distance calculated but for c4, I can.

[dlbis (Daniel Lavoie)]

My problem seems so far to be linked to the type of triple load I am using. In summary, direct load of triples is working (INSERT DATA or LOAD command in the CLI) but INSERT using a SELECT does not work. However, my database was first created without the spatial indexing and that might also be part of the problem.

I will recreate my database with spatial indexing from the start to confirm, but now, I just don't have the time. And to solve my problem, I have exported the triples inserted using INSERT SELECT and loaded them with the CLI. That way, the spatial index was updated. Hard to believe, but still true!

[serge (Serge Colle - Senior Solution Architect)]

I think this issue and the issue mentioned in Geo Spatcial query return Eval Query -1 Error where geospatial loaded via virtual graph do not work are due to the same root cause.