Hi, I just recently started with virtual graphs in Stardog. At first, I'm trying out the direct mapping method, before switching to SMS or R2RML, to get an idea of the data in a MySQL DB.
However, when I try the following SPARQL query without LIMIT:
I receive an error in Stardog Studio: QueryEval: Operation not supported. When I set a LIMIT of 15 it works, but with a LIMIT of 20 or higher it fails again. Similar for CONSTRUCT queries to quickly grasp the virtualized RDF, I get the same error when I don't set any LIMIT (it works though if I set "any" LIMIT, e.g. LIMIT 10000)
Hi Jess! I looked into the stardog.log file (I noticed that nothing is added here from actions done via Stardog Studio. Running with Stardog --web-console flag. Not sure if this is intended behaviour).
I found the following for my SELECT query without LIMIT:
ERROR 2019-06-25 15:09:06,420 [stardog-user-5] com.complexible.stardog.protocols.http.server.StardogHttpServiceLoader:accept(235): An unexpected exception was handled by the server
com.stardog.stark.query.QueryExecutionFailure: Operation not supported.
at com.complexible.common.rdf.query.IteratorAsTupleQueryResult.hasNext(IteratorAsTupleQueryResult.java:82) ~[stardog-utils-rdf-6.2.2.jar:?]
at com.stardog.stark.query.ClosingSpliterator.forEachRemaining(ClosingSpliterator.java:37) ~[stardog-stark-query-api-6.2.2.jar:?]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_121]
at com.stardog.stark.query.io.QueryResultWriters.write(QueryResultWriters.java:142) ~[stardog-stark-query-io-6.2.2.jar:?]
at com.stardog.stark.query.io.QueryResultWriters.write(QueryResultWriters.java:127) ~[stardog-stark-query-io-6.2.2.jar:?]
at com.complexible.stardog.protocols.http.server.ProtocolUtils.writeTupleResponse(ProtocolUtils.java:571) ~[stardog-protocols-http-server-6.2.2.jar:?]
at com.complexible.stardog.protocols.http.server.ProtocolUtils.executeReadQuery(ProtocolUtils.java:481) ~[stardog-protocols-http-server-6.2.2.jar:?]
at com.complexible.stardog.protocols.http.annex.QueryPanelService.executeQuery(QueryPanelService.java:162) ~[stardog-webconsole-annex-6.2.2.jar:?]
at com.complexible.stardog.protocols.http.annex.QueryPanelService.post(QueryPanelService.java:113) ~[stardog-webconsole-annex-6.2.2.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-6.2.2.jar:?]
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.2.3.jar:1.2.3]
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.2.3.jar:1.2.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
Caused by: com.complexible.stardog.plan.eval.operator.OperatorException: Operation not supported.
at com.complexible.stardog.virtual.VirtualGraphIterator.computeNext(VirtualGraphIterator.java:153) ~[stardog-virtual-core-6.2.2.jar:?]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) ~[guava-26.0-jre.jar:?]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) ~[guava-26.0-jre.jar:?]
at com.complexible.stardog.virtual.StatementToSolutionIterator.hasNext(StatementToSolutionIterator.java:47) ~[stardog-virtual-core-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.impl.ServiceOperatorImpl.computeNext(ServiceOperatorImpl.java:89) ~[stardog-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.impl.ServiceOperatorImpl.computeNext(ServiceOperatorImpl.java:34) ~[stardog-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.tryToComputeNext(AbstractSkippingIterator.java:143) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.hasNext(AbstractSkippingIterator.java:130) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.impl.SingleProjectionOp.computeNext(SingleProjectionOp.java:82) ~[stardog-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.impl.SingleProjectionOp.computeNext(SingleProjectionOp.java:29) ~[stardog-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.tryToComputeNext(AbstractSkippingIterator.java:143) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.hasNext(AbstractSkippingIterator.java:130) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.util.AutoCloseOperator.computeNext(AutoCloseOperator.java:112) ~[stardog-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.util.AutoCloseOperator.computeNext(AutoCloseOperator.java:25) ~[stardog-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.tryToComputeNext(AbstractSkippingIterator.java:143) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.hasNext(AbstractSkippingIterator.java:130) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.util.OpBasedBindingSetIteration.computeNext(OpBasedBindingSetIteration.java:116) ~[stardog-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.util.OpBasedBindingSetIteration.computeNext(OpBasedBindingSetIteration.java:39) ~[stardog-6.2.2.jar:?]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) ~[guava-26.0-jre.jar:?]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) ~[guava-26.0-jre.jar:?]
at com.complexible.common.rdf.query.IteratorAsTupleQueryResult.hasNext(IteratorAsTupleQueryResult.java:78) ~[stardog-utils-rdf-6.2.2.jar:?]
... 14 more
Caused by: com.mysql.cj.jdbc.exceptions.OperationNotSupportedException: Operation not supported.
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:116) ~[mysql-connector-java-8.0.13.jar:8.0.13]
at com.mysql.cj.jdbc.result.ResultSetImpl.getRow(ResultSetImpl.java:1663) ~[mysql-connector-java-8.0.13.jar:8.0.13]
at com.complexible.stardog.virtual.vega.mapping.TemplateTermMap$1.createValue(TemplateTermMap.java:231) ~[stardog-virtual-core-6.2.2.jar:?]
at com.complexible.stardog.virtual.ValueGenerator$AbstractValueGenerator.generateValue(ValueGenerator.java:100) ~[stardog-virtual-core-6.2.2.jar:?]
at com.complexible.stardog.virtual.VirtualGraphStatementIterator.computeValue(VirtualGraphStatementIterator.java:40) ~[stardog-virtual-core-6.2.2.jar:?]
at com.complexible.stardog.virtual.VirtualGraphStatementIterator.computeValue(VirtualGraphStatementIterator.java:26) ~[stardog-virtual-core-6.2.2.jar:?]
at com.complexible.stardog.virtual.VirtualGraphIterator.computeNext(VirtualGraphIterator.java:142) ~[stardog-virtual-core-6.2.2.jar:?]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) ~[guava-26.0-jre.jar:?]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) ~[guava-26.0-jre.jar:?]
at com.complexible.stardog.virtual.StatementToSolutionIterator.hasNext(StatementToSolutionIterator.java:47) ~[stardog-virtual-core-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.impl.ServiceOperatorImpl.computeNext(ServiceOperatorImpl.java:89) ~[stardog-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.impl.ServiceOperatorImpl.computeNext(ServiceOperatorImpl.java:34) ~[stardog-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.tryToComputeNext(AbstractSkippingIterator.java:143) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.hasNext(AbstractSkippingIterator.java:130) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.impl.SingleProjectionOp.computeNext(SingleProjectionOp.java:82) ~[stardog-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.impl.SingleProjectionOp.computeNext(SingleProjectionOp.java:29) ~[stardog-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.tryToComputeNext(AbstractSkippingIterator.java:143) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.hasNext(AbstractSkippingIterator.java:130) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.util.AutoCloseOperator.computeNext(AutoCloseOperator.java:112) ~[stardog-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.util.AutoCloseOperator.computeNext(AutoCloseOperator.java:25) ~[stardog-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.tryToComputeNext(AbstractSkippingIterator.java:143) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.common.collect.AbstractSkippingIterator.hasNext(AbstractSkippingIterator.java:130) ~[stardog-utils-common-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.util.OpBasedBindingSetIteration.computeNext(OpBasedBindingSetIteration.java:116) ~[stardog-6.2.2.jar:?]
at com.complexible.stardog.plan.eval.operator.util.OpBasedBindingSetIteration.computeNext(OpBasedBindingSetIteration.java:39) ~[stardog-6.2.2.jar:?]
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) ~[guava-26.0-jre.jar:?]
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) ~[guava-26.0-jre.jar:?]
at com.complexible.common.rdf.query.IteratorAsTupleQueryResult.hasNext(IteratorAsTupleQueryResult.java:78) ~[stardog-utils-rdf-6.2.2.jar:?]
... 14 more
It looks like there has been a change in behavior with the newer MySQL JDBC driver. One possible workaround is to use the MySQL Connector/J version 5.1. I have created a new ticket and we will get this fixed shortly.
OK! I'm indeed using the newer MySQL Connector/J version 8.0
In the meantime, I also noticed that during startup of Stardog, I see the following line:
Loading class 'com.mysql.jdbc.Driver'. This is deprecated. The new driver class is 'com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
I copied the mysql-connector-java-8.0.13.jar from MySQL installation folder into stardog-6.2.2\server\dbms. I assume this is the newest Connector/J (v8). So, to proceed, I either:
change the named of the driver class in the virtual graph configuration, to use the newest JDBC connector for MySQL
load the jar of the older Connector/J (v5.1) and keep the name of the driver class
?
It is not necessary to change the name of the driver in the configuration file. The warning is harmless. Switching to the 5.1 series driver however, will workaround the error that you are seeing. The other option is to ensure that your tables have primary keys.
Okay, I'm making slow progress, but getting there. I now have the 5.1 driver and I have no more issues on the sample beatles dataset and the default sakila dataset from MySQL. However, when I test it on a real world dataset (which misses foreign keys), I get the following error in:
Exception in thread "stardog-user-3" java.nio.BufferOverflowException
at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:363)
at java.nio.ByteBuffer.put(ByteBuffer.java:859)
at io.undertow.conduits.ChunkedStreamSinkConduit.createLastChunk(ChunkedStreamSinkConduit.java:339)
at io.undertow.conduits.ChunkedStreamSinkConduit.terminateWrites(ChunkedStreamSinkConduit.java:315)
at io.undertow.conduits.DeflatingStreamSinkConduit.flush(DeflatingStreamSinkConduit.java:352)
at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162)
at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:119)
at org.xnio.channels.Channels.flushBlocking(Channels.java:63)
at io.undertow.io.UndertowOutputStream.close(UndertowOutputStream.java:344)
at io.undertow.io.BlockingSenderImpl.close(BlockingSenderImpl.java:170)
at io.undertow.io.DefaultIoCallback.onComplete(DefaultIoCallback.java:54)
at io.undertow.io.BlockingSenderImpl.invokeOnComplete(BlockingSenderImpl.java:227)
at io.undertow.io.BlockingSenderImpl.send(BlockingSenderImpl.java:60)
at io.undertow.io.BlockingSenderImpl.send(BlockingSenderImpl.java:79)
at com.complexible.stardog.protocols.http.server.StardogHttpServiceLoader$S4ErrorHandler.accept(StardogHttpServiceLoader.java:281)
at com.stardog.http.server.undertow.ErrorHandling.sendErrorResponse(ErrorHandling.java:70)
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:195)
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120)
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
When I try to materialize the virtual triples via the console, I get a class java.lang.NullPointerException. The stardog.log does not contain any more details for the above exception or the BufferOverflowException...
I can see that the direct mappings resulted in some seemingly valid mappings being generated (checked in Stardog Studio).
In my dataset, I notice there are some columns from different tables having the same name (these would ideally be foreign keys, but currently they are absent). In my case, I try to make sense of an existing and rather complex SQL database (missing foreign keys) using graphs and direct mapping, before optimizing the virtual graph with more adapted mappings with some selected ontologies. If you have any recommendations for a workflow, I would be glad to hear them
Let me check into the buffer overflow exception. Does it happen every time you execute the query?
Regarding the NullPointerException, are you able to export and share the mappings? The virtual mappings command can be used for this, or saving them from Stardog Studio.
