Error when adding sets of zip files with large number of ttl's inside

johannes · November 17, 2017, 3:00pm

Hi,

I was trying to import zip files with 10000 turtle files each. Some work but large ones produce the following error. Especially if I select them via wildcard (e.g. 00418*.zip they fail with this error after a while).
What’s happening? It’d definitely not a parse error as indicated because separately the same files work. I guess it is a memory issue somehow…

Adding data from file: /home/alethio/data/block-rdf/0004180000_0004189999.zip
There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
The detailed stack trace for the error is:
com.complexible.stardog.cli.CliException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
    at com.complexible.stardog.cli.impl.ConnectionCommand.call(ConnectionCommand.java:80)
    at com.complexible.stardog.cli.CLIBase.execute(CLIBase.java:55)
    at com.complexible.stardog.cli.CLI.main(CLI.java:120)
Caused by: com.complexible.stardog.protocols.http.client.BaseHttpClient$HttpClientException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
    at com.complexible.stardog.protocols.http.client.BaseHttpClient.checkResponseCode(BaseHttpClient.java:486)
    at com.complexible.stardog.protocols.http.client.BaseHttpClient.execute(BaseHttpClient.java:320)
    at com.complexible.stardog.protocols.http.client.HttpClientImpl.commit(HttpClientImpl.java:527)
    at com.complexible.stardog.protocols.http.client.HttpConnection._commit(HttpConnection.java:388)
    at com.complexible.stardog.api.impl.AbstractConnection.commit(AbstractConnection.java:240)
    at com.complexible.stardog.cli.impl.ConnectionCommand.commitAndClose(ConnectionCommand.java:163)
    at com.complexible.stardog.cli.impl.Add.execute(Add.java:114)
    at com.complexible.stardog.cli.impl.ConnectionCommand.call(ConnectionCommand.java:68)
    ... 2 more

The server logs of this:

Adding data from file: /home/alethio/data/block-rdf/0004180000_0004189999.zip
There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
The detailed stack trace for the error is:
com.complexible.stardog.cli.CliException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
        at com.complexible.stardog.cli.impl.ConnectionCommand.call(ConnectionCommand.java:80)
        at com.complexible.stardog.cli.CLIBase.execute(CLIBase.java:55)
        at com.complexible.stardog.cli.CLI.main(CLI.java:120)
Caused by: com.complexible.stardog.protocols.http.client.BaseHttpClient$HttpClientException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf
.rio.RDFParseException: null
        at com.complexible.stardog.protocols.http.client.BaseHttpClient.checkResponseCode(BaseHttpClient.java:486)
        at com.complexible.stardog.protocols.http.client.BaseHttpClient.execute(BaseHttpClient.java:320)
        at com.complexible.stardog.db.DelegatingDatabaseConnection.rollback(DelegatingDatabaseConnection.java:306) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.StardogKernel$KernelDbConnection.rollback(StardogKernel.java:3152) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DelegatingDatabaseConnection.rollback(DelegatingDatabaseConnection.java:306) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.StardogKernel$NotifyingDatabaseConnection.rollback(StardogKernel.java:3452) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.ProtocolUtils.rollbackQuietly(ProtocolUtils.java:602) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.ProtocolUtils.executeCommit(ProtocolUtils.java:619) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.TransactionService.commit(TransactionService.java:148) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:186) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) ~[shiro-core-1.2.3.jar:1.2.3]
        at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) ~[shiro-core-1.2.3.jar:1.2.3]
        at com.stardog.http.server.undertow.ErrorHandling.lambda$safeDispatch$1(ErrorHandling.java:71) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: com.complexible.stardog.server.UnknownTransactionException: There is no current transaction
        ... 19 more
ERROR 2017-11-13 16:50:34,413 [XNIO-1 task-54] com.complexible.stardog.protocols.http.server.StardogHttpServiceLoader:accept(231): An unexpected exception was handled by the server
com.complexible.tx.api.HeuristicRollbackException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
        at com.complexible.tx.api.impl.DefaultTransaction.runPreparePhase(DefaultTransaction.java:406) ~[stardog-5.0.4.jar:?]
        at com.complexible.tx.api.impl.DefaultTransaction.commit(DefaultTransaction.java:288) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DatabaseConnectionImpl.commit(DatabaseConnectionImpl.java:454) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DelegatingDatabaseConnection.commit(DelegatingDatabaseConnection.java:293) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DatabaseImpl$MonitoredDbConnection.commit(DatabaseImpl.java:1714) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DatabaseImpl$DBConnectionWrapper.commit(DatabaseImpl.java:1479) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DelegatingDatabaseConnection.commit(DelegatingDatabaseConnection.java:293) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.StardogKernel$KernelDbConnection.commit(StardogKernel.java:3162) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DelegatingDatabaseConnection.commit(DelegatingDatabaseConnection.java:293) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.StardogKernel$NotifyingDatabaseConnection.commit(StardogKernel.java:3430) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.ProtocolUtils.executeCommit(ProtocolUtils.java:616) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.TransactionService.commit(TransactionService.java:148) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:186) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) ~[shiro-core-1.2.3.jar:1.2.3]
        at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) ~[shiro-core-1.2.3.jar:1.2.3]
        at com.stardog.http.server.undertow.ErrorHandling.lambda$safeDispatch$1(ErrorHandling.java:71) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Waiting for running tasks to complete...done. Executor service has been shut down.
Stardog server 5.0.4 shutdown on Mon Nov 13 17:00:19 UTC 2017.

jess · November 17, 2017, 3:01pm

Can you check the server log file?

johannes · November 20, 2017, 3:28pm

For the log, please check the original post.

jess · November 21, 2017, 12:30pm

Are you using Java 9?

johannes · November 22, 2017, 12:46pm

Hi jess, thanks for the answer and sorry for the confusion. I had pasted the wrong log lines in the post and due to the new user policy of this forum was not able to correct that. I added the correct logs to the original post - please have a look there.

The logs you saw do indeed indicate Java 9 - that was an earlier issue which I had already resolved.

Thanks,
Johannes

jess · November 22, 2017, 1:38pm

Can you try this on Stardog 5.0.5.1?

johannes · November 23, 2017, 4:41pm

Just tried. Crashed again after a while. The first warning is repeated many times. No idea what this is, atm...

https://pastebin.com/7DH7sgZq

jess · November 28, 2017, 8:13pm

Thanks for the detailed log. Unfortunately this isn't enough to debug the issue. Are you able to share a minimal set of the data which reproduces the error? You can send it via something like Dropbox or send me a link privately to jess@stardog.com.

system · December 12, 2017, 8:14pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Unable to load data Support	18	673	November 10, 2020
Error bulk loading Bug	3	714	May 30, 2017
RDFParseException (Unexpected end of file) after Stardog finishes parsing triples Support	8	875	September 12, 2018
File name can cause file not found exceptions during bulk load Bug	2	948	July 31, 2017
SPARQL LOAD from a file gets 000012 error Support	3	276	May 17, 2022

Error when adding sets of zip files with large number of ttl's inside

Related Topics