Error when adding sets of zip files with large number of ttl's inside

Hi,

I was trying to import zip files with 10000 turtle files each. Some work but large ones produce the following error. Especially if I select them via wildcard (e.g. 00418*.zip they fail with this error after a while).
What’s happening? It’d definitely not a parse error as indicated because separately the same files work. I guess it is a memory issue somehow…

Adding data from file: /home/alethio/data/block-rdf/0004180000_0004189999.zip
There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
The detailed stack trace for the error is:
com.complexible.stardog.cli.CliException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
    at com.complexible.stardog.cli.impl.ConnectionCommand.call(ConnectionCommand.java:80)
    at com.complexible.stardog.cli.CLIBase.execute(CLIBase.java:55)
    at com.complexible.stardog.cli.CLI.main(CLI.java:120)
Caused by: com.complexible.stardog.protocols.http.client.BaseHttpClient$HttpClientException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
    at com.complexible.stardog.protocols.http.client.BaseHttpClient.checkResponseCode(BaseHttpClient.java:486)
    at com.complexible.stardog.protocols.http.client.BaseHttpClient.execute(BaseHttpClient.java:320)
    at com.complexible.stardog.protocols.http.client.HttpClientImpl.commit(HttpClientImpl.java:527)
    at com.complexible.stardog.protocols.http.client.HttpConnection._commit(HttpConnection.java:388)
    at com.complexible.stardog.api.impl.AbstractConnection.commit(AbstractConnection.java:240)
    at com.complexible.stardog.cli.impl.ConnectionCommand.commitAndClose(ConnectionCommand.java:163)
    at com.complexible.stardog.cli.impl.Add.execute(Add.java:114)
    at com.complexible.stardog.cli.impl.ConnectionCommand.call(ConnectionCommand.java:68)
    ... 2 more

The server logs of this:

Adding data from file: /home/alethio/data/block-rdf/0004180000_0004189999.zip
There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
The detailed stack trace for the error is:
com.complexible.stardog.cli.CliException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
        at com.complexible.stardog.cli.impl.ConnectionCommand.call(ConnectionCommand.java:80)
        at com.complexible.stardog.cli.CLIBase.execute(CLIBase.java:55)
        at com.complexible.stardog.cli.CLI.main(CLI.java:120)
Caused by: com.complexible.stardog.protocols.http.client.BaseHttpClient$HttpClientException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf
.rio.RDFParseException: null
        at com.complexible.stardog.protocols.http.client.BaseHttpClient.checkResponseCode(BaseHttpClient.java:486)
        at com.complexible.stardog.protocols.http.client.BaseHttpClient.execute(BaseHttpClient.java:320)
        at com.complexible.stardog.db.DelegatingDatabaseConnection.rollback(DelegatingDatabaseConnection.java:306) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.StardogKernel$KernelDbConnection.rollback(StardogKernel.java:3152) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DelegatingDatabaseConnection.rollback(DelegatingDatabaseConnection.java:306) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.StardogKernel$NotifyingDatabaseConnection.rollback(StardogKernel.java:3452) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.ProtocolUtils.rollbackQuietly(ProtocolUtils.java:602) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.ProtocolUtils.executeCommit(ProtocolUtils.java:619) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.TransactionService.commit(TransactionService.java:148) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:186) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) ~[shiro-core-1.2.3.jar:1.2.3]
        at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) ~[shiro-core-1.2.3.jar:1.2.3]
        at com.stardog.http.server.undertow.ErrorHandling.lambda$safeDispatch$1(ErrorHandling.java:71) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: com.complexible.stardog.server.UnknownTransactionException: There is no current transaction
        ... 19 more
ERROR 2017-11-13 16:50:34,413 [XNIO-1 task-54] com.complexible.stardog.protocols.http.server.StardogHttpServiceLoader:accept(231): An unexpected exception was handled by the server
com.complexible.tx.api.HeuristicRollbackException: There was a fatal failure during preparation of 9f736eff-c32c-4fc0-b9b5-bf2428b1f8d0 org.openrdf.rio.RDFParseException: null
        at com.complexible.tx.api.impl.DefaultTransaction.runPreparePhase(DefaultTransaction.java:406) ~[stardog-5.0.4.jar:?]
        at com.complexible.tx.api.impl.DefaultTransaction.commit(DefaultTransaction.java:288) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DatabaseConnectionImpl.commit(DatabaseConnectionImpl.java:454) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DelegatingDatabaseConnection.commit(DelegatingDatabaseConnection.java:293) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DatabaseImpl$MonitoredDbConnection.commit(DatabaseImpl.java:1714) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DatabaseImpl$DBConnectionWrapper.commit(DatabaseImpl.java:1479) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DelegatingDatabaseConnection.commit(DelegatingDatabaseConnection.java:293) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.StardogKernel$KernelDbConnection.commit(StardogKernel.java:3162) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.db.DelegatingDatabaseConnection.commit(DelegatingDatabaseConnection.java:293) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.StardogKernel$NotifyingDatabaseConnection.commit(StardogKernel.java:3430) ~[stardog-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.ProtocolUtils.executeCommit(ProtocolUtils.java:616) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.complexible.stardog.protocols.http.server.TransactionService.commit(TransactionService.java:148) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:186) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) ~[shiro-core-1.2.3.jar:1.2.3]
        at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) ~[shiro-core-1.2.3.jar:1.2.3]
        at com.stardog.http.server.undertow.ErrorHandling.lambda$safeDispatch$1(ErrorHandling.java:71) ~[stardog-protocols-http-server-5.0.4.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Waiting for running tasks to complete...done. Executor service has been shut down.
Stardog server 5.0.4 shutdown on Mon Nov 13 17:00:19 UTC 2017.

Can you check the server log file?

1 Like

For the log, please check the original post.

Are you using Java 9?

Hi jess, thanks for the answer and sorry for the confusion. I had pasted the wrong log lines in the post and due to the new user policy of this forum was not able to correct that. I added the correct logs to the original post - please have a look there.

The logs you saw do indeed indicate Java 9 - that was an earlier issue which I had already resolved.

Thanks,
Johannes

Can you try this on Stardog 5.0.5.1?

1 Like

Just tried. Crashed again after a while. The first warning is repeated many times. No idea what this is, atm…

https://pastebin.com/7DH7sgZq

Thanks for the detailed log. Unfortunately this isn’t enough to debug the issue. Are you able to share a minimal set of the data which reproduces the error? You can send it via something like Dropbox or send me a link privately to jess@stardog.com.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.