When running a relatively simple query and obtaining the results I noticed that stardog became unresponsive. Looking at the query log nothing was running but htop showed that stardog uses 2800% CPU.
Not sure what I can add to this but what is going on after a query?
What version of Stardog are you seeing this behavior with?
stardog-5.3.4 is the version. I noticed in the log files something about GC but the dataset only contains 21,729,811 triples.
Are you able to take a jstack of the stardog process during this heavy CPU usage while it isn’t doing anything?
I did obtain this from the log file:
INFO 2018-09-10 11:29:53,786 [main] com.complexible.stardog.cli.impl.ServerStart:call(249): Memory options
INFO 2018-09-10 11:29:53,790 [main] com.complexible.stardog.cli.impl.ServerStart:call(250): Memory mode: DEFAULT
INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(251): Min Heap Size: 2.0G
INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(252): Max Heap Size: 1.9G
INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(253): Max Direct Mem: 1.0G
INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(254): Max Mapped Mem: 115M
INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(255): System Memory: 78G
ERROR 2018-09-10 11:32:26,532 [stardog-user-6] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7653148 column 64 to Json
com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7653148 column 64 to Json
at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at com.google.gson.internal.LinkedTreeMap.find(LinkedTreeMap.java:169) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.LinkedTreeMap.put(LinkedTreeMap.java:94) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonObject.add(JsonObject.java:57) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.Streams.parse(Streams.java:48) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:84) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) [stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) [stardog-webconsole-annex-5.3.4.jar:?]
at java.lang.invoke.LambdaForm$DMH/2022417982.invokeSpecial_LL_V(LambdaForm$DMH) ~[?:?]
at java.lang.invoke.LambdaForm$BMH/1034835398.reinvoke(LambdaForm$BMH) ~[?:?]
at java.lang.invoke.LambdaForm$MH/815678150.invoke_MT(LambdaForm$MH) ~[?:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) [stardog-protocols-http-server-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes$$Lambda$317/851404690.run(Unknown Source) [stardog-protocols-http-server-5.3.4.jar:?]
… 5 more
ERROR 2018-09-10 11:35:07,840 [stardog-user-13] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7649521 column 17 to Json
com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7649521 column 17 to Json
at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.StringBuilder.toString(StringBuilder.java:407) ~[?:1.8.0_181]
at com.google.gson.stream.JsonReader.nextQuotedValue(JsonReader.java:1007) ~[gson-2.6.2.jar:?]
at com.google.gson.stream.JsonReader.nextName(JsonReader.java:791) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.Streams.parse(Streams.java:48) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:84) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) [stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) [stardog-webconsole-annex-5.3.4.jar:?]
at java.lang.invoke.LambdaForm$DMH/2022417982.invokeSpecial_LL_V(LambdaForm$DMH) ~[?:?]
at java.lang.invoke.LambdaForm$BMH/1034835398.reinvoke(LambdaForm$BMH) ~[?:?]
at java.lang.invoke.LambdaForm$MH/815678150.invoke_MT(LambdaForm$MH) ~[?:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) [stardog-protocols-http-server-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes$$Lambda$317/851404690.run(Unknown Source) [stardog-protocols-http-server-5.3.4.jar:?]
… 5 more
ERROR 2018-09-10 11:35:41,408 [stardog-user-30] com.complexible.stardog.protocols.http.server.StardogHttpServiceLoader:accept(226): An exception was handled by the server: The password must be longer or equal than 4 characters.
ERROR 2018-09-10 11:38:47,359 [stardog-user-42] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7639684 column 18 to Json
com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7639684 column 18 to Json
at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.StringBuilder.toString(StringBuilder.java:407) ~[?:1.8.0_181]
at com.google.gson.stream.JsonReader.nextQuotedValue(JsonReader.java:1007) ~[gson-2.6.2.jar:?]
at com.google.gson.stream.JsonReader.nextName(JsonReader.java:791) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.Streams.parse(Streams.java:48) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:84) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) [stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) [stardog-webconsole-annex-5.3.4.jar:?]
at java.lang.invoke.LambdaForm$DMH/2022417982.invokeSpecial_LL_V(LambdaForm$DMH) ~[?:?]
at java.lang.invoke.LambdaForm$BMH/1034835398.reinvoke(LambdaForm$BMH) ~[?:?]
at java.lang.invoke.LambdaForm$MH/815678150.invoke_MT(LambdaForm$MH) ~[?:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) [stardog-protocols-http-server-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes$$Lambda$317/851404690.run(Unknown Source) [stardog-protocols-http-server-5.3.4.jar:?]
… 5 more
ERROR 2018-09-10 14:20:00,522 [stardog-user-61] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7641469 column 8 to Json
com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7641469 column 8 to Json
at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at com.google.gson.JsonObject.<init>(JsonObject.java:33) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:740) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718) ~[gson-2.6.2.jar:?]
at com.google.gson.internal.Streams.parse(Streams.java:48) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:84) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) [stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) [stardog-webconsole-annex-5.3.4.jar:?]
at java.lang.invoke.LambdaForm$DMH/2022417982.invokeSpecial_LL_V(LambdaForm$DMH) ~[?:?]
at java.lang.invoke.LambdaForm$BMH/1034835398.reinvoke(LambdaForm$BMH) ~[?:?]
at java.lang.invoke.LambdaForm$MH/815678150.invoke_MT(LambdaForm$MH) ~[?:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) [stardog-protocols-http-server-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes$$Lambda$317/851404690.run(Unknown Source) [stardog-protocols-http-server-5.3.4.jar:?]
… 5 more
ERROR 2018-09-10 14:23:02,206 [stardog-user-114] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7585119 column 10 to Json
com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7585119 column 10 to Json
at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
ERROR 2018-09-10 14:23:02,212 [stardog-admin-71] com.stardog.http.server.undertow.ErrorHandling:writeError(138): Unexpected error on the server
java.io.UncheckedIOException: java.io.IOException: Broken pipe
at com.complexible.stardog.protocols.http.server.StatusService.status(StatusService.java:138) ~[stardog-protocols-http-server-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) ~[?:1.8.0_181]
at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) ~[?:1.8.0_181]
at sun.nio.ch.IOUtil.write(IOUtil.java:148) ~[?:1.8.0_181]
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) ~[?:1.8.0_181]
at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:161) ~[xnio-nio-3.3.6.Final.jar:3.3.6.Final]
at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:645) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]
at io.undertow.conduits.ChunkedStreamSinkConduit.doWrite(ChunkedStreamSinkConduit.java:163) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]
at io.undertow.conduits.ChunkedStreamSinkConduit.write(ChunkedStreamSinkConduit.java:127) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]
at io.undertow.conduits.ChunkedStreamSinkConduit.write(ChunkedStreamSinkConduit.java:216) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]
at io.undertow.conduits.DeflatingStreamSinkConduit.performFlushIfRequired(DeflatingStreamSinkConduit.java:419) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]
at io.undertow.conduits.DeflatingStreamSinkConduit.flush(DeflatingStreamSinkConduit.java:371) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]
at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162) ~[xnio-api-3.3.6.Final.jar:3.3.6.Final]
at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:119) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]
at org.xnio.channels.Channels.flushBlocking(Channels.java:63) ~[xnio-api-3.3.6.Final.jar:3.3.6.Final]
at io.undertow.io.UndertowOutputStream.flush(UndertowOutputStream.java:280) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.flush(UTF8JsonGenerator.java:1048) ~[jackson-core-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:2486) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:378) ~[jackson-core-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.core.JsonGenerator.writeObjectField(JsonGenerator.java:1652) ~[jackson-core-2.8.2.jar:2.8.2]
at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$GaugeSerializer.serialize(StatusService.java:172) ~[stardog-protocols-http-server-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$GaugeSerializer.serialize(StatusService.java:159) ~[stardog-protocols-http-server-5.3.4.jar:?]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:633) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:536) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:30) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:292) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:2484) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:378) ~[jackson-core-2.8.2.jar:2.8.2]
at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$MetricRegistrySerializer.serialize(StatusService.java:332) ~[stardog-protocols-http-server-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$MetricRegistrySerializer.serialize(StatusService.java:310) ~[stardog-protocols-http-server-5.3.4.jar:?]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:292) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1419) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1147) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:977) ~[jackson-databind-2.8.2.jar:2.8.2]
at com.complexible.stardog.protocols.http.server.StatusService.status(StatusService.java:135) ~[stardog-protocols-http-server-5.3.4.jar:?]
… 6 more
ERROR 2018-09-10 14:29:27,664 [stardog-user-245] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7578063 column 17 to Json
com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7578063 column 17 to Json
at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]
at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]
at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]
at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]
at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
You may want to allocate more memory to Stardog (it looks like it’s using the default 2G heap and 2G off-heap out of 78G on the machine) via the STARDOG_SERVER_JAVA_ARGS environment variable.
That said, if you see this behavior again (lots of CPU usage while doing nothing), running jstack on the process ID would be very helpful to determine what the process is doing at that exact time.