Full CPU usage when doing nothing


(Jasper) #1

When running a relatively simple query and obtaining the results I noticed that stardog became unresponsive. Looking at the query log nothing was running but htop showed that stardog uses 2800% CPU.

Not sure what I can add to this but what is going on after a query?


(stephen) #2

What version of Stardog are you seeing this behavior with?


(Jasper) #3

stardog-5.3.4 is the version. I noticed in the log files something about GC but the dataset only contains 21,729,811 triples.


(stephen) #4

Are you able to take a jstack of the stardog process during this heavy CPU usage while it isn’t doing anything?


(Jasper) #5

I did obtain this from the log file:

INFO 2018-09-10 11:29:53,786 [main] com.complexible.stardog.cli.impl.ServerStart:call(249): Memory options

INFO 2018-09-10 11:29:53,790 [main] com.complexible.stardog.cli.impl.ServerStart:call(250): Memory mode: DEFAULT

INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(251): Min Heap Size: 2.0G

INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(252): Max Heap Size: 1.9G

INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(253): Max Direct Mem: 1.0G

INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(254): Max Mapped Mem: 115M

INFO 2018-09-10 11:29:53,791 [main] com.complexible.stardog.cli.impl.ServerStart:call(255): System Memory: 78G

ERROR 2018-09-10 11:32:26,532 [stardog-user-6] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7653148 column 64 to Json

com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7653148 column 64 to Json

at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]

at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]

at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

at com.google.gson.internal.LinkedTreeMap.find(LinkedTreeMap.java:169) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.LinkedTreeMap.put(LinkedTreeMap.java:94) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonObject.add(JsonObject.java:57) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.Streams.parse(Streams.java:48) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:84) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) [stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) [stardog-webconsole-annex-5.3.4.jar:?]

at java.lang.invoke.LambdaForm$DMH/2022417982.invokeSpecial_LL_V(LambdaForm$DMH) ~[?:?]

at java.lang.invoke.LambdaForm$BMH/1034835398.reinvoke(LambdaForm$BMH) ~[?:?]

at java.lang.invoke.LambdaForm$MH/815678150.invoke_MT(LambdaForm$MH) ~[?:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) [stardog-protocols-http-server-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes$$Lambda$317/851404690.run(Unknown Source) [stardog-protocols-http-server-5.3.4.jar:?]

… 5 more

ERROR 2018-09-10 11:35:07,840 [stardog-user-13] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7649521 column 17 to Json

com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7649521 column 17 to Json

at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]

at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]

at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

at java.lang.StringBuilder.toString(StringBuilder.java:407) ~[?:1.8.0_181]

at com.google.gson.stream.JsonReader.nextQuotedValue(JsonReader.java:1007) ~[gson-2.6.2.jar:?]

at com.google.gson.stream.JsonReader.nextName(JsonReader.java:791) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.Streams.parse(Streams.java:48) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:84) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) [stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) [stardog-webconsole-annex-5.3.4.jar:?]

at java.lang.invoke.LambdaForm$DMH/2022417982.invokeSpecial_LL_V(LambdaForm$DMH) ~[?:?]

at java.lang.invoke.LambdaForm$BMH/1034835398.reinvoke(LambdaForm$BMH) ~[?:?]

at java.lang.invoke.LambdaForm$MH/815678150.invoke_MT(LambdaForm$MH) ~[?:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) [stardog-protocols-http-server-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes$$Lambda$317/851404690.run(Unknown Source) [stardog-protocols-http-server-5.3.4.jar:?]

… 5 more

ERROR 2018-09-10 11:35:41,408 [stardog-user-30] com.complexible.stardog.protocols.http.server.StardogHttpServiceLoader:accept(226): An exception was handled by the server: The password must be longer or equal than 4 characters.

ERROR 2018-09-10 11:38:47,359 [stardog-user-42] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7639684 column 18 to Json

com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7639684 column 18 to Json

at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]

at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]

at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

at java.lang.StringBuilder.toString(StringBuilder.java:407) ~[?:1.8.0_181]

at com.google.gson.stream.JsonReader.nextQuotedValue(JsonReader.java:1007) ~[gson-2.6.2.jar:?]

at com.google.gson.stream.JsonReader.nextName(JsonReader.java:791) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.Streams.parse(Streams.java:48) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:84) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) [stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) [stardog-webconsole-annex-5.3.4.jar:?]

at java.lang.invoke.LambdaForm$DMH/2022417982.invokeSpecial_LL_V(LambdaForm$DMH) ~[?:?]

at java.lang.invoke.LambdaForm$BMH/1034835398.reinvoke(LambdaForm$BMH) ~[?:?]

at java.lang.invoke.LambdaForm$MH/815678150.invoke_MT(LambdaForm$MH) ~[?:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) [stardog-protocols-http-server-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes$$Lambda$317/851404690.run(Unknown Source) [stardog-protocols-http-server-5.3.4.jar:?]

… 5 more

ERROR 2018-09-10 14:20:00,522 [stardog-user-61] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7641469 column 8 to Json

com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7641469 column 8 to Json

at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]

at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]

at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

at com.google.gson.JsonObject.<init>(JsonObject.java:33) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:740) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:735) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:743) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.bind.TypeAdapters$29.read(TypeAdapters.java:718) ~[gson-2.6.2.jar:?]

at com.google.gson.internal.Streams.parse(Streams.java:48) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:84) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) [stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) [stardog-webconsole-annex-5.3.4.jar:?]

at java.lang.invoke.LambdaForm$DMH/2022417982.invokeSpecial_LL_V(LambdaForm$DMH) ~[?:?]

at java.lang.invoke.LambdaForm$BMH/1034835398.reinvoke(LambdaForm$BMH) ~[?:?]

at java.lang.invoke.LambdaForm$MH/815678150.invoke_MT(LambdaForm$MH) ~[?:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) [stardog-protocols-http-server-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes$$Lambda$317/851404690.run(Unknown Source) [stardog-protocols-http-server-5.3.4.jar:?]

… 5 more

ERROR 2018-09-10 14:23:02,206 [stardog-user-114] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7585119 column 10 to Json

com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7585119 column 10 to Json

at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]

at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]

at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

ERROR 2018-09-10 14:23:02,212 [stardog-admin-71] com.stardog.http.server.undertow.ErrorHandling:writeError(138): Unexpected error on the server

java.io.UncheckedIOException: java.io.IOException: Broken pipe

at com.complexible.stardog.protocols.http.server.StatusService.status(StatusService.java:138) ~[stardog-protocols-http-server-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]

at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]

at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Caused by: java.io.IOException: Broken pipe

at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) ~[?:1.8.0_181]

at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) ~[?:1.8.0_181]

at sun.nio.ch.IOUtil.write(IOUtil.java:148) ~[?:1.8.0_181]

at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) ~[?:1.8.0_181]

at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:161) ~[xnio-nio-3.3.6.Final.jar:3.3.6.Final]

at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:645) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]

at io.undertow.conduits.ChunkedStreamSinkConduit.doWrite(ChunkedStreamSinkConduit.java:163) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]

at io.undertow.conduits.ChunkedStreamSinkConduit.write(ChunkedStreamSinkConduit.java:127) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]

at io.undertow.conduits.ChunkedStreamSinkConduit.write(ChunkedStreamSinkConduit.java:216) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]

at io.undertow.conduits.DeflatingStreamSinkConduit.performFlushIfRequired(DeflatingStreamSinkConduit.java:419) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]

at io.undertow.conduits.DeflatingStreamSinkConduit.flush(DeflatingStreamSinkConduit.java:371) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]

at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162) ~[xnio-api-3.3.6.Final.jar:3.3.6.Final]

at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:119) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]

at org.xnio.channels.Channels.flushBlocking(Channels.java:63) ~[xnio-api-3.3.6.Final.jar:3.3.6.Final]

at io.undertow.io.UndertowOutputStream.flush(UndertowOutputStream.java:280) ~[undertow-core-1.4.11.Final.jar:1.4.11.Final]

at com.fasterxml.jackson.core.json.UTF8JsonGenerator.flush(UTF8JsonGenerator.java:1048) ~[jackson-core-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:2486) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:378) ~[jackson-core-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.core.JsonGenerator.writeObjectField(JsonGenerator.java:1652) ~[jackson-core-2.8.2.jar:2.8.2]

at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$GaugeSerializer.serialize(StatusService.java:172) ~[stardog-protocols-http-server-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$GaugeSerializer.serialize(StatusService.java:159) ~[stardog-protocols-http-server-5.3.4.jar:?]

at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:633) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:536) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:30) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:292) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:2484) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:378) ~[jackson-core-2.8.2.jar:2.8.2]

at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$MetricRegistrySerializer.serialize(StatusService.java:332) ~[stardog-protocols-http-server-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$MetricRegistrySerializer.serialize(StatusService.java:310) ~[stardog-protocols-http-server-5.3.4.jar:?]

at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:292) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1419) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1147) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:977) ~[jackson-databind-2.8.2.jar:2.8.2]

at com.complexible.stardog.protocols.http.server.StatusService.status(StatusService.java:135) ~[stardog-protocols-http-server-5.3.4.jar:?]

… 6 more

ERROR 2018-09-10 14:29:27,664 [stardog-user-245] com.complexible.annex.api.SchemaHandler:getClassTreeInfo(187): Failed parsing JSON source: JsonReader at line 7578063 column 17 to Json

com.google.gson.JsonParseException: Failed parsing JSON source: JsonReader at line 7578063 column 17 to Json

at com.google.gson.JsonParser.parse(JsonParser.java:88) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:59) ~[gson-2.6.2.jar:?]

at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[gson-2.6.2.jar:?]

at com.complexible.annex.api.SchemaHandler.getClassTreeInfo(SchemaHandler.java:177) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.complexible.stardog.protocols.http.annex.SchemaService.classTree(SchemaService.java:61) ~[stardog-webconsole-annex-5.3.4.jar:?]

at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-5.3.4.jar:?]

at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.3.0.jar:1.3.0]

at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.3.0.jar:1.3.0]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]

at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded


(stephen) #6

You may want to allocate more memory to Stardog (it looks like it’s using the default 2G heap and 2G off-heap out of 78G on the machine) via the STARDOG_SERVER_JAVA_ARGS environment variable.

That said, if you see this behavior again (lots of CPU usage while doing nothing), running jstack on the process ID would be very helpful to determine what the process is doing at that exact time.