Killed queries still executing & causing delayed query responses

Support
1

Our Stardog server was running simple queries very slowly, and we noticed that there were ~8 running queries that had been executing for some time. I killed them via the stardog-admin CLI (which stated that it was successful), but two are still showing up in Studio as "killed". I killed them quite awhile ago, but you can see the runtime is still going up.

Screen Shot 2022-12-19 at 10.36.57 AM
Screen Shot 2022-12-19 at 10.36.57 AM2322×480 43.6 KB

The queries are both:

select distinct ?graph where { graph ?graph { } filter ( ?graph not in ( <tag:stardog:api:graph:aliases>,<tag:stardog:api:sql:schema> ) ) } limit 1000

(these are not queries we executed, but seem to have come from Stardog - we had some consultants running into exceptions dropping named graphs with pystardog so I suspect there were some uncommitted transactions).

Querying the database or trying to load data is extremely slow. Any thoughts on how to get rid of these, and will it speed up execution if we do?

Thanks!

2

Hi Rebecca,

Do you have CLI access to the server? If so, can you execute

stardog-admin server metrics --threads "jvm.threads"

(possibly with the --server <endpoint> parameter) and post the output here?

Thanks,
Pavel

3

Yep! Here's the output:

jvm.threads              : "stardog-admin-6" id=8104 state=RUNNABLE
    at sun.management.ThreadImpl.dumpThreads0(Native Method)
    at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:496)
    at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:484)
    at com.complexible.common.util.ThreadDump.dump(ThreadDump.java:49)
    at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$MetricRegistrySerializer.serialize(StatusService.java:521)
    at com.complexible.stardog.protocols.http.server.StatusService$MetricsModule$MetricRegistrySerializer.serialize(StatusService.java:484)
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
    at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
    at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1518)
    at com.fasterxml.jackson.databind.ObjectWriter._writeValueAndClose(ObjectWriter.java:1219)
    at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1044)
    at com.complexible.stardog.protocols.http.server.StatusService.status(StatusService.java:296)
    at java.lang.invoke.LambdaForm$DMH/1368173251.invokeSpecial_LL_V(LambdaForm$DMH)
    at java.lang.invoke.LambdaForm$BMH/32743161.reinvoke(LambdaForm$BMH)
    at java.lang.invoke.LambdaForm$MH/890365384.invoke_MT(LambdaForm$MH)
    at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192)
    at com.stardog.http.server.undertow.jaxrs.ExtractRoutes$$Lambda$970/740058394.run(Unknown Source)
    at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120)
    at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)

"XNIO-1 Accept" id=40 state=RUNNABLE (running in native)
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
    at org.xnio.nio.WorkerThread.run(WorkerThread.java:532)

"Reference Handler" id=2 state=WAITING
    - waiting on <0x46640454> (a java.lang.ref.Reference$Lock)
    - locked <0x46640454> (a java.lang.ref.Reference$Lock)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:502)
    at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
    at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

"Finalizer" id=3 state=WAITING
    - waiting on <0x48d2c3f4> (a java.lang.ref.ReferenceQueue$Lock)
    - locked <0x48d2c3f4> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.Object.wait(Native Method)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

"Stardog.Executor-7943" id=8445 state=TIMED_WAITING
    - waiting on <0x69734e71> (a java.util.concurrent.SynchronousQueue$TransferStack)
    - locked <0x69734e71> (a java.util.concurrent.SynchronousQueue$TransferStack)
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
    at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
    at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)

"ca56693d-58de-4206-a272-a9ec91949cbf_QuartzSchedulerThread" id=24 state=TIMED_WAITING
    - waiting on <0x0369240c> (a java.lang.Object)
    - locked <0x0369240c> (a java.lang.Object)
    at java.lang.Object.wait(Native Method)
    at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:427)

2 threads with the following stack trace:
"DestroyJavaVM" id=41 state=RUNNABLE
"Signal Dispatcher" id=4 state=RUNNABLE

3 threads with the following stack trace:
"XNIO-1 task-7" id=48 state=WAITING
    - waiting on <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    - locked <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
"XNIO-1 task-6" id=47 state=WAITING
    - waiting on <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    - locked <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
"XNIO-1 task-1" id=42 state=WAITING
    - waiting on <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    - locked <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at org.jboss.threads.EnhancedQueueExecutor$PoolThreadNode.park(EnhancedQueueExecutor.java:2194)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1481)
    at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)
    at java.lang.Thread.run(Thread.java:750)

3 threads with the following stack trace:
"TransactionOracle-3" id=59 state=TIMED_WAITING
    - waiting on <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"SessionValidationThread-1" id=51 state=TIMED_WAITING
    - waiting on <0x29483d06> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x29483d06> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"memory-monitor" id=28 state=TIMED_WAITING
    - waiting on <0x11f6e2a1> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x11f6e2a1> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)

4 threads with the following stack trace:
"XNIO-1 I/O-4" id=39 state=RUNNABLE (running in native)
"XNIO-1 I/O-3" id=38 state=RUNNABLE (running in native)
"XNIO-1 I/O-2" id=37 state=RUNNABLE (running in native)
"XNIO-1 I/O-1" id=36 state=RUNNABLE (running in native)
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
    at org.xnio.nio.WorkerThread.run(WorkerThread.java:551)

5 threads with the following stack trace:
"XNIO-1 task-8" id=115 state=TIMED_WAITING
    - waiting on <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    - locked <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
"XNIO-1 task-2" id=46 state=TIMED_WAITING
    - waiting on <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    - locked <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
"XNIO-1 task-4" id=45 state=TIMED_WAITING
    - waiting on <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    - locked <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
"XNIO-1 task-5" id=44 state=TIMED_WAITING
    - waiting on <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    - locked <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
"XNIO-1 task-3" id=43 state=TIMED_WAITING
    - waiting on <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    - locked <0x74305b3d> (a org.jboss.threads.EnhancedQueueExecutor)
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at org.jboss.threads.EnhancedQueueExecutor$PoolThreadNode.park(EnhancedQueueExecutor.java:2234)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1493)
    at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1282)
    at java.lang.Thread.run(Thread.java:750)

7 threads with the following stack trace:
"TransactionOracle-7" id=77 state=WAITING
    - waiting on <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"TransactionOracle-6" id=73 state=WAITING
    - waiting on <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"TransactionOracle-5" id=65 state=WAITING
    - waiting on <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"TransactionOracle-4" id=62 state=WAITING
    - waiting on <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"TransactionOracle-2" id=56 state=WAITING
    - waiting on <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"TransactionOracle-1" id=53 state=WAITING
    - waiting on <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"TransactionOracle-0" id=13 state=WAITING
    - waiting on <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x0ac309ca> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)

9 threads with the following stack trace:
"stardog-user-65" id=8300 state=WAITING
    - waiting on <0x7800d1a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x7800d1a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"stardog-user-64" id=8298 state=WAITING
    - waiting on <0x7800d1a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x7800d1a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"stardog-user-61" id=8150 state=WAITING
    - waiting on <0x7800d1a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x7800d1a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"stardog-user-55" id=8116 state=WAITING
    - waiting on <0x7800d1a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x7800d1a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"stardog-admin-11" id=8109 state=WAITING
    - waiting on <0x7277830e> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x7277830e> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"stardog-admin-10" id=8108 state=WAITING
    - waiting on <0x7277830e> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x7277830e> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"stardog-admin-7" id=8105 state=WAITING
    - waiting on <0x7277830e> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x7277830e> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"stardog-admin-status-2" id=50 state=WAITING
    - waiting on <0x67e9b4d2> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x67e9b4d2> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
"stardog-admin-status-1" id=49 state=WAITING
    - waiting on <0x67e9b4d2> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    - locked <0x67e9b4d2> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)

10 threads with the following stack trace:
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-10" id=23 state=TIMED_WAITING
    - waiting on <0x6b21996a> (a java.lang.Object)
    - locked <0x6b21996a> (a java.lang.Object)
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-9" id=22 state=TIMED_WAITING
    - waiting on <0x4144fff8> (a java.lang.Object)
    - locked <0x4144fff8> (a java.lang.Object)
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-8" id=21 state=TIMED_WAITING
    - waiting on <0x6781fe9b> (a java.lang.Object)
    - locked <0x6781fe9b> (a java.lang.Object)
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-7" id=20 state=TIMED_WAITING
    - waiting on <0x2d02ff23> (a java.lang.Object)
    - locked <0x2d02ff23> (a java.lang.Object)
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-6" id=19 state=TIMED_WAITING
    - waiting on <0x4a8fc920> (a java.lang.Object)
    - locked <0x4a8fc920> (a java.lang.Object)
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-5" id=18 state=TIMED_WAITING
    - waiting on <0x2c5254b6> (a java.lang.Object)
    - locked <0x2c5254b6> (a java.lang.Object)
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-4" id=17 state=TIMED_WAITING
    - waiting on <0x444f1281> (a java.lang.Object)
    - locked <0x444f1281> (a java.lang.Object)
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-3" id=16 state=TIMED_WAITING
    - waiting on <0x7e0e16cc> (a java.lang.Object)
    - locked <0x7e0e16cc> (a java.lang.Object)
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-2" id=15 state=TIMED_WAITING
    - waiting on <0x4a7c16e4> (a java.lang.Object)
    - locked <0x4a7c16e4> (a java.lang.Object)
"ca56693d-58de-4206-a272-a9ec91949cbf_Worker-1" id=14 state=TIMED_WAITING
    - waiting on <0x74b51cf9> (a java.lang.Object)
    - locked <0x74b51cf9> (a java.lang.Object)
    at java.lang.Object.wait(Native Method)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:568)
4

Hm, it's not showing any query executing threads. If you have access to STARDOG_HOME directory, could you send us stardog.log file?

5

We log it all to CloudWatch but I can't access the directory. It looked like our consultants ran into a lot of query exceptions (syntax errors) for a bit, but other than those, could this be something?

WARN  2022-12-19 18:28:09,481 [stardog-user-64] com.complexible.common.cancellation.DefaultCancellationPoint:wait(101): This cancellation point hasn't been closed, please report this stack trace to support:
"java.lang.RuntimeException: 
	at com.complexible.common.cancellation.DefaultCancellationPoint.<init>(DefaultCancellationPoint.java:52) ~[stardog-utils-common-8.1.0.jar:?]
	at com.complexible.common.cancellation.CancellationManager.registerCancellationPoint(CancellationManager.java:178) ~[stardog-utils-common-8.1.0.jar:?]
	at com.complexible.stardog.KernelQueryManagerImpl$KernelQueryMonitor.registerCP(KernelQueryManagerImpl.java:326) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.process.impl.KernelProcessManagerImpl$KernelProcessMonitor.makeCancellable(KernelProcessManagerImpl.java:311) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.process.impl.KernelProcessManagerImpl$KernelProcessMonitor.access$600(KernelProcessManagerImpl.java:260) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.process.impl.KernelProcessManagerImpl.registerProcess(KernelProcessManagerImpl.java:176) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.KernelQueryManagerImpl$KernelQueryMonitor.preExecution(KernelQueryManagerImpl.java:306) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.plan.eval.QueryEngine.getExecutablePlan(QueryEngine.java:708) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.plan.eval.QueryEngine.executePlan(QueryEngine.java:436) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.plan.eval.QueryEngine.executePlan(QueryEngine.java:228) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.plan.eval.QueryEngine.executePlan(QueryEngine.java:416) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.plan.eval.QueryEngine.executeSelectPlan(QueryEngine.java:375) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.plan.eval.QueryEngine.executeSelect(QueryEngine.java:358) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.query.DefaultQueryFactory$SelectQuery.execute(DefaultQueryFactory.java:270) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.query.DefaultQueryFactory$SelectQuery.execute(DefaultQueryFactory.java:256) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.query.DelegatingQuery.execute(DelegatingQuery.java:68) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.db.DatabaseImpl$TimerPausingReadQuery.execute(DatabaseImpl.java:1803) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.db.DatabaseImpl$TimerPausingReadQuery.execute(DatabaseImpl.java:1790) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.query.DelegatingQuery.execute(DelegatingQuery.java:60) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.StardogKernel$SecuredReadQuery.execute(StardogKernel.java:4904) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.StardogKernel$SecuredReadQuery.execute(StardogKernel.java:4891) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.query.DelegatingQuery.execute(DelegatingQuery.java:60) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.MonitoredQuery$MonitoredReadQuery.execute(MonitoredQuery.java:53) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.MonitoredQuery$MonitoredReadQuery.execute(MonitoredQuery.java:35) ~[stardog-8.1.0.jar:?]
	at com.complexible.stardog.protocols.http.server.ProtocolUtils.executeReadQuery(ProtocolUtils.java:542) ~[stardog-protocols-http-server-8.1.0.jar:?]
	at com.complexible.stardog.protocols.http.server.ProtocolUtils.executeReadQuery(ProtocolUtils.java:515) ~[stardog-protocols-http-server-8.1.0.jar:?]
	at com.complexible.stardog.protocols.http.server.SPARQLProtocol.executeQuery(SPARQLProtocol.java:250) ~[stardog-protocols-http-server-8.1.0.jar:?]
	at com.complexible.stardog.protocols.http.server.SPARQLProtocol.postSparqlQuery(SPARQLProtocol.java:174) ~[stardog-protocols-http-server-8.1.0.jar:?]
	at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-8.1.0.jar:?]
	at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.9.1.jar:1.9.1]
	at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.9.1.jar:1.9.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_342]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_342]
	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_342]"

I'll try and get the rest of the logs for this time period from CloudWatch, too.

6

Here's a subset of them. Let me know if I need to go farther back. The consultants are working on a load script where they need to check if a named graph exists, drop it if it does, and then load new data. I'm not sure what steps they took this morning but any operations should be logged in here.

stardog.log (1.2 MB)

7

We will inspect the logs but for now the only way to get rid of these queries is to restart the server.

Best,
Pavel

8

Thanks - I appreciate the fast responses.

9

maybe I'm missing something, but why are you then not dropping the graph always? If no such graph exists, then it doesn't matter or you just use SILENT key word, e.g.
DROP SILENT GRAPH <http://data.org/g1>

1 Like
10

Thank you! This is what I recommended, but we were working async because of time zone differences so that was their original implementation.