Stardog.log missing information when peforming a "data add"?

Support
1

Hi,

Until recently, while doing a “data add” command with a turtle file, I could follow the upload in the stardog.log that was updated with informations as:
INFO 2017-12-18 12:02:31,072 [Stardog.Executor-1159] com.complexible.stardog.StardogKernel:printInternal(314): Parsing triples: 2% complete in 00:00:08 (1.8M triples - 201.2K tr
iples/sec)

and:
INFO 2017-12-18 12:11:22,875 [Stardog.Executor-1290] com.complexible.stardog.StardogKernel:printInternal(314): Indexing triples: 17% complete in 00:00:06 (2432.2K triples/sec)

But since a few days, I no more see those information.

I only see, at the beginning of the upload, lines like:
INFO 2017-12-19 10:29:41,582 [XNIO-1 task-19] com.complexible.stardog.dht.impl.PagedDiskHashTable:write(480): Preparing cache: 20% complete in 00:00:06

at the end of the upload, lines like :
INFO 2017-12-19 10:48:03,015 [XNIO-1 task-21] com.complexible.stardog.search.waldo.DefaultIndexer:print(381): Indexing text: 2.0M literals in 00:00:10.943

This did happen (no more having those lines) after I renamed the current “stardog.log” to restart with an empty one.

Any tip about how I could see those lines again ?
Thanks
Fabian

2

You can try restarting Stardog and see if that clears things up.

stardog-admin server stop && stardog-admin server start

I can imagine odd things happening depending on how you cleared the logs, truncate, delete, etc

You can configure Stardog to do log rotation for you if that is what you need

logging.access.enabled = true
logging.access.type = text
logging.access.file = access.log
logging.access.rotation.type = size
logging.access.rotation.limit = 1000000

logging.audit.enabled = true
logging.audit.type = binary
logging.audit.file = audit.log
logging.audit.rotation.type = time
logging.audit.rotation.interval = 1d
3

Well, Stardog was restarted a few times since then, and I still have the same behavior (no “parsing triples” or “indexing triples” in stardog.log).

Then you talk about log rotation, but this is about other log files, and not stardog.log, is that right ? (access.log, audit.log)

Thank you
Fabian

4

I figured there was a good chance you’d already tried restarting but I thought I’d check.

That’s correct. I’m being a dummy. (haven’t had my morning coffee yet). That’s right, it’s not going to rotate stardog.log but I’m not sure how it’s going to react if you delete the file and recreate it or if you truncate it while Stardog is running.

What version of Stardog are you running?

5

:wink:

it’s version 5.0.5.1

6

Hi Fabian,

Nothing has changed wrt logging progress in the recent versions and renaming the file should not make a difference after restart. Note that the progress for each phase (Parsing, Indexing, etc.) will be displayed only if that phase takes more than 5 seconds. It is possible some phases will be skipped while others are shown. Is it possible those phases completed too quickly for the progress to be printed?

The other possibility is a change in the log4j2 config. The progress is displayed via the log4j logger and you can change the logger behavior with a log4j2 config file in $STARDOG_HOME. Is there a custom log config file?

Best,
Evren

7

The upload is a 90 million triples upload, it takes20 to 80 minutes, so the different phases should appear I guess. (They did appear before I did rename the stardog.log file to “stardog_20171218_beforeCrash.log”.
Then a new “stardog.log” was created.
We had some VM crash because of memory errors, and then the log did change behavior.

Finally, I do not see any log2j confi in the folder, here is my list of files:
drwxr-xr-x 3 ais ais 4096 Nov 6 19:50 bin/
drwxr-xr-x 8 ais ais 4096 Nov 6 19:50 client/
drwxr-xr-x 4 ais ais 4096 Nov 6 19:50 docs/
-rw-rw-r-- 1 ais ais 1096 Dez 6 11:53 license.bin
drwxr-xr-x 5 ais ais 4096 Nov 6 19:50 server/
drwxrwxr-x 2 ais ais 4096 Dez 19 10:16 .spilling/
-rw-rw-r-- 1 ais ais 1654618508 Dez 18 14:04 stardog_20171218_beforeCrash.log
-rw-rw-r-- 1 ais ais 711686 Dez 21 08:46 stardog.log
drwxrwxr-x 3 ais ais 4096 Nov 29 11:48 system/
-rw-rw-r-- 1 ais ais 0 Nov 29 11:48 system.lock
drwxrwxr-x 6 ais ais 4096 Dez 18 13:50 test/
-rw-rw-r-- 1 ais ais 3 Dez 18 09:16 test2.txt
-rw-rw-r-- 1 ais ais 10 Dez 18 09:24 test3.txt
drwxr-xr-x 3 ais ais 4096 Nov 6 19:50 webconsole/
-rw-rw-r-- 1 ais ais 0 Nov 29 11:48 zookeeper.log

8

You might also want to check the default log4j2.xml configuration in $STARDOG/server/dbms/log4j2.xml in case somebody changed it. I’m not 100% sure, but it might also look for a configuration file in the home directory of whatever user is running the stardog server process

9

Hi Stephen,

So I did find only one log4j2.xml, the one from /server/dbms folder. And it was not changed as I am the only one working on this test system.

But can you tell me what the configuration should be to see the detailed upload messages as explained at the beginning of this thread ? I could try to create a new log4j2.xml for the STARDOG home folder, right ?

Thanks for your help
Fabian

10

Fabian,

You could indeed create a new log4j2.xml file for STARDOG_HOME. I would copy the one from server/dbms into STARDOG_HOME and then make sure that the following is present under Loggers:

<Logger name="com.complexible.stardog" level="INFO" additivity="false">
			<AppenderRef ref="stardogAppender"/>
		</Logger>
11

Hi Fabian,

We finally identified the root cause for this issue and it is not related to log configuration. We will have the fix included in the next release.

Best,
Evren

12

Great. I did try some configuration as proposed, and, yes, it did not work. So waiting for the fix :slight_smile:

13

In the meantime - when I clear (or rename) the full $STARDOG folder, it worked again, for me.

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.