Running out of space when loading Wikidata


I am loading Wikidata into Stardog using a Docker container created as follows:

$ docker run -it \
  -v ~/data/wikidata/stardog/stardog-home/:/var/opt/stardog \
  -v ~/data/wikidata/wikidata-20230522-all-BETA-splitted/:/wikidata \
  -p 5820:5820 \
  -e STARDOG_SERVER_JAVA_ARGS="-Xmx60g -Xms60g -XX:MaxDirectMemorySize=160g" \

The /wikidata directory contains a Wikidata dump split into several small ttl.gz files.

I also created a file in the /opt/var/stardog directory with the following lines:


Then, I started loading in the container:

$ /opt/stardog/bin/stardog-admin db create -n wikidata /wikidata/x*.nt.gz

After starting the load, I got the following error message: No space left on device

So, I checked the space in the container, and it says:

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
overlay               3.9G  1.9G  1.8G  53% /
tmpfs                  64M     0   64M   0% /dev
shm                    64M     0   64M   0% /dev/shm
/dev/mapper/vg0-lv01  5.4T  2.2T  3.1T  42% /wikidata
/dev/mapper/vg0-lv02  3.9G  1.9G  1.8G  53% /etc/hosts
tmpfs                 126G     0  126G   0% /proc/acpi
tmpfs                 126G     0  126G   0% /proc/scsi
tmpfs                 126G     0  126G   0% /sys/firmware

My first question is why /var/opt/stardog does not appear as a mounted directory with the df command. My second question is which disk is getting out of space. Since /var/opt/stardog corresponds to a folder in the host system, it should have enough space. I guess that it can be related to a temporary directory used by Stardog when loading data. Perhaps, it is the temporary directory for IO using by Java, which is described in the link below, and can be set in using the property If this is the case, then I could just create a volume from this directory. Am I right?

Try adding the following to your file and rerun. = 104857600


I got the same error:

INFO  2023-08-13T10:59:01,748+0000 [stardog-user-1] com.complexible.stardog.protocols.http.server.StardogUndertowErrorHandler:accept(71): [OPERATION_FAILURE] Server encountered an error No space left on device
        at Method) ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at ~[?:?]
        at com.complexible.common.nio.Channels2.writeFullyAtPosition( ~[stardog-utils-common-9.1.0.jar:?]
        at ~[stardog-utils-common-9.1.0.jar:?]
        at ~[stardog-utils-common-9.1.0.jar:?]
        at$Writer.flush( ~[stardog-utils-common-9.1.0.jar:?]
        at com.complexible.stardog.index.disk.compression.AbstractDataCompressor$2.flush( ~[stardog-9.1.0.jar:?]
        at ~[stardog-utils-common-9.1.0.jar:?]
        at ~[stardog-utils-common-9.1.0.jar:?]
        at ~[stardog-utils-common-9.1.0.jar:?]
        at com.complexible.stardog.index.disk.btree.impl.DiskExternalSorter.writeBucket( ~[stardog-9.1.0.jar:?]
        at com.complexible.stardog.index.disk.btree.impl.DiskExternalSorter.flushNow( ~[stardog-9.1.0.jar:?]
        at com.complexible.stardog.index.disk.btree.impl.DiskExternalSorter.lambda$flush$0( ~[stardog-9.1.0.jar:?]
        at$TrustedFutureInterruptibleTask.runInterruptibly( ~[guava-31.1-jre.jar:?]
        at ~[guava-31.1-jre.jar:?]
        at ~[guava-31.1-jre.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker( [?:?]
        at java.util.concurrent.ThreadPoolExecutor$ [?:?]
        at [?:?]

Would you obtain a diagnostics report and attach it to a direct message?



I read the original email this morning before coffee. Missed your direct questions in the last paragraph. My apologies.

I have looked at the diagnostics report and believe you are correct about

It might be easier to:

  1. create a java_tmp directory within /var/opt/stardog
  2. then add "" to your existing STARDOG_SERVER_JAVA_ARGS
  3. restart stardog server
  4. try db create again

The setting I sent you earlier can stay. It might help if your total disk usage gets high once the database is properly loading.

You are not seeing /var/opt/stardog in the df -h command because it is mounted to the same location as /wikidata. Here is a portion of the output from the "mount" command:

/dev/mapper/vg0-lv01 /wikidata ext4 rw,relatime,errors=remount-ro 0 0
/dev/mapper/vg0-lv02 /etc/resolv.conf ext4 rw,relatime 0 0
/dev/mapper/vg0-lv02 /etc/hostname ext4 rw,relatime 0 0
/dev/mapper/vg0-lv02 /etc/hosts ext4 rw,relatime 0 0
/dev/mapper/vg0-lv01 /var/opt/stardog ext4 rw,relatime,errors=remount-ro 0 0

Both /var/opt/stardog and /wikidata are mounted on device /dev/mapper/fb0-lv01. So only one is used as the label in df -h command. The same is true for the 3 mounts to vg0-lv02, only one appears in df -h.


Thank Mathew! It seems it is working because directory java_tmp is growing. I will inform you if the loading was successful.


I confirm that Stardog finished loading the Wikidata dump.

Loaded 17,954,260,186 triples to wikidata from 22,156 file(s) in 17:15:41.580 @ 288.9K triples/sec.
Successfully created database 'wikidata'.

Congratulations and thank you.