Server won't start after crash during load on large repository ( 195M )

Support Bug
1 
INFO  2017-11-22 16:49:13,676 [main] com.complexible.stardog.virtual.DefaultVirtualGraphRegistry:syncCache(165): Initializing R2RML registry
    INFO  2017-11-22 16:49:13,727 [main] com.complexible.stardog.virtual.DefaultVirtualGraphRegistry:syncCache(174): Loaded R2RML registry with 0 sources
    INFO  2017-11-22 16:49:16,571 [main] com.complexible.stardog.StardogKernel:start(2248): Initializing Stardog
    WARN  2017-11-22 16:49:22,524 [main] com.complexible.stardog.dht.dictionary.HashDictionary:<init>(299): Dictionary is corrupted, will try to repair automatically
    java.io.IOException: Cannot read pges on disk: 662219292 != 931415373
            at com.complexible.stardog.dht.impl.PagedDiskHashTable.readPagesOnDisk(PagedDiskHashTable.java:394) ~[stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.dht.impl.PagedDiskHashTable.readFooter(PagedDiskHashTable.java:363) ~[stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.dht.impl.PagedDiskHashTable.read(PagedDiskHashTable.java:299) ~[stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.dht.impl.PagedDiskHashTable.<init>(PagedDiskHashTable.java:205) ~[stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.dht.impl.PagedDiskHashTableBuilder.build(PagedDiskHashTableBuilder.java:161) ~[stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.dht.dictionary.HashDictionary.<init>(HashDictionary.java:289) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.index.disk.DefaultDiskIndexReader.toMappingDictionary(DefaultDiskIndexReader.java:138) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.index.disk.DefaultDiskIndexReader.read(DefaultDiskIndexReader.java:125) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.index.io.IndexIO.read(IndexIO.java:319) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.db.DatabaseFactoryImpl.read(DatabaseFactoryImpl.java:126) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.db.DatabaseFactoryImpl.read(DatabaseFactoryImpl.java:55) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.StardogKernel.openDatabase(StardogKernel.java:2582) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.StardogKernel.initDatabases(StardogKernel.java:2315) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.StardogKernel.start(StardogKernel.java:2250) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.StardogKernel.initialize(StardogKernel.java:782) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.Stardog.initKernel(Stardog.java:219) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.Stardog.<init>(Stardog.java:211) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.Stardog.<init>(Stardog.java:65) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.Stardog$StardogBuilder.create(Stardog.java:560) [stardog-5.0.5.1.jar:?]
            at com.complexible.stardog.cli.impl.ServerStart.call(ServerStart.java:145) [stardog-cli-5.0.5.1.jar:?]
            at com.complexible.stardog.cli.impl.ServerStart.call(ServerStart.java:44) [stardog-cli-5.0.5.1.jar:?]
            at com.complexible.stardog.cli.CLIBase.execute(CLIBase.java:55) [stardog-cli-5.0.5.1.jar:?]
            at com.complexible.stardog.cli.admin.CLI.main(CLI.java:189) [stardog-cli-5.0.5.1.jar:?]
    INFO  2017-11-22 16:49:32,718 [main] com.complexible.stardog.dht.dictionary.HashDictionary:printInternal(314): Repairing dictionary: 1% complete in 00:00:10
    INFO  2017-11-22 16:49:39,767 [main] com.complexible.stardog.dht.dictionary.HashDictionary:printInternal(314): Repairing dictionary: 2% complete in 00:00:17
    INFO  2017-11-22 16:49:44,524 [main] com.complexible.stardog.dht.dictionary.HashDictionary:printInternal(314): Repairing dictionary: 3% complete in 00:00:21
    INFO  2017-11-22 16:49:48,898 [main] com.complexible.stardog.dht.dictionary.HashDictionary:printInternal(314): Repairing dictionary: 4% complete in 00:00:26
    INFO  2017-11-22 16:49:53,314 [main] com.complexible.stardog.dht.dictionary.HashDictionary:printInternal(314): Repairing dictionary: 5% complete in 00:00:30
    Loading Databases: 100% complete in 00:00:43
    INFO  2017-11-22 16:50:00,548 [main] com.complexible.stardog.StardogKernel:write(77): Loading Databases: 100% complete in 00:00:43
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
            at com.complexible.stardog.dht.impl.PagedDiskHashTable.readPagesOnDisk(PagedDiskHashTable.java:391)
            at com.complexible.stardog.dht.impl.PagedDiskHashTable.readFooter(PagedDiskHashTable.java:363)
            at com.complexible.stardog.dht.impl.PagedDiskHashTable.read(PagedDiskHashTable.java:299)
            at com.complexible.stardog.dht.impl.PagedDiskHashTable.<init>(PagedDiskHashTable.java:205)
            at com.complexible.stardog.dht.impl.PagedDiskHashTableBuilder.build(PagedDiskHashTableBuilder.java:161)
            at com.complexible.stardog.dht.dictionary.HashDictionary.<init>(HashDictionary.java:289)
            at com.complexible.stardog.index.disk.DefaultDiskIndexReader.toMappingDictionary(DefaultDiskIndexReader.java:138)
            at com.complexible.stardog.index.disk.DefaultDiskIndexReader.read(DefaultDiskIndexReader.java:125)
            at com.complexible.stardog.index.io.IndexIO.read(IndexIO.java:319)
            at com.complexible.stardog.index.io.IndexIO.repair(IndexIO.java:1026)
            at com.complexible.stardog.db.DatabaseFactoryImpl.read(DatabaseFactoryImpl.java:136)
            at com.complexible.stardog.db.DatabaseFactoryImpl.read(DatabaseFactoryImpl.java:55)
            at com.complexible.stardog.StardogKernel.openDatabase(StardogKernel.java:2582)
            at com.complexible.stardog.StardogKernel.initDatabases(StardogKernel.java:2315)
            at com.complexible.stardog.StardogKernel.start(StardogKernel.java:2250)
            at com.complexible.stardog.StardogKernel.initialize(StardogKernel.java:782)
            at com.complexible.stardog.Stardog.initKernel(Stardog.java:219)
            at com.complexible.stardog.Stardog.<init>(Stardog.java:211)
            at com.complexible.stardog.Stardog.<init>(Stardog.java:65)
            at com.complexible.stardog.Stardog$StardogBuilder.create(Stardog.java:560)
            at com.complexible.stardog.cli.impl.ServerStart.call(ServerStart.java:145)
        at com.complexible.stardog.cli.impl.ServerStart.call(ServerStart.java:44)
        at com.complexible.stardog.cli.CLIBase.execute(CLIBase.java:55)
        at com.complexible.stardog.cli.admin.CLI.main(CLI.java:189)
Waiting for running tasks to complete...done. Executor service has been shut down.
Stardog server 5.0.5.1 shutdown on Wed Nov 22 16:50:00 CET 2017.

So we now have non functioning server, any help greatly appreciated.

2

What are the STARDOG_JAVA_ARGS being used to run the server? e.g., how much memory is dedicated to the server? Your stacktrace mentions running out of java heap space.

3

Well that much I figured as well :slight_smile:

We use that standard setting for stardog, the machine has 4G totally available.

I’d rather have Stardog gracefully fail initializing the database in stead of not starting at all.

the memory errors are the only thing we really dislike about stardog so far.

Bart