Loading Yago into Stardog Community

I have been loading Yago for over 4 hours by now, on a 32GB RAM laptop with a very fast SSD. You mentioned faster Yago queries under 5.0, but so far it doesn’t load.

I don’t know how many triples are in Yago but it’s 90Gb uncompressed so it’s going to take a while. You can monitor progress by tailing the stardog.log. They said queries are faster but didn’t say anything about bulk loading which wouldn’t be specific to Yago so you should be able to follow the general guidelines for bulk loading performance at https://docs.stardog.com/#_optimizing_bulk_data_loading

Are you bulk loading, adding the data at db creation time, or adding data after database creation. The former should give you better performance.

You mentioned that you have 32GB of ram but how much memory did you allocate to Stardog? What is the output from running stardog-admin server status?

While trying to load it with 4Gb/8Gb heap/offheap I was through 20% of the parsing stage in about 4hrs. I’ll have to get on my monster machine to see how the load goes when you throw hardware at it. I did notice a warning in the logs while loading and posted a separate report at Error bulk loading

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.