Hi community,
I have two databases that take the same data (don’t want to explain this). I thought I could load the data concurrently. However, stardog is not liking thread pools, and the data won’t load.
Here is my code:
private ExecutorService fixedThreadPool = Executors.newFixedThreadPool(2);
// load data concurrently
public void loadDataByPathConcurrent(String path, RDFFormat f) {
if(path != "") {
fixedThreadPool.execute(new Runnable() {
@Override
public void run() {
Connection conn = queryConnPool.obtain();
conn.begin();
try {
logger.info("queryDB loading data");
conn.add().io().format(f).stream(new FileInputStream(path));
} catch (StardogException | FileNotFoundException e) {
logger.warning("data cannot load from " + path);
e.printStackTrace();
}
conn.commit();
queryConnPool.release(conn);
logger.info("data committed");
}
});
fixedThreadPool.execute(new Runnable() {
@Override
public void run() {
Connection conn = explainConnPool.obtain();
conn.begin();
try {
logger.info("explainDB loading data");
conn.add().io().format(f).stream(new FileInputStream(path));
} catch(StardogException | FileNotFoundException e) {
logger.warning("data cannot load from " + path);
e.printStackTrace();
}
conn.commit();
explainConnPool.release(conn);
logger.info("data committed");
}
});
logger.info(strategy + " data loaded from " + path);
}
else {
logger.warning(strategy + " data path is empty");
}
}
// load data sequentially
public void loadDataByPathSequential(String path, RDFFormat f) {
if(path != "") {
Connection conn1 = queryConnPool.obtain();
conn1.begin();
Connection conn2 = explainConnPool.obtain();
conn2.begin();
try {
conn1.add().io().format(f).stream(new FileInputStream(path));
conn2.add().io().format(f).stream(new FileInputStream(path));
conn1.commit();
conn2.commit();
queryConnPool.release(conn1);
explainConnPool.release(conn2);
} catch (StardogException | FileNotFoundException e) {
e.printStackTrace();
}
}
}
I have two functions, loadDataByPathConcurrent() and loadDataByPathSequential().
The sequential one works great, it can load the data, but the concurrent one doesn’t do anything.
Is there anything wrong with this code, or my logic ?
Also, it seems that stardog doesn’t perform query in concurrency. I am using stardog community version, and hoping to leverage the concurrent 8 connections to one database (I presume this is right, one database has 8 concurrent connections, not the whole stardog service has 8 concurrent connections),
so that when the query demand is high, I can use up to 8 connections to execute 8 queries and return results at any moment.
private TupleQueryResult r = null; // contains the query results
public TupleQueryResult query(String queryString, boolean enableReasoning) {
fixedThreadPool.execute(new Runnable() {
@Override
public void run() {
logger.info("querying");
if(enableReasoning) {
ReasoningConnection conn = explainConnPool.obtain().as(ReasoningConnection.class);
r = conn.select(queryString).execute();
explainConnPool.release(conn);
}
else {
logger.info("no reasoning");
Connection conn = queryConnPool.obtain();
r = conn.select(queryString).execute();
queryConnPool.release(conn);
}
}
});
return r;
}
However, this function doesn’t do anything…
Am I doing anything wrong? What is the best way to have Stardog work with concurrency? I noticed that Stardog comes with an API for concurrency, but the Java doc is light-documented.
Thanks,
Robert
