"delegating connection" error with Stardog API usage

Hi there, we recently starting using the Stardog API to connect our application to Stardog 7 and sometimes, (perhaps after lots of consecutive writes?) this error fills the logs and Stardog no longer responds to queries. Can you tell me what it means?

        at com.complexible.stardog.api.impl.DelegatingConnection.select(DelegatingConnection.java:210)
        at com.complexible.stardog.api.impl.DelegatingConnection.select(DelegatingConnection.java:210)
        at com.complexible.stardog.api.impl.DelegatingConnection.select(DelegatingConnection.java:210)

This snippet from the stack is missing enough context to understand anything. The delegating connection is simply a construct in the source code to promote reuse. Can you share more details on what's going on? Does "no longer responds to queries" mean that your select() calls are blocking in the client? Can you capture a full stack trace of the client and server using jstack? Can you share the complete stack and/or complete log file?

Here a is log file from our application. Everything starts out working well, but then at some point, it will start throwing StackOverflowError errors for DelegatingConnection. We use the rdf4j api in our application and just recently switched from SPARQLRepository in the rdf4j api to using the StardogRepository repository in stardog api. We were trying to see if it would help with some performance issues we were seeing, but unfortunately we sometimes get the StackOverflowError. Now sure why, so any help you can give us would be appreciated.
darklight.2020-08-15_0551.log (2.1 MB)

Thanks for the log. We'll take a look and get back to you.

Could you share a code snippet showing how you are going about creating the Connection object(s)?

Does this happen consistently?

This issue does seem to happen consistently. It isn't immediate, but after our application has been for some time, that is when it will usually happen. I don't have an exact timeframe for how long it will take to fail though. Here are some code snippets.

Setting up the connection:

  private RDFDataSourceService createSemanticRepo(final IRDFDataSourceInfo sourceInfo) {
    final String baseUrl = sourceInfo.getUrl();
    final String id = sourceInfo.getRepositoryId();
    final String username = sourceInfo.getUsername();

    // Get the password from secure storage
    String password = "";
    if (sourceInfo instanceof IEncryptedDataValue) {
      byte[] pwd = ((IEncryptedDataValue) sourceInfo).getSecurePassword();
      if (pwd != null && pwd.length > 0) {
        password = new String(pwd);
      }
    } else {
      password = new String(sourceInfo.getPassword());
    }

    final Repository repo;
    if (sourceInfo.getConnectionType() == RDFConnectionType.STARDOG) {
      String queryEndpoint = baseUrl + "/" + id;
      repo = new StardogRepository(ConnectionConfiguration.from(queryEndpoint).reasoning(sourceInfo.isReasoner()).credentials(username, password));
    } else {
      repo = new HTTPRepository(baseUrl, id);
      ((HTTPRepository) repo).setHttpClientSessionManager(new CustomHttpClientSessionManager());
      if (username != null && password != null) {
        ((HTTPRepository) repo).setUsernameAndPassword(username, password);
      }
    }

    repo.init();
    try (RepositoryConnection conn = repo.getConnection()) {
      ImportLoader loader = new ImportLoader(this.importResolver, conn.getParserConfig(), conn.getValueFactory());
      SemanticRepository semanticRepository = new SemanticRepository(id, repo, this.settings, loader);
      return new RDFDataSourceService(semanticRepository, sourceInfo.getId(), sourceInfo.isReasoner(), this.job);
    }
  }

SemanticQuery class:

  private T internalExecute(final IPipelineContextLogger contextLogger) throws MemoryException {
    T result = createEmptyResult();
    long startTime = System.currentTimeMillis();
    String query = "";

    try (RepositoryConnection conn = this.repo.getConnection()) {
      for (int i = 0; i < this.queries.length; i++) {
        query = this.queries[i];
        O op = createOperation(query, conn);
        if (this.dataset != null) {
          op.setDataset(this.dataset);
        }
        op.setIncludeInferred(this.reasoning);
        for (Entry<String, Value> entry : this.bindings.entrySet()) {
          op.setBinding(entry.getKey(), entry.getValue());
        }
        LOG.trace("Query to {}:\n {}", this.repo.toString(), op.toString());
        result = executeOperation(op, result);
      }

      return result;
    } catch (RDF4JException e) {
      e.printStackTrace();
      throw new ChampionException("Failed to execute query " + query, e);
    } 
  }

Semantic Tuple Query:

class SemanticTupleQuery extends SemanticQuery<TupleQuery, List<Value[]>> {
  public SemanticTupleQuery(final String[] queries, final Repository repo, final Dataset dataset, final ImportLoader loader) {
    super(queries, repo, dataset, loader);
  }

  @Override
  protected TupleQuery createOperation(final String query, final RepositoryConnection conn) throws RDF4JException {
    return conn.prepareTupleQuery(QueryLanguage.SPARQL, query);
  }

  @Override
  protected List<Value[]> executeOperation(final TupleQuery tupleQuery, final List<Value[]> tuples) throws RDF4JException {
    try (TupleQueryResult result = tupleQuery.evaluate()) {
      List<String> vars = result.getBindingNames();
      int n = vars.size();
      while (result.hasNext()) {
        Value[] tuple = new Value[n];
        BindingSet binding = result.next();
        for (int i = 0; i < n; i++) {
          String var = vars.get(i);
          tuple[i] = binding.getValue(var);
        }
        tuples.add(tuple);
      }
    }

    return tuples;
  }

  @Override
  protected List<Value[]> createEmptyResult() {
    return new ArrayList<>();
  }
}

Semantic Update Query:

class SemanticUpdateQuery extends SemanticQuery<Update, Void> {
  public SemanticUpdateQuery(final String[] queries, final Repository repo, final Dataset dataset, final ImportLoader loader) {
    super(queries, repo, dataset, loader);
  }

  @Override
  protected Update createOperation(final String query, final RepositoryConnection conn) throws RDF4JException {
    return conn.prepareUpdate(QueryLanguage.SPARQL, query);
  }

  @Override
  protected Void executeOperation(final Update updateQuery, final Void result) throws RDF4JException {
    updateQuery.execute();
    return result;
  }

  @Override
  protected Void createEmptyResult() {
    return null;
  }		
}

I hope this helps. Please let me know if you need anything else.

Hello, I was just wondering if there was any progress on this? Do you need anymore information from us?

Any luck with tracking down the issue? We are still fighting it and it is happening fairly often. So any help would be greatly appreciated. Thanks.

This is still an issue for us. Is it still being looked into? We would really appreciate an update if possible. Thank you.

Alex,

Apologies, we haven't seen anything like that before. Given that there's a delay before the error happens, one possible theory is that it's somehow related to the connection pool behind the StardogRepository object. Unfortunately it's not configurable at this point. However, you can experiment with creating more connections, for example, by creating a new connection per operation/query, to see if it makes it happen faster (or if there's any correlation).

If you could create an independent test for us to reproduce the issue, that'd be ideal, but I understand it may not be easy given the non-deterministic nature of the problem.

Thanks,
Pavel