# java.lang.OutOfMemoryError: Java heap space
**URL:** https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240
**Category:** Support
**Created:** [April 15, 2017, 3:32pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240 "2017-04-15T15:32:09Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@yayamamo](https://community.stardog.com/u/yayamamo)
#### Post date: [April 15, 2017, 3:32pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/1 "2017-04-15T15:32:09Z")
Hi,
I loaded 8.2 billion triples into Stardog 4.2.4, and the following message is in the stardog.log and no result was obtained when I issued a SPARQL query such as listing up all the classes in the data.
`java.lang.OutOfMemoryError: Java heap space`
I set the following environment variables.
`STARDOG_JAVA_ARGS=-Xms16g -Xmx16g -XX:MaxDirectMemorySize=256g`
So, could you tell me what I should do to work around this issue?
The followings are the memory options recorded in the log file.
[main] com.complexible.stardog.cli.impl.ServerStart:call(250): Min Heap Size: 16G
[main] com.complexible.stardog.cli.impl.ServerStart:call(251): Max Heap Size: 15G
[main] com.complexible.stardog.cli.impl.ServerStart:call(252): Max Off-Heap : 256G
[main] com.complexible.stardog.cli.impl.ServerStart:call(253): System Memory: 1.2T
Regards.
---
### Author:  [@zachary.whitley](https://community.stardog.com/u/zachary.whitley)
#### Post date: [April 15, 2017, 5:25pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/2 "2017-04-15T17:25:11Z")
You may want to consider allocating more heap space.
---
### Author:  [@hmottestad](https://community.stardog.com/u/hmottestad)
#### Post date: [April 15, 2017, 8:10pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/3 "2017-04-15T20:10:52Z")
We’ve had a lot of success with overly large heap size. For our workload we’ve not seen much allocation from our off-heap. I would recommend experimenting, maybe trying to have 64GB heap and 128GB off-heap.
---
### Author:  [@zachary.whitley](https://community.stardog.com/u/zachary.whitley)
#### Post date: [April 15, 2017, 8:56pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/4 "2017-04-15T20:56:10Z")
This may not be a problem as soon as Stardog 5 comes out.
---
### Author:  [@yayamamo](https://community.stardog.com/u/yayamamo)
#### Post date: [April 16, 2017, 4:20pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/5 "2017-04-16T16:20:16Z")
I cannot wait for the release! Could you tell me when?
---
### Author:  [@yayamamo](https://community.stardog.com/u/yayamamo)
#### Post date: [April 21, 2017, 2:02am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/6 "2017-04-21T02:02:10Z")
Well, as this document ([Home | Stardog Documentation Latest](https://docs.stardog.com/#_capacity_planning)) says, 16GB JVM and 256GB Off-hap memory are recommended for 50 billion, I set the above parameters even though our dataset has only 8.2 billion triples, but I learnt that it needs more. This is unpractical for us to use.
---
### Author:  [@evren](https://community.stardog.com/u/evren)
#### Post date: [April 21, 2017, 3:55pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/7 "2017-04-21T15:55:40Z")
What method were you using to load the data? For billion triples and more the recommended way is to use `db create` command with the files on the server side.
Best,
Evren
---
### Author:  [@yayamamo](https://community.stardog.com/u/yayamamo)
#### Post date: [April 22, 2017, 1:23pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/8 "2017-04-22T13:23:41Z")
I loaded the data as follows:
`stardog data add -g --server-side http://localhost:1111/MyDB <.nt files>`
Since there are many named graphs, I cannot load the entire data at once when creating the DB.
---
### Author:  [@pavel](https://community.stardog.com/u/pavel)
#### Post date: [April 23, 2017, 7:16pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/9 "2017-04-23T19:16:18Z")
You can still use the `db create` command if you put each named graph IRIs into the files which should be loaded into that named graph, for example, using the TriG format. It’s likely it will be faster than `data add`.
Cheers,
Pavel
---
### Author:  [@evren](https://community.stardog.com/u/evren)
#### Post date: [April 23, 2017, 9:51pm UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/10 "2017-04-23T21:51:15Z")
You can also use the API which allows you to define the named graph for each file. See the example code here:
>
Best,
Evren
---
### Author:  [@yayamamo](https://community.stardog.com/u/yayamamo)
#### Post date: [April 24, 2017, 1:24am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/11 "2017-04-24T01:24:58Z")
The problem I have is to get OutOfMemoryError when I issue a SPARQL query. Will to load an RDF dataset of 8.2 billion triples at creating a db be the solution to this?
---
### Author:  [@evren](https://community.stardog.com/u/evren)
#### Post date: [April 24, 2017, 2:10am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/12 "2017-04-24T02:10:02Z")
Sorry, no, loading and query answering are not related. I mixed up this thread with the other question you were asking about loading.
The very large direct memory recommendations in the documentation are for getting the best loading speed. During query answering Stardog 4 does not use direct memory very much so you should increase the heap memory. You would see a warning about -XX:+UseCompressedOops when you increase the heap size to more than 32gb but you can ignore this.
It is also possible that something in your query might be improved for better performance and/or memory usage. If you can share the query along with the query plan printed by the `stardog query explain` command then we can provide some suggestions.
Best,
Evren
PS: Stardog 5 beta version will be released in a week or so. Stardog 5 uses direct memory heavily for query answering and it will have different modes for memory usage. We will provide detailed information about that in the documentation.
---
### Author:  [@yayamamo](https://community.stardog.com/u/yayamamo)
#### Post date: [April 24, 2017, 3:51am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/13 "2017-04-24T03:51:22Z")
Thank you for your information.
After modifying the memory environment, I got the following errors when issuing some SPARQL queries including the query of [[Stardog query plan for query 2 · GitHub](https://gist.github.com/yayamamo/923303c1e598c39b982f36ca81a6ba25)], while OutOfMemoryError doesn't occur now.
> ERROR 2017-04-24 11:44:04,751 [SPEC-Server-1-12] com.complexible.common.protocols.server.rpc.ServerHandler:exceptionCaught(401): exceptionCaughtServerHandler
> ERROR 2017-04-24 11:44:04,755 [StardogServer.WorkerGroup-11] com.complexible.stardog.protocols.http.server.HttpMessageEncoder:createErrorResponse(293): The result encoder received an error message it could not encode, error was:
> java.io.IOException: 接続が相手からリセットされました
> ERROR 2017-04-24 12:40:00,013 [StardogServer.WorkerGroup-7] com.complexible.stardog.protocols.http.server.HttpMessageEncoder:write(171): There was an error writing the HTTP response
> org.openrdf.query.QueryEvaluationException: com.complexible.stardog.plan.eval.operator.OperatorException: Query execution cancelled: Execution time exceeded query timeout 300000
Best regards
---
### Author:  [@pavel](https://community.stardog.com/u/pavel)
#### Post date: [April 24, 2017, 7:14am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/14 "2017-04-24T07:14:31Z")
This means the query execution time exceeded the default timeout value and the server killed it.
This might indicate either a hard query (given the amount of data) or a sub-optimal query plan. If you send us the output of the `query explain` command for that query, we might be able to tell. If it’s sensitive, feel free to send privately.
You may also disable the timeout (or set it to a higher value) using the `query.timeout` property in `stardog.properties`. `query.timeout=0m` will disable it.
Cheers,
Pavel
---
### Author:  [@yayamamo](https://community.stardog.com/u/yayamamo)
#### Post date: [April 24, 2017, 7:16am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/15 "2017-04-24T07:16:12Z")
Thanks, and here is the query plan I got.
>
Best regards
---
### Author:  [@pavel](https://community.stardog.com/u/pavel)
#### Post date: [April 24, 2017, 7:26am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/16 "2017-04-24T07:26:00Z")
The problem are the loop joins which usually indicate disconnected queries (i.e. there are parts of the query which do not have a join condition). Can we see the query?
---
### Author:  [@yayamamo](https://community.stardog.com/u/yayamamo)
#### Post date: [April 24, 2017, 7:48am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/17 "2017-04-24T07:48:56Z")
This is the query.
```
PREFIX rdf:
PREFIX rdfs:
PREFIX skos:
PREFIX up:
PREFIX faldo:
SELECT DISTINCT ?parent_label ?label ?begin_location ?end_location ?seq_length ?comment (GROUP_CONCAT(?substitution; SEPARATOR = ", ") AS ?substitutions) ?seq ?feature_identifier
FROM
FROM
WHERE {
{
SELECT ?gene
{
skos:exactMatch ?gene .
} ORDER BY ?gene LIMIT 1
}
skos:exactMatch ?gene ;
rdfs:seeAlso ?id_upid .
?id_upid rdfs:seeAlso ?protein .
?protein a up:Protein ;
up:annotation ?annotation .
?annotation rdf:type ?type .
?type rdfs:label ?label .
?type rdfs:subClassOf* ?parent_type .
?parent_type rdfs:subClassOf up:Sequence_Annotation ;
rdfs:label ?parent_label .
?annotation up:range ?range .
OPTIONAL { ?annotation rdfs:comment ?comment . }
?range faldo:begin/faldo:position ?begin_location ;
faldo:end/faldo:position ?end_location .
?protein up:sequence ?isoform .
BIND( REPLACE( STR(?protein), "http://purl.uniprot.org/uniprot/", "") AS ?up_id)
FILTER( REGEX(STR(?isoform), ?up_id))
?isoform rdf:value ?value .
OPTIONAL {
?annotation up:substitution ?substitution .
?isoform rdf:value ?seq .
}
OPTIONAL {
?isoform rdf:value ?seq_txt .
BIND (STRLEN(?seq_txt) AS ?seq_length) .
}
OPTIONAL {
?annotation rdf:type ?type .
BIND (STR(?annotation) AS ?feature_identifier) .
FILTER REGEX(STR(?annotation), "http://purl.uniprot.org/annotation")
}
}
GROUP BY ?parent_label ?label ?begin_location ?end_location ?seq_length ?comment ?seq ?feature_identifier
ORDER BY ?parent_label ?label ?begin_location ?end_location
```
---
### Author:  [@pavel](https://community.stardog.com/u/pavel)
#### Post date: [April 24, 2017, 8:14am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/18 "2017-04-24T08:14:56Z")
The main problem is this part:
```auto
OPTIONAL {
?annotation up:substitution ?substitution .
?isoform rdf:value ?seq .
}
```
Currently Stardog evaluates this pattern separately from the rest which results in a Cartesian product. This is a known problem and we have an open ticket for it.
In most cases, however, the intention would be to do
```auto
OPTIONAL {
?annotation up:substitution ?substitution .
}
OPTIONAL {
?isoform rdf:value ?seq .
}
```
(possibly with the reverse order). That’d eliminate the loop join and the query will be faster (can’t say without trying how much faster though since there’re other complex parts in the query, e.g. grouping on many variables).
PS. There’s already a `?isoform rdf:value ?seq_txt .` in another OPTIONAL in your query so maybe you can drop `?isoform rdf:value ?seq` completely.
PPS. `?annotation rdf:type ?type .` is both in the main pattern and in an OPTIONAL block which is unnecessary. If you remove it from the OPTIONAL, it’ll eliminate a potentially expensive join.
---
### Author:  [@yayamamo](https://community.stardog.com/u/yayamamo)
#### Post date: [April 28, 2017, 4:28am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/19 "2017-04-28T04:28:17Z")
Thank you very much for telling me the valuable information.
Now the performance has been improved.
---
### Author:  [@system](https://community.stardog.com/u/system)
#### Post date: [May 12, 2017, 4:28am UTC](https://community.stardog.com/t/java-lang-outofmemoryerror-java-heap-space/240/20 "2017-05-12T04:28:29Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.