# ICV slow when graph without ICV has 30 million triples

**URL:** <https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239>\
**Category:** Support\
**Created:** [April 15, 2017, 2:41pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239 "2017-04-15T14:41:32Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![stephen](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/stephen/32/27_2.png) [@stephen](https://community.stardog.com/u/stephen)\
**Post date:** [April 21, 2017, 7:05pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/21 "2017-04-21T19:05:41Z")

</div>

> [@hmottestad](#):
>
> [http://data.einnsyn.no/brukermeta/](http://data.einnsyn.no/brukermeta/) or [http://data.einnsyn.no/brukereGraph](http://data.einnsyn.no/brukereGraph) graph?

Ah sorry, I was looking at the prefix declaration, not the graph. It's `<http://data.einnsyn.no/brukereGraph>`

---

<div class="post-metadata">

**Author:** ![stephen](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/stephen/32/27_2.png) [@stephen](https://community.stardog.com/u/stephen)\
**Post date:** [April 21, 2017, 7:06pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/22 "2017-04-21T19:06:43Z")

</div>

Each of those queries are individual inserts into `<http://data.einnsyn.no/brukereGraph>`, or inserts of multiple users?

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 21, 2017, 7:07pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/23 "2017-04-21T19:07:16Z")

</div>

Inserting the same user. Exact same query running multiple times.

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 21, 2017, 7:08pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/24 "2017-04-21T19:08:21Z")

</div>

Before I can give you the rest of the data I need to clean up a lot of my literals.

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 21, 2017, 7:25pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/25 "2017-04-21T19:25:45Z")

</div>

Hi Stephen,

Thanks for your patience 🙂

Here is a dump (obscured) of the entire database: [temp/export (6).trig at master · hmottestad/temp · GitHub](https://github.com/hmottestad/temp/blob/master/export%20(6).trig)

If you add the geospecies file multiple times in separate graphs it should slow down insertions.

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 21, 2017, 7:28pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/26 "2017-04-21T19:28:44Z")

</div>

I think the big difference here might be that I have a graph with all the organisations (metadata) delivering data to the solution, and all those are validated by ICV rules since admins from each org can edit the data online. So a lot more data here that the ICV engine has to handle.

---

<div class="post-metadata">

**Author:** ![stephen](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/stephen/32/27_2.png) [@stephen](https://community.stardog.com/u/stephen)\
**Post date:** [April 24, 2017, 11:33am UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/27 "2017-04-24T11:33:23Z")

</div>

Just to confirm, is this happening with Stardog 4.2.4?

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 24, 2017, 2:56pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/28 "2017-04-24T14:56:50Z")

</div>

Looks like we might still be on 4.2.3. Are you not able to reproduce it in  
4.2.4?

---

<div class="post-metadata">

**Author:** ![stephen](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/stephen/32/27_2.png) [@stephen](https://community.stardog.com/u/stephen)\
**Post date:** [April 24, 2017, 3:07pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/29 "2017-04-24T15:07:27Z")

</div>

No, unless I’m doing something wrong here:

```auto
# Create DB with your trig file and options
Stephens-iMac:community stephen$ stardog-admin db create -n testSesame -o security.named.graphs=true icv.enabled=true icv.reasoning.enabled=true icv.active.graphs=http://data.einnsyn.no/osloKommuneVirksomheterGraph,http://data.einnsyn.no/virksomheterGraph,http://data.einnsyn.no/brukereGraph,http://data.einnsyn.no/innsynskravGraph,http://www.arkivverket.no/standarder/noark5/arkivstruktur/ontologyGraph preserve.bnode.ids=true reasoning.consistency.automatic=true query.all.graphs=true reasoning.type=SL reasoning.schema.graphs=http://www.arkivverket.no/standarder/noark5/arkivstruktur/ontologyGraph -- havard.trig
Bulk loading data to new database testSesame.
Loaded 4,142 triples to testSesame from 1 file(s) in 00:00:00.266 @ 15.6K triples/sec.
Successfully created database 'testSesame'.

# Insert one user into the user graph
Stephens-iMac:community stephen$ time sq testSesame havardInsert.rq
Update query processed successfully in 00:00:00.087.

real	0m1.789s
user	0m3.451s
sys	0m0.260s

# Shell script that adds geospecies data 10 times
Stephens-iMac:community stephen$ ./havard.sh
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:11.618
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:12.404
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:13.574
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:13.485
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:14.008
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:14.264
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:17.401
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:22.053
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:25.347
Adding data from file: geospecies.rdf.gz
Added 2,201,532 triples in 00:00:28.208

# Same insert query
Stephens-iMac:community stephen$ time sq testSesame havardInsert.rq
Update query processed successfully in 00:00:00.097.

real	0m1.784s
user	0m3.195s
sys	0m0.265s

```

If I’m doing something horribly wrong, please let me know. However the second insert takes essentially the same amount of time as the first one.

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 24, 2017, 3:31pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/30 "2017-04-24T15:31:25Z")

</div>

Could I get the scripts?

---

<div class="post-metadata">

**Author:** ![stephen](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/stephen/32/27_2.png) [@stephen](https://community.stardog.com/u/stephen)\
**Post date:** [April 24, 2017, 3:33pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/31 "2017-04-24T15:33:08Z")

</div>

Sure, they’re nothing fancy:

```auto
Stephens-iMac:community stephen$ cat havard.sh
#!/bin/bash

for i in `seq 10`; do
	 stardog data add --named-graph http://www.newgraph.org/$i/ testSesame geospecies.rdf.gz
done

```

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 24, 2017, 3:41pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/32 "2017-04-24T15:41:40Z")

</div>

We are using enterprise trial license, would that make a difference?

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 24, 2017, 3:44pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/33 "2017-04-24T15:44:40Z")

</div>

With that code, I don’t see you actually adding any of the ICV constraints.

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 24, 2017, 3:45pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/34 "2017-04-24T15:45:39Z")

</div>

If you run these two queries, then the second one should fail.

```auto

insert data {

  graph <http://data.einnsyn.no/brukereGraph> {
  	<http://example.com/user1> a bruker:Sluttbruker;
                                 bruker:passord "password";
                                  bruker:epost "test@example.com";
                                 bruker:brukernavn "username1".
  
  }

}

```

Second one

```auto
insert data {

  graph <http://data.einnsyn.no/brukereGraph> {
  	<http://example.com/user2> a bruker:Sluttbruker;
                                 bruker:passord "password";
                                  bruker:epost "test@example.com";
                                 bruker:brukernavn "username1".
  
  }

}

```

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 24, 2017, 3:49pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/35 "2017-04-24T15:49:00Z")

</div>

We have the following in our script to add the ICV constraints:

```auto
  ./stardog-admin icv drop testSesame
  ./stardog-admin icv add testSesame /tmp/virksomhet_constraints.ttl
  ./stardog-admin icv add testSesame /tmp/user_constraints.ttl

```

---

<div class="post-metadata">

**Author:** ![stephen](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/stephen/32/27_2.png) [@stephen](https://community.stardog.com/u/stephen)\
**Post date:** [April 24, 2017, 3:49pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/36 "2017-04-24T15:49:36Z")

</div>

> [@hmottestad](#):
>
> With that code, I don't see you actually adding any of the ICV constraints.

I suppose that could have something to do with it. 😳

Yeah, let me try again with the constraints. Sorry about that!

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 24, 2017, 3:50pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/37 "2017-04-24T15:50:51Z")

</div>

I put them here in case you can't find the email: [GitHub - hmottestad/temp](https://github.com/hmottestad/temp)

---

<div class="post-metadata">

**Author:** ![stephen](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/stephen/32/27_2.png) [@stephen](https://community.stardog.com/u/stephen)\
**Post date:** [April 24, 2017, 6:27pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/38 "2017-04-24T18:27:06Z")

</div>

Yes, now that I’m trying the correct way, I can reproduce this issue in 4.2.4. I’ll look into what’s going on. Thanks!

---

<div class="post-metadata">

**Author:** ![hmottestad](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/hmottestad/32/19_2.png) [@hmottestad](https://community.stardog.com/u/hmottestad)\
**Post date:** [April 24, 2017, 6:28pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/39 "2017-04-24T18:28:30Z")

</div>

Great to know. Hope you can figure something out for us.

---

<div class="post-metadata">

**Author:** ![stephen](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.stardog.com/stephen/32/27_2.png) [@stephen](https://community.stardog.com/u/stephen)\
**Post date:** [April 27, 2017, 2:10pm UTC](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239/40 "2017-04-27T14:10:37Z")

</div>

Hi Håvard, I’ve got nothing solid to report as of yet, but I just wanted to pop in and assure you that we’re still looking into this issue!

[Previous page](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239.md?page=1)

[Next page](https://community.stardog.com/t/icv-slow-when-graph-without-icv-has-30-million-triples/239.md?page=3)