High max cardinality causes stack overflow in ICV

Support Bug
#1

Hi,

I’m trying to limit the number cardinality of a predicate on objects. Eg. users can have max 100 saved cases.

Stardog 4.2.3 won’t let me insert any users when I have that restriction, even though I’m not breaking it. The errors from stardog:

[ERROR] com.complexible.tx.api.impl.DefaultTransaction - There was a fatal failure during preparation of 3e9cf482-0222-449c-a10b-df9126941ad1
java.lang.StackOverflowError: null
	at com.complexible.stardog.plan.PlanNodes$FindVisitor.visit(PlanNodes.java:902)
	at com.complexible.stardog.plan.FilterPlanNodeImpl.accept(FilterPlanNodeImpl.java:75)
	at com.complexible.stardog.plan.BasePlanNodeVisitor.unaryVisit(BasePlanNodeVisitor.java:43)
	at com.complexible.stardog.plan.PlanNodes$FindVisitor.visit(PlanNodes.java:911)
	at com.complexible.stardog.plan.FilterPlanNodeImpl.accept(FilterPlanNodeImpl.java:75)
	at com.complexible.stardog.plan.BasePlanNodeVisitor.unaryVisit(BasePlanNodeVisitor.java:43)
	at com.complexible.stardog.plan.PlanNodes$FindVisitor.visit(PlanNodes.java:911)
	at com.complexible.stardog.plan.FilterPlanNodeImpl.accept(FilterPlanNodeImpl.java:75)
	at com.complexible.stardog.plan.BasePlanNodeVisitor.unaryVisit(BasePlanNodeVisitor.java:43)
	at com.complexible.stardog.plan.PlanNodes$FindVisitor.visit(PlanNodes.java:911)
	at com.complexible.stardog.plan.FilterPlanNodeImpl.accept(FilterPlanNodeImpl.java:75)
	....

The data

@prefix arkiv: <http://www.arkivverket.no/standarder/noark5/arkivstruktur/> .

<http://data.einnsyn.no/bruker/0bd4896d-82a5-4758-9e3c-f88a90f2743d> a <http://data.einnsyn.no/brukermeta/Adminbruker> ;
	<http://data.einnsyn.no/brukermeta/brukernavn> "bruker" ;
	<http://data.einnsyn.no/brukermeta/epost> "bruker@example.org" ;
	<http://data.einnsyn.no/brukermeta/telefon> "12345678" ;
	<http://data.einnsyn.no/brukermeta/oppdatertDato> "2017-02-28T13:27:24.977+01:00"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
	<http://data.einnsyn.no/brukermeta/passord> "abc" ;
	<http://data.einnsyn.no/brukermeta/opprettetDato> "2017-02-28T13:27:24.977+01:00"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
	a <http://data.einnsyn.no/brukermeta/Bruker> .

And the ICV file, which I would attach instead of pasting here, but I am only allowed to attach images :frowning:

@prefix : <http://data.einnsyn.no/brukermeta/> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@base <http://data.einnsyn.no/brukermeta/> .

<http://data.einnsyn.no/brukermeta/> rdf:type owl:Ontology .

#################################################################
#    Object Properties
#################################################################

###  http://data.einnsyn.no/brukermeta/forsendelsesmåte
:forsendelsesmåte rdf:type owl:ObjectProperty .


###  http://data.einnsyn.no/brukermeta/lagretSak
:lagretSak rdf:type owl:ObjectProperty .


###  http://data.einnsyn.no/brukermeta/lagretSøk
:lagretSøk rdf:type owl:ObjectProperty .


###  http://data.einnsyn.no/brukermeta/rettetMot
:rettetMot rdf:type owl:ObjectProperty .


###  http://data.einnsyn.no/brukermeta/virksomhet
:virksomhet rdf:type owl:ObjectProperty .


#################################################################
#    Data properties
#################################################################

###  http://data.einnsyn.no/brukermeta/brukernavn
:brukernavn rdf:type owl:DatatypeProperty ;
            rdfs:range xsd:string .


###  http://data.einnsyn.no/brukermeta/epost
:epost rdf:type owl:DatatypeProperty ;
       rdfs:range xsd:string .


###  http://data.einnsyn.no/brukermeta/passord
:passord rdf:type owl:DatatypeProperty ;
         rdfs:range xsd:string .


###  http://data.einnsyn.no/brukermeta/telefon
:telefon rdf:type owl:DatatypeProperty ;
         rdfs:range xsd:string .


#################################################################
#    Classes
#################################################################

###  http://data.einnsyn.no/brukermeta/Adminbruker
:Adminbruker rdf:type owl:Class ;
             rdfs:subClassOf :Bruker ,
                             [ rdf:type owl:Restriction ;
                               owl:onProperty :telefon ;
                               owl:qualifiedCardinality "1"^^xsd:nonNegativeInteger ;
                               owl:onDataRange xsd:string
                             ] .


###  http://data.einnsyn.no/brukermeta/Bruker
:Bruker rdf:type owl:Class ;
        rdfs:subClassOf [ rdf:type owl:Restriction ;
                          owl:onProperty :brukernavn ;
                          owl:qualifiedCardinality "1"^^xsd:nonNegativeInteger ;
                          owl:onDataRange xsd:string
                        ] ,
                        [ rdf:type owl:Restriction ;
                          owl:onProperty :epost ;
                          owl:qualifiedCardinality "1"^^xsd:nonNegativeInteger ;
                          owl:onDataRange xsd:string
                        ] ,
                        [ rdf:type owl:Restriction ;
                          owl:onProperty :passord ;
                          owl:qualifiedCardinality "1"^^xsd:nonNegativeInteger ;
                          owl:onDataRange xsd:string
                        ] ;
        owl:hasKey ( :brukernavn
                   ) .


###  http://data.einnsyn.no/brukermeta/Forsendelsesmåte
:Forsendelsesmåte rdf:type owl:Class .


###  http://data.einnsyn.no/brukermeta/Innsynskrav
:Innsynskrav rdf:type owl:Class ;
             rdfs:subClassOf [ owl:intersectionOf ( [ rdf:type owl:Restriction ;
                                                      owl:onProperty :forsendelsesmåte ;
                                                      owl:allValuesFrom :Forsendelsesmåte
                                                    ]
                                                    [ rdf:type owl:Restriction ;
                                                      owl:onProperty :forsendelsesmåte ;
                                                      owl:qualifiedCardinality "1"^^xsd:nonNegativeInteger ;
                                                      owl:onClass :Forsendelsesmåte
                                                    ]
                                                  ) ;
                               rdf:type owl:Class
                             ] .


###  http://data.einnsyn.no/brukermeta/Innsynskravdel
:Innsynskravdel rdf:type owl:Class ;
                rdfs:subClassOf [ rdf:type owl:Restriction ;
                                  owl:onProperty :rettetMot ;
                                  owl:allValuesFrom <http://www.arkivverket.no/standarder/noark5/arkivstruktur/Registrering>
                                ] .


###  http://data.einnsyn.no/brukermeta/LagretSak
:LagretSak rdf:type owl:Class .


###  http://data.einnsyn.no/brukermeta/LagretSøk
:LagretSøk rdf:type owl:Class .


###  http://data.einnsyn.no/brukermeta/Sluttbruker
:Sluttbruker rdf:type owl:Class ;
             rdfs:subClassOf :Bruker ,
                             [ rdf:type owl:Restriction ;
                               owl:onProperty :lagretSak ;
                               owl:maxCardinality "100"^^xsd:nonNegativeInteger
                             ] .


###  http://data.einnsyn.no/brukermeta/Virsomhetsbruker
:Virsomhetsbruker rdf:type owl:Class ;
                  rdfs:subClassOf :Bruker ,
                                  [ rdf:type owl:Restriction ;
                                    owl:onProperty :virksomhet ;
                                    owl:cardinality "1"^^xsd:nonNegativeInteger
                                  ] ,
                                  [ rdf:type owl:Restriction ;
                                    owl:onProperty :telefon ;
                                    owl:qualifiedCardinality "1"^^xsd:nonNegativeInteger ;
                                    owl:onDataRange xsd:string
                                  ] .


###  http://www.arkivverket.no/standarder/noark5/arkivstruktur/Registrering
<http://www.arkivverket.no/standarder/noark5/arkivstruktur/Registrering> rdf:type owl:Class .


###  Generated by the OWL API (version 4.2.6.20160910-2108) https://github.com/owlcs/owlapi
#2

Hi Håvard,

Large maxCardinality constraints are not supported for precisely this reason. The way it gets rewritten results in a query with an extremely large amount of variables and a bunch of constraints on them, which causes a StackOverflowError.

Instead, you could try expressing this as a SPARQL constraint like so:

SELECT ?x {
  ?x a :SluttBruker ;
     :lagretSak ?y
}
GROUP BY ?x
HAVING count(?y) > 100
#3

Can you expand upon “not supported”? I’m asking because it appears as though it is supported and caused a stack overflow error. If there is a limit and he just didn’t happen to hit it, what would be that limit? I’d be happy to update the docs and put in a pull request with the info.

#4

The limit is essentially the stack size. Given an infinite stack size (with infinite memory, disk, time, etc.) this would theoretically be “supported.” In this case it is not feasible given a normal system, especially given that there are other ways of enforcing the constraint (e.g., the above SPARQL)

#5

Hi Stephen,

Thanks for this solution.

I’m not sure how the ICV engine is designed, but I presume that there is a focus on efficiency so that only modified data gets validated.

How is the performance of the above SPARQL, does it end up having to validate the entire database when inserting data?

Håvard

#6

ICV validations are only run on the newly inserted data but validation may require looking at a much larger set than this. If you require that an entity points to at most N “values” for a given property, the check includes finding all values for that property. Additionally, the equality reasoning makes this a bit more complex. Validation requires counting all unique values for the property.

#7

I don’t know exactly how ICV is implemented either but this is how I understand it to work. The constraint is always going to be evaluated in the context of the data being added and the data currently in the database. Logically I believe ICV can be though of as opening a transaction, adding your data, then execute your constraint query, if there are any bindings roll back the transaction, if there aren’t then commit. I think the only way to apply ICV to only the data being added would be to create an in memory database and checking the consistency before adding the data to your on disk store.

I have a side question that I came across looking into my response. What is the difference between the icv.enabled and the icv.consistency.automatic? I’m a little confused since the docs refer to icv.enabled as “guard mode” but I thought you could enable icv without guard mode and allow additions that would violate ICV checks but just report the violations so I would have thought that icv.consistency.automatic would be “guard mode”

#8

This could probably be more visible, but if you dig down into the javadocs you get your answer:

public static final ConfigProperty<Boolean> ICV_CONSISTENCY_AUTOMATIC

Option to specify that ICV guard mode validation will check the consistency 
of the database as well. By default, ICV validation only checks the violations of 
constraints explicitly added to the database. However, OWL axioms that exist in 
the database may cause inconsistencies similar to constraint violations. Setting 
this option ensures that both reasoning consistencies and constraint violations 
will be checked during guard mode validation.
#9

What is the difference between reasoning.consistency.automatic and icv.consistency.automatic then?

Ok, I think I’ve got it. RC=reasoning consisteny, IC=ICV Consistency. You can check for RC, RC&IC or IC which correspond to the properties, icv.enabled, reasoning.consistency.automatic, and icv.consistency.automatic respectively.

So if I have this right, setting reasoning.consistency.automatic is redundant if you have already enabled icv.consistency.automatic in which case you could possibly eliminate the icv.enabled option and just use a combination of icv.enabled.automatic and reasoning.enabled.automatic?

I also noticed that the property name icv.enabled might be a little confusing for some people. So you can have icv.enabled=false and still execute the command stardog icv validate. I can see someone saying, “Whaa? I though I disabled it.” and that it’s more like icv.guard.enabled and that icv checking isn’t something you enable or disable. You can always check.