Stardog Drop and Restore DB does not work when stardog is configured with LDAP access

(Ivan Ermilov) #1

Affected stardog version: 6.1.2

Configuration (stardog.properties):

# Default options
query.all.graphs = true
security.named.graphs = true
# Base 64 compatible passwords are a requirement for DataPlatform
password.length.max = 64
password.regex = [\\w+\\/=]+

# Access Logs
logging.access.enabled = true
logging.access.type = text
logging.access.file = /logs/access.log
logging.access.rotation.type = size
logging.access.rotation.limit = 1000000

# Audit Log

# Note, while it is unlikely that both access and audit logging should be enabled, Stardog will not prevent you from
# doing that. But it's strictly superfluous since audit logging is a strict superset of access logging; i.e., the former
# can be created from the latter offline.

# Slow Query Log
logging.slow_query.enabled = true
logging.slow_query.time = 30s
logging.slow_query.type = text

logging.create.failure.fatal = true

backup.location = /backups
export.dir = /export

# LDAP configuration
security.realms = ldap
ldap.provider.url = ldap://example.com:389
ldap.security.principal = cn=username,cn=Users,dc=example,dc=com
ldap.security.credentials = password
ldap.user.dn.template = cn={0},cn=Users,dc=example,dc=com
ldap.group.lookup.string = CN=Groups,dc=example,dc=com
ldap.users.cn = users
ldap.superusers.cn = superusers
ldap.cache.invalidate.time = 1h

There is also sdpass file with the following contents:
::*:username:password

steps to reproduce (initialization script creates stardog database on startup):

$ stardog-admin db list
+-----------+
| Databases |
+-----------+
| stardog   |
+-----------+
$ stardog-admin db create -n stardog2 -v -o versioning.enabled=false preserve.bnode.ids=false --
Successfully created database 'stardog2'.
$ stardog-admin db backup stardog2
Backup of stardog2 saved to /backups/stardog2/2019-05-15 in 00:00:00.029
$ ls /backups/stardog2/2019-05-15
dict.bk  index.bk  metadata.bk  options.bk  version
### Restore with -o flag fails
$ stardog-admin db restore -o /backups/stardog2/2019-05-15
user:internal.devops:read:db:stardog2 exists
### The db is not there anymore
$ stardog-admin db list
+-----------+
| Databases |
+-----------+
| stardog   |
+-----------+
$ stardog-admin db drop stardog2
Database 'stardog2' does not exist.

Stardog Log:

INFO  2019-05-15 12:02:38,865 [stardog-user-1] com.complexible.stardog.docs.db.BitesConnectableFactory:createDocumentStore(133): Creating document store for stardog at file:////data/stardog/docs
INFO  2019-05-15 12:02:38,917 [stardog-user-1] com.complexible.stardog.StardogKernel:write(77): Successfully created database 'stardog'.
INFO  2019-05-15 12:08:07,722 [stardog-user-24] com.complexible.stardog.docs.db.BitesConnectableFactory:createDocumentStore(133): Creating document store for stardog2 at file:////data/stardog2/docs
INFO  2019-05-15 12:08:07,724 [stardog-user-24] com.complexible.stardog.StardogKernel:write(77): Successfully created database 'stardog2'.
INFO  2019-05-15 12:09:00,700 [stardog-user-16] com.complexible.stardog.docs.db.BitesConnectableFactory:createDocumentStore(133): Creating document store for stardog2 at file:////data/stardog2/docs
ERROR 2019-05-15 12:09:00,718 [stardog-user-16] com.complexible.stardog.protocols.http.server.StardogHttpServiceLoader:accept(235): An unexpected exception was handled by the server
com.complexible.stardog.security.ResourceExistsException: user:internal.devops:read:db:stardog2 exists
        at com.complexible.stardog.security.index.IndexAuthorizationManager.add(IndexAuthorizationManager.java:251) ~[stardog-core-security-6.1.2.jar:?]
        at com.complexible.stardog.security.impl.SystemSecurityManagerImpl.add(SystemSecurityManagerImpl.java:285) ~[stardog-core-security-6.1.2.jar:?]
        at com.complexible.stardog.security.index.IndexBackedSystemSecurityManager.add(IndexBackedSystemSecurityManager.java:104) ~[stardog-core-security-6.1.2.jar:?]
        at com.complexible.stardog.security.StardogSecurityManagerImpl.add(StardogSecurityManagerImpl.java:622) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.security.StardogSecurityManagerImpl.addUserPerm(StardogSecurityManagerImpl.java:110) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.StardogKernel$NotifyingSecurityManager.addUserPerm(StardogKernel.java:3889) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.StardogKernel.lambda$dbPermissions$10(StardogKernel.java:2153) ~[stardog-6.1.2.jar:?]
        at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) ~[shiro-core-1.2.3.jar:1.2.3]
        at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) ~[shiro-core-1.2.3.jar:1.2.3]
        at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) ~[shiro-core-1.2.3.jar:1.2.3]
        at com.complexible.stardog.StardogKernel.dbPermissions(StardogKernel.java:2119) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.StardogKernel.internalRestore(StardogKernel.java:1971) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.StardogKernel.restore(StardogKernel.java:1872) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.StardogKernel.restore(StardogKernel.java:1792) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.protocols.http.server.AdminDatabaseService.restore(AdminDatabaseService.java:138) ~[stardog-protocols-http-server-6.1.2.jar:?]
        at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-6.1.2.jar:?]
        at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.2.3.jar:1.2.3]
        at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.2.3.jar:1.2.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
ERROR 2019-05-15 12:10:09,915 [stardog-user-11] com.complexible.stardog.protocols.http.server.StardogHttpServiceLoader:accept(235): An unexpected exception was handled by the server
com.complexible.stardog.server.UnknownDatabaseException: Database 'stardog2' does not exist.
        at com.complexible.stardog.server.UnknownDatabaseException.create(UnknownDatabaseException.java:28) ~[stardog-core-shared-6.1.2.jar:?]
        at com.complexible.stardog.StardogKernel.assertDbExists(StardogKernel.java:2696) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.StardogKernel.internalDrop(StardogKernel.java:2219) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.StardogKernel.drop(StardogKernel.java:2207) ~[stardog-6.1.2.jar:?]
        at com.complexible.stardog.protocols.http.server.AdminDatabaseService.dropDatabase(AdminDatabaseService.java:259) ~[stardog-protocols-http-server-6.1.2.jar:?]
        at com.stardog.http.server.undertow.jaxrs.ExtractRoutes.lambda$handleIt$5(ExtractRoutes.java:192) ~[stardog-protocols-http-server-6.1.2.jar:?]
        at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120) [shiro-core-1.2.3.jar:1.2.3]
        at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108) [shiro-core-1.2.3.jar:1.2.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Removing LDAP portion of the configuration makes Stardog behave as intended. Here is the working configuration:

# Default options
query.all.graphs = true
security.named.graphs = true
# Base 64 compatible passwords are a requirement for DataPlatform
password.length.max = 64
password.regex = [\\w+\\/=]+

# Access Logs
logging.access.enabled = true
logging.access.type = text
logging.access.file = /logs/access.log
logging.access.rotation.type = size
logging.access.rotation.limit = 1000000

# Audit Log

# Note, while it is unlikely that both access and audit logging should be enabled, Stardog will not prevent you from
# doing that. But it's strictly superfluous since audit logging is a strict superset of access logging; i.e., the former
# can be created from the latter offline.

# Slow Query Log
logging.slow_query.enabled = true
logging.slow_query.time = 30s
logging.slow_query.type = text

logging.create.failure.fatal = true

backup.location = /backups
export.dir = /export

# LDAP configuration
1 Like