Running Stardog in local Kubernetes cluster

Hi Stardog Community,

I wanted to try out Stardog in Kubernetes by running it in a local cluster.

I signed up to get a licence key, created a k3d cluster, and installed it using the helm charts like this:

#!/bin/bash -eu

k3d cluster create --servers 3 --image rancher/k3s:v1.21.8-rc1-k3s1

kubectl create namespace stardog
kubectl -n stardog create secret generic stardog-license --from-file stardog-license-key.bin=./stardog-license-key.bin
helm repo add stardog Stardog Helm Chart | helm-charts
helm install stardog --namespace stardog stardog/stardog
--set persistence.storageClass=local-path
--set persistence.size=200Mi
--set zookeeper.persistence.storageClass=local-path
--set zookeeper.persistence.size=200Mi

The install takes a few minutes and then all stardog & zookeeper pods are running, and I can see the stardog loadbalancer service running:
$ kubectl -n stardog get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
stardog-stardog LoadBalancer 10.43.114.130 172.19.0.2,172.19.0.3,172.19.0.4 5820:31499/TCP 14m
stardog-zookeeper ClusterIP 10.43.90.151 2181/TCP,2888/TCP,3888/TCP 14m
stardog-zookeeper-headless ClusterIP None 2181/TCP,2888/TCP,3888/TCP 14m

If I go to that LoadBalancer, 172.19.0.2:5820 it asks me to log in and I can login as admin. But after that it redirects me to https://apps.stardog.com/ which looks like a hosted version. Is there a step I'm missing, or has anyone else tried it out on k3d/kind/minikube?

Thanks!
John

Hi John,

That sounds like everything is working as expected in your local k8s cluster. You can use Stardog directly via that IP:port with the cli, or if you login through the browser apps.stardog.com provides you with our suite of FE tools you can also use to interact with your local Stardog. For example, you can use studio through the browser and have it connect to Stardog.

Let us know if you have any questions about that.

Paul

Thanks Paul.

I was thinking that if I'm connect to apps.stardog.com that it wouldn't be able to connect to a Stardog instance running in my local kubernetes cluster as the loadbalancer service is a local 172 ip address. So I ran:

docker run --name=stardog-studio -p 8888:8080 -d stardog/stardog-studio:3.2.0

...and I can go to localhost:8888 in my browser, point it to 172.19.0.2:5820 and login there, so that's good.

From there, I tried to create a test database in the UI but it pops up a 405 Not Allowed. I did an apt install stardog to get the stardog & stardog-admin CLIs and tried to create a db using the stardog-admin cli but that gives the the following error:

~/stardog $ cat stardog.sdpass
::*:admin:admin
~/stardog $ STARDOG_HOME=/home/stardog /opt/stardog/bin/stardog-admin db create -n test --server 172.19.0.2:5820
Unable to connect to server at http://localhost:5820 Connection refused (Connection refused).

Should I be able to create a db through the UI logged in as admin? I'm wondering if that's an issue with my kubernetes install or something else.

Thanks,
John.

A few things.

  1. You can use the studio delivered to your browser to connect to a local install of Stardog. Simply give it the same ip:port that the service exposes. You don't need to run studio in docker locally to connect to a local install of Stardog.

  2. Try running just the stardog-admin command to create the db: /opt/stardog/bin/stardog-admin --server http://172.19.0.2:5820 db create -n test

Paul

Also, it wouldn't hurt to check the stardog logs on the pods to make sure everything started fine and without errors. Assuming the pods went to running in k8s, that should be the case, but it's easy to check just in case something else went wrong. Both the stdout of the pods may have info as well as the full log in /var/opt/stardog/stardog.log inside the stardog pods.

Thanks for the stardog-admin example Paul - I realized I was passing in the --server in the wrong place, instead of an arg to stardog-admin I had put it at the end after db create -n john and it wasn't picking it up. I noticed that some of the other commands do complain about that e.g. stardog-admin db list --server http://etc gives an error about an unexpected --server param, but not on the db create.

I created a few databases from the command line, and tried listing them - but I seem to be getting changing results like below:

~/stardog$ /opt/stardog/bin/stardog-admin --server http://172.19.0.2:5820 db list
+-----------+
| Databases |
+-----------+
| test2 |
+-----------+
~/stardog$ /opt/stardog/bin/stardog-admin --server http://172.19.0.2:5820 db list
+-----------+
| Databases |
+-----------+
| test2 |
| test4 |
| test5 |
+-----------+
~/stardog$ /opt/stardog/bin/stardog-admin --server http://172.19.0.2:5820 db list
+-----------+
| Databases |
+-----------+
| test |
| test2 |
| test3 |
| test4 |
+-----------+
~/stardog$ /opt/stardog/bin/stardog-admin --server http://172.19.0.2:5820 db list
+-----------+
| Databases |
+-----------+
| test2 |
| test4 |
| test5 |
+-----------+
~/stardog$ /opt/stardog/bin/stardog-admin --server http://172.19.0.2:5820 db list
+-----------+
| Databases |
+-----------+
| test2 |
| test4 |
| test5 |
+-----------+
~/work/stardog$ /opt/stardog/bin/stardog-admin --server http://172.19.0.2:5820 db list
+-----------+
| Databases |
+-----------+
| test2 |
+-----------+

Are you using a Stardog free license? I suspect what happened is you have 3 single node Stardogs that are running instead of a single stardog cluster with 3 nodes. Then when you make requests to the k8s service is selects one of the standalone Stardogs to handle the request. We can verify this by checking the output of Stardog when it starts in /var/opt/stardog/stardog.log inside the containers.

For Stardog free you'll need to set cluster.enabled=false and replicaCount=1 when you deploy Stardog with helm or you'll need to get a license that supports the cluster.

Yes, I believe it's a free license - I just signed up yesterday. The only settings of the helm chart I overwrote were the persistence storageClass & size ones (at the top of this thread). I don't see any mention of cluster in /var/opt/stardog/stardog.log so it could be three separate instances. I'll do a reinstall and set the cluster.enabled=false and replicaCount=1.

Sounds good. Let us know if you have any trouble.

If the Stardogs were forming a cluster you would see messages about the nodes joining (and who else is a member) at startup.

Thanks Paul - that did the trick! I get consistent results from the stardog-admin db list command now - I can also select one of those databases I've created from the dropdown when I run stardog-studio via docker.

That's great to hear!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.