SparkClassNotFoundException: Unable to Find Stardog Spark Connector in PySpark

Support
23

Please do not decode the token back.

base64_token = base64.b64encode(jwt_token.encode("utf-8"))

24

Yes I used without decoding as well it is giving IllegalArgumentException.

base64_token = base64.b64encode(jwt_token.encode("utf-8"))

This statement will generate byte representation of the token and when I use that token it is giving

java.lang.IllegalArgumentException: Illegal base64 character 27

25

Thanks for your patience. I followed the below steps and did not encounter the issue. Can you please check if any step is missing.

Here is what I did.

Step 1:

My stardog is running at https://sd-ffcba75745.stardog.cloud:5820. I created the user called "dummy" on this Stardog Free server with the password, say "dummyuser".

username = dummy
password = dummyuser
Stardog endpoint = https://sd-ffcba75745.stardog.cloud:5820

I ran the below command:

curl -u dummy:dummyuser https://sd-ffcba75745.stardog.cloud:5820/admin/token

And I got this token.

{"token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiSSSSSSSZCI6InN0YXJkb2ctY2xvdWQiLCJpc3MiOiJodHRwOi8vc3RhcmRvZy5jb20vdjEiLCJzdGFyZG9nLXVzZXJuYW1lIjoiZHVtbXkiLCJl"}

Step 2:

I set jwt_token equal to token copied from previous step.

jwt_token = "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiSSSSSSSZCI6InN0YXJkb2ctY2xvdWQiLCJpc3MiOiJodHRwOi8vc3RhcmRvZy5jb20vdjEiLCJzdGFyZG9nLXVzZXJuYW1lIjoiZHVtbXkiLCJl"

  • Encode the jwt_token
    base64_token = base64.b64encode(jwt_token.encode("utf-8"))

Step 3:

Pass base64_token as password to the code that you have and also set stardog.token.authentication option to True.

df = spark.read
.format("com.complexible.stardog.spark.datasource")
.option("stardog.endpoint", "endpoint")
.option("stardog.database", "spark")
.option("stardog.token.authentication", True)
.option("stardog.username", "userName")
.option("stardog.password", base64_token)
.option("config", config_file)
.load()