Can stardog User defined function return a dictionary (map)

Can stardog User-defined function return a dictionary (map) as an array literal to be used later as input parameter to another user-defined function?

Not really. It might be possible to figure out a way to use arrays to represent maps, e.g. if a pair of consecutive elements represents a key-value pair, but there's no direct support.

We'd be curious to learn more about the use case you have in mind though.