Maybe I was not doing it right but it seems we can't use the prefix in templates. I'm currently doing
BIND(template("http://linkedlifedata.com/resource/umls/id/{CUI}") AS ?umls_uri)
where I would rather do
BIND(template("umls:{CUI}") AS ?umls_uri)
and define the prefix for UMLS in the database list of prefixes
Is that possible already with maybe a different syntax? If not I think that would be a great feature 
On a related note, I also noticed that doing
BIND(template("CUI_URI") AS ?umls_uri)
to just cast a full URI present in the data source as is into a IRI for the mappings doesn't work at intended :-/ (even if the setting to percent encode the IRI is off) Stardog automatically adds an extra http:// prefix to the existing one. To go around that I had to remove http using the SQL query to let Stardog add it back... (but I then only kept the CUI in the source data to get to the above code )