How can feature engineering be performed across multiple intermediate stages to progressively transform raw data into model-ready features? This includes creating temporary features, applying data transformations, aggregations, enrichments, and validations at different stages of the pipeline before generating the final input dataset for a machine learning model. The objective is to design a scalable and maintainable feature engineering workflow that improves data quality, captures complex patterns, and produces optimized features for model training and inference.
Stardog isn’t intended to be a general-purpose feature engineering platform. While SPARQL can perform transformations and aggregations, chaining many intermediate stages with temporary features is not what RDF knowledge graphs are optimized for.
Where Stardog can play a valuable role is if some of the features depend on semantics rather than purely relational transformations.
The workflow you describe is closer to a traditional feature engineering pipeline. Stardog can contribute by integrating data across sources, deriving semantic features through reasoning, enriching data with graph relationships, and validating data with SHACL. However, it doesn't replace transformation frameworks such as Spark, SQL, or dbt. Stardog can provide semantically enriched, validated data that is then consumed by a feature engineering pipeline to produce the final model-ready dataset.
Happy to revisit my answer if I’ve misunderstood the question.