Join Nostr
2026-11-10 05:28:17 UTC

Apache - The ASF on Nostr: How can we know what data was used to train a language model? Apache Sourcelume is ...

How can we know what data was used to train a language model?

Apache Sourcelume is developing machine-readable provenance records for datasets and models. A sample training pipeline demonstrates how a model's training process can generate a provenance record, linking to dataset records and documenting token counts, data shares, and loss.

Read the blog to learn more about what the project is working towards: https://news.apache.org/foundation/entry/tiny-model-full-provenance-where-apache-sourcelume-is-and-where-it-is-heading