Today, Numerai is open sourcing our originality, concordance, and consistency criteria. The code may be found here.
In Numerai’s tournament, data scientists compete to build machine learning algorithms that power our hedge fund. In order for a submission to be eligible for payout, it must be original, concordant, and consistent. They are described on the help page. For example, here’s the definition of originality:
Originality is a measure of whether a set of predictions is uncorrelated with predictions already submitted. Numerai wants to encourage new models over duplicate submissions.