Filter Evaluation Metrics

[memeeerit]

We need to know how well our filters are doing.

• Determine what proportion of data coming from the generic parser typically needs to be rejected.
• Determine the accuracy and false positive/negative rates of the filter pipeline as a whole.
• Determine the accuracy and false positive/negative rates of the local filters as a whole
• Determine the accuracy and false positive/negative rates of the openAI filter. These should be computed in two settings, one with a thusfar unfiltered dataset, and one only using data the local filters passed through.
• Any other metrics you think are useful

Save and track with git any tools developed to accomplish this, as we may want to use them again for strategy-specific evaluations.