maximuspowers/muat-pca-10-medium-classifier

Pattern Classifier

This model was trained to classify which patterns a subject model was trained on, based on neuron activation signatures.

Dataset

• Training Dataset: maximuspowers/muat-pca-10-medium
• Input Mode: signature
• Number of Patterns: 14

Patterns

The model predicts which of the following 14 patterns the subject model was trained to classify as positive:

1. palindrome
2. sorted_ascending
3. sorted_descending
4. alternating
5. contains_abc
6. starts_with
7. ends_with
8. no_repeats
9. has_majority
10. increasing_pairs
11. decreasing_pairs
12. vowel_consonant
13. first_last_match
14. mountain_pattern

Model Architecture

• Signature Encoder: [512, 256, 256, 128]
• Activation: relu
• Dropout: 0.2
• Batch Normalization: True

Training Configuration

• Optimizer: adam
• Learning Rate: 0.001
• Batch Size: 16
• Loss Function: BCE with Logits (with pos_weight for training, unweighted for validation)

Test Set Performance

• F1 Macro: 0.0941
• F1 Micro: 0.1068
• Hamming Accuracy: 0.8137
• Exact Match Accuracy: 0.0300
• BCE Loss: 0.5680

Per-Pattern Performance (Test Set)

Pattern	Precision	Recall	F1 Score
palindrome	9.5%	21.3%	13.2%
sorted_ascending	8.3%	20.3%	11.8%
sorted_descending	9.1%	13.7%	10.9%
alternating	9.8%	26.4%	14.3%
contains_abc	10.6%	16.3%	12.8%
starts_with	2.8%	5.8%	3.7%
ends_with	9.7%	18.9%	12.8%
no_repeats	3.5%	5.1%	4.2%
has_majority	3.7%	2.1%	2.7%
increasing_pairs	9.6%	25.8%	14.0%
decreasing_pairs	7.3%	18.3%	10.5%
vowel_consonant	4.5%	4.8%	4.7%
first_last_match	7.3%	11.0%	8.7%
mountain_pattern	6.2%	9.5%	7.5%