Multi-Task NLP via Task Vector Composition
Built a parameter-efficient multi-task learning system using task vector composition, reaching 87% accuracy on sentiment analysis and 74% F1 on named-entity recognition. Held the model to 110M parameters—the same footprint as base BERT—while balancing performance across tasks, cutting storage 50% versus keeping separate fine-tuned models. Fine-tuned BERT independently on SST-2 and CoNLL-2003, extracted task-specific weight deltas, and merged them with scaled averaging to flexibly prioritize tasks.
PythonPyTorchTransformersBERTHuggingFace
Investigating Antifragility in Large Language Models
Investigated the robustness of LLMs adapted for image classification with LIFT, measuring performance under FGSM adversarial attacks across epsilon values from 0.01 to 0.8. Found that synaptic filtering degraded LLM accuracy from 83% to 66% by disrupting self-attention, revealing fundamental architectural differences between LLMs and traditional deep neural networks. Implemented two synaptic filtering approaches—percentage-based and magnitude-based—to test the antifragility hypothesis on 60K MNIST examples.
PythonPyTorchTransformersLLaMA 3FGSM