Language Model that simulates frustration
FrustrationLM is a tiny, open-source language model designed to specialize in one specific behavior: expressing frustration. Unlike general-purpose language models that aim to answer questions across countless topics, FrustrationLM focuses on a single emotional style.
The project began with experiments in training a tiny transformer from scratch using only frustration-related text. While this demonstrated the overall training pipeline, the dataset wasn't large enough for the model to learn language itself. Instead of continuing down that path, FrustrationLM now starts from DistilGPT-2's pretrained weights and is fine-tuned on a curated frustration-focused dataset.
The dataset contains frustration-related conversations, reactions, expressions, and writing styles. The objective isn't to teach the model facts or general knowledge, but to consistently express one emotional behavior through its responses.
Once training is complete, the model is converted to the GGUF format so it can be run locally with compatible inference software. Every part of the project—including the training code, model weights, and released GGUF files—is open source, allowing anyone to inspect the implementation, reproduce the results, or build on top of the project.
I've been programming for years, but AI engineering was a completely new field for me. I wanted to move beyond using language models and actually understand how they're built. FrustrationLM started as a learning project—a way to explore the architecture of language models by building one myself.
As I learned more about AI, I became interested in a broader question: what makes intelligence possible? Today, some researchers focus primarily on making increasingly capable language models through scaling, while others explore ideas such as embodiment, interaction with the real world, and internal objectives that guide behavior. I find these questions fascinating, and they influenced how I began thinking about this project.
Humans don't learn every skill from scratch. We build on years of experience, language, observation, and feedback from the world around us. A teenager can learn to drive in a relatively short amount of time because they're already equipped with general knowledge, intuition, and an understanding that some outcomes are better than others. That made me wonder whether future AI systems might also benefit from richer forms of learning than text alone.
One idea that particularly interests me is whether emotions could play a role as part of an AI system's internal objectives. In humans, emotions often influence attention, learning, motivation, and decision-making. They can improve behavior in some situations while also introducing biases in others. I don't know whether emotions belong in future AI systems, but I think it's a worthwhile question to explore.
FrustrationLM is my first small step in exploring that idea. Rather than attempting to build a complete cognitive architecture or a robot, I chose a much simpler experiment: a tiny language model specialized in a single emotional behavior. Its goal isn't to prove a theory about AGI or artificial emotions, but to help me better understand how language models learn, how specialized behaviors emerge, and whether emotion-focused models are an interesting direction for future experimentation.
Everything in this project is open source because I believe learning is most valuable when other people can inspect, reproduce, question, and build upon the work.
