All research

Two-tier detection of hate speech in live chat (master's thesis)

Parsa Kazemi · University of Messina, FCRLAB ·

I built a system that reads live chat from YouTube and Twitch and decides whether each message is normal, offensive or hateful. Chat moves too fast to read, and the two usual tools pull against each other. Fast classifiers keep up but ignore context. Large language models understand context but need seconds per message.

The idea

The pipeline uses two tiers. A small fine-tuned DistilBERT model labels every message in about 31 milliseconds. A large language model then re-checks only the messages the first tier was unsure about (confidence below 0.80). It reads them back from storage, so the slow model never blocks the fast one.

Results

I tested it on 1,748 live messages that I labelled by hand. 80 of them are toxic (21 hate, 59 offensive).

  • The first tier alone reached a toxic-class F1 of 0.13. With the language model re-checking, it reached 0.37, and 0.39 with a chain of four cooperating agents.
  • I tested four ways of giving the slow model more context. Only the multi-agent chain helped in a significant way (McNemar test, p < 0.001). Memory of an author’s earlier messages did not help, because 83.1% of authors write exactly one message in the data.
  • With two cluster machines the system handled 61.1 messages per second, with zero messages lost or duplicated across more than 280,000 replayed. A Raspberry Pi 4 managed 6.2 messages per second.

Limits

  • Only 80 toxic messages, so small differences between variants cannot be told apart.
  • One annotator (me), so there is no agreement score.
  • English text only, from two platforms.
  • The benchmark oversamples hard cases. It is a stress test, not a random sample of chat.

The code is public: see the link below. The full thesis text is available on request.

nlp · thesis