ZEIT Discourse Analysis

A prototype system for analyzing discussion quality and discourse dynamics in ZEIT Online comment sections. This project combines sentiment analysis, deliberation quality assessment, and thematic clustering to provide insights into reader discussions.

Overview

This repository analyzes ZEIT reader comments using a deliberation quality framework based on the Discourse Quality Index (DQI). The analysis evaluates comments across four dimensions:

• Justification: How well are claims supported with reasoning?
• Respect: How respectfully do commenters treat each other?
• Reciprocity: How well do commenters engage with each other's arguments?
• Contribution: How constructively do comments advance the discussion?

Additionally, the system performs sentiment analysis, identifies polarization patterns, extracts main arguments, and detects helpful hints or corrections within discussions.

Project Structure

The repository consists of three main components:

1. Analysis (/analysis)

Core analysis pipeline that processes comment data:

• Sentiment analysis using machine learning models
• Deliberation quality scoring via OpenAI API
• Article-level aggregation (summaries, main arguments, helpful hints)
• Metric calculation (polarization index, vividness scores)
• Data storage in DuckDB

Setup:

cd analysis
python -m venv venv
source venv/bin/activate 
pip install .

Requirements: Python with dependencies including duckdb, torch, transformers, openai

2. Summary (/summary)

Quarto-based statistical report with visualizations:

• Aggregated metrics across articles and ressorts
• Distribution analyses
• Correlation studies
• Exportable HTML report

Setup:

cd summary
python -m venv venv
source venv/bin/activate
pip install .

Requirements: Python with jupyter, seaborn, pandas, numpy, scipy

3. Dashboard (/dashboard)

Interactive PHP web interface for exploring results:

• Browse individual articles and their comment analyses
• Filter by ressort
• View deliberation scores, sentiment distributions, and summaries
• Accessible at: https://zeit-discourse.netznarbe.de (password-protected, access via Hendrik Lehmann)

Setup:

cd dashboard
composer install

Requirements: PHP, Composer

Data Flow

1. Raw comment data (CSV) → Analysis pipeline
2. Analysis results → DuckDB database
3. Exported data → Dashboard (JSON) and Summary (JSON/CSV)
4. Interactive exploration via Dashboard or static report via Summary

Client

ZEIT ONLINE
Contact: Hendrik Lehmann
Project Duration: October 2025 (prototype phase)

Documentation

• Detailed codebook: /analysis/docs/codebook.md
• Deliverables specification: See project documentation