Paper Discussion Round

Example: "Attention Is All You Need" (Vaswani et al., 2017)

Contribution: Replaces recurrence and convolution entirely with self-attention for sequence transduction. Key insight: attention alone can capture long-range dependencies more efficiently than RNNs.

Critical analysis: Strengths include O(1) sequential operations for self-attention (vs O(n) for RNNs), parallelizable training, and strong empirical results on WMT translation. Weaknesses include O(n²) memory complexity for sequence length n, no explicit positional inductive bias (sinusoidal encodings are a patch, not a principled solution), and the paper's theoretical justification for why attention works is thin — the success is largely empirical.

Extensions you should mention: Sparse attention (Longformer, BigBird), relative positional encodings (RoPE, ALiBi), flash attention for memory efficiency, and the open question of whether attention is truly sufficient or whether hybrid architectures (state space models) may be more principled.

Example: "RLHF / InstructGPT" (Ouyang et al., 2022)

Contribution: Fine-tunes GPT-3 with human feedback via reinforcement learning from human feedback (RLHF) to align language model outputs with human preferences.

Critical analysis: Strengths include dramatic improvement in helpfulness and safety without sacrificing capability, demonstrates that alignment is tractable, and the reward model approach is elegant. Weaknesses include reliance on human annotator quality and consistency, reward hacking (model learns to exploit the reward model rather than genuinely improve), RLHF instability during training (PPO is notoriously finicky), and the philosophical question of whose preferences are being optimized.

Extensions: DPO (Direct Preference Optimization) as a simpler alternative to PPO, constitutional AI, debate as alignment, and RLAIF (using AI feedback instead of human feedback).

Example: "Neural Tangent Kernel" (Jacot et al., 2018)

Contribution: Shows that infinitely wide neural networks trained with gradient descent behave as kernel methods with a specific kernel (the neural tangent kernel). Provides a theoretical framework for understanding neural network training dynamics.

Critical analysis: Strengths include providing the first rigorous connection between neural networks and kernel methods, enabling closed-form predictions about training dynamics, and opening a new theoretical research direction. Weaknesses include the infinite-width assumption not holding for practical networks, NTK regime predicts lazy training (small parameter changes) which does not capture feature learning, and the gap between NTK predictions and actual network behavior grows with depth and practical hyperparameters.

Extensions: Mean-field theory as an alternative framework, feature learning regime analysis, tensor programs, and the ongoing debate about whether kernel perspectives or feature-learning perspectives better explain deep learning success.

Example: "Scaling Laws for Neural Language Models" (Kaplan et al., 2020)

Contribution: Establishes power-law relationships between model performance and compute, dataset size, and parameter count. Provides predictive equations for optimal resource allocation.

Critical analysis: Strengths include actionable predictions for resource allocation, empirically validated across multiple orders of magnitude, and changed how the field thinks about scaling. Weaknesses include power laws not predicting emergent capabilities (phase transitions), the Chinchilla paper later showed the compute-optimal ratio was wrong (Kaplan undertrained relative to model size), and the laws may break at frontier scales.

Extensions: Chinchilla scaling laws (Hoffmann et al., 2022), emergent abilities debate, scaling laws for downstream tasks, and whether scaling laws hold for multimodal models.

Example: "Constitutional AI" (Bai et al., 2022)

Contribution: Trains a harmless AI assistant using AI-generated self-critiques guided by a set of constitutional principles, reducing reliance on human feedback for safety.

Critical analysis: Strengths include scalability (less human labor), transparency (principles are explicit and auditable), and strong empirical results on harmlessness without sacrificing helpfulness. Weaknesses include the constitution itself may encode biases of its authors, self-critique quality depends on the base model's capability, circular dependency (using AI to align AI), and principles can conflict (helpfulness vs harmlessness) without clear resolution mechanisms.

Extensions: Debate as alignment, recursive reward modeling, mechanistic interpretability for validating alignment, and the broader question of whether behavioral alignment is sufficient or whether we need guaranteed alignment.