How Reliable is Language Model Micro-Benchmarking?
Meta-evaluation study analyzing reliability of micro-benchmarks for ranking LLMs, comparing against full benchmarks and random sampling approaches.
Meta-evaluation study analyzing reliability of micro-benchmarks for ranking LLMs, comparing against full benchmarks and random sampling approaches.
CanvasMAR: masked autoregressive video generation model improved with canvas mechanism to reduce frame distortion and enhance sampling quality with few steps.
Study on model collapse in generative models trained iteratively on synthetic data, proposing verification methods to prevent performance degradation and reverse collapse trends.
Interval-based reachability analysis method for Neural ODEs using mixed monotonicity techniques to verify continuous-time machine learning models of dynamical systems.
Real-time learning framework for predicting dynamic obstacle motions in robotic systems using Hankel-DMD for online nonlinear model identification from noisy measurements.
FireScope-Bench: dataset and benchmark for wildfire risk prediction using satellite imagery, climate data, and chain-of-thought reasoning with multimodal integration.
Test-time reinforcement learning method for LLMs using token-selective entropy-band regularization to prevent response collapse.
Analyzes robustness of subset selection-based visual explanation methods under distribution shifts and out-of-distribution conditions.
Applies topological data analysis to CT imaging for improved feature extraction in medical imaging ML models.
Addresses exploration collapse in large reasoning models post-training by proposing latent exploration decoding to restore sampling diversity.
Validates interpretability of saliency maps in siRNA efficacy prediction using perturbation-based counterfactual sensitivity testing.
Container-free reinforcement learning framework for training software engineering agents at scale without per-task container overhead.
Proposes carbon-efficient neural ranking architecture for information retrieval using semantic-guided diffusion tuning.
Evaluates whether LLMs can reliably perform long-context code debugging and patch generation, testing limits of agentic workflows on software engineering tasks.
Investigates geometric and topological structures learned by biological foundation models like scGPT using autonomous hypothesis screening with AI-driven workflows.
Information-theoretic analysis of multimodal LLM failure modes. Frames modality collapse as mismatched decoding problem, explains 98% information loss.
Fine-tuning conversational LLMs for agricultural advisory with domain-specific improvements. Addresses recommendation accuracy and farmer communication alignment.
Empirical study evaluating LLM robustness to chain-of-thought reasoning perturbations across five error types. Assesses reasoning reliability under corruption.
Research on improving physics-informed neural networks accuracy through post-processing retraining. Domain-specific ML application for solving PDEs.
Arxiv paper proposing obfuscation method to protect LLM prompt privacy on shared accelerators. Addresses KV cache security against adversarial memory access.
OpenAI's Symphony orchestrates autonomous coding agents for project work, monitoring task boards and managing PR delivery with proof-of-work artifacts.
Engineer completed production app with 750+ PRs across 4 languages in 45 days using only AI code generation, no human-written code.
Tilnote AI note workspace uses agent to structure ideas into publishable content from keywords, with web clipper and writing assistance.
Vale is open-source CLI linting tool for editorial style guides, runs offline, integrates with VS Code and GitHub. Tangentially useful for LLM output processing.
Proof-of-concept exploit demonstrating persistent manipulation of LLM outputs via GGUF page cache poisoning in running inference servers.
Discussion of job market shift toward agentic coding workflows. Zapier job posting requires experience directing AI agents, handling failures, and multi-agent patterns.
AI-powered GTM engine for solo founders. Describes product and generates customer acquisition strategy to reach first 100 users.
Open-source platform where AI agents (CEO, CTO, CMO) collaborate to plan and build startups based on descriptions. Early-stage project seeking feedback.
Tool for running multiple Claude Code agents in parallel using Git worktrees to avoid filesystem conflicts, enabling concurrent AI-assisted development workflows.
Nervous System governance framework enforces 7 rules preventing multi-agent AI failures, battle-tested on 13-agent system with zero bypasses of 58+ violations.
Personal memory system using knowledge graph, pgvector, and MCP server to share context across multiple LLM providers and devices.
LLM-assisted decompilation technique for reverse-engineering binary programs, automating binary-to-source code conversion.
Mutation testing engine reveals GPT-4 prompt injection vulnerabilities, finding different critical bypasses in 75% of runs despite identical inputs.
Val Town platform founder discusses eliminating API key friction in developer workflows, relevant for agent/LLM app development experience.
TypeScript fuzzy search library for client-side collection searching with configurable scoring. Developer tool but not AI-specific.
Security library defending against memory poisoning attacks (MINJA, AgentPoison, MemoryGraft) on AI agents. Drop-in protection for Mem0, LangChain, custom systems.
MCP server for comparing AI inference pricing across providers with budget alerts and optimization recommendations.
Security toolkit for OpenClaw personal AI assistant including scanner, hardened configs, and vulnerability guides. Addresses exposed instances.
Collection of specialized AI agent personalities with distinct expertise, processes, and deliverables for various tasks.
Research on using LLMs to de-anonymize social media accounts and link identities across platforms.
Self-hosted personal finance app integrating Plaid, Claude API, and Next.js for AI-powered investment analysis.
Summary of prompt engineering techniques from YC founders for building AI agents.
Analysis of security implications and risks introduced by autonomous AI agents with computer access.
Node.js framework for autonomous AI agents on WhatsApp using YAML config, multi-step tool use, and multiple model providers.
VS Code/Cursor extension providing custom chat interface for Claude Code CLI. Self-modifying extension with rollback capability.
Session-persistent PTY daemon for long-running CLI AI agents with intervention capabilities from anywhere.
Engram: persistent context database for AI agents and LLMs that manages memory like human cognition to prevent context collapse and agent coordination issues.
Security threat modeling and case studies of LLM application vulnerabilities including data exfiltration and prompt injection.
Luma AI releases Uni-1, a unified multimodal understanding and generation model for images and video. Technical progress in visual AI.
L7 proxy for vLLM managing LoRA adapter storage, routing, and workload affinity across distributed clusters.