# Robots Policy Regression Auditor Local source edition 1.0.0. Test a proposed robots.txt against your current policy. Download the changed crawl decisions and check your required outcomes by crawler and path. ## Run locally Extract the complete ZIP. Use Python 3.12 from a terminal in the extracted directory. Create a virtual environment and activate it, then run: python -m pip install -r requirements.txt python run.py example.json results Use a new output folder for each run. Start with example.json and replace its synthetic values with your own data. Preserve field names, types and units. Files and reports are processed on your computer. ## Inputs Paste baseline and candidate robots.txt text (up to 64 KiB and 256 directives each, empty text is allowed). Supply 1–4 crawler product tokens beginning with a letter and 1–500 unique paths for a single site. Paths start with /, include the optional query, use percent-encoded ASCII and are at most 512 characters. Each path may include an expect object mapping listed crawler tokens to allow or block. Use public-safe paths and policy files. No website or login is required. ## Results DECISIONS.csv covers every crawler and path. REVIEW.csv contains changes and outstanding expectation failures. New and existing failures are separated from fixes. SUMMARY.csv totals the results for each crawler. Matching uses Protego 0.7.0 and the supplied file bodies. HTTP fetch status, crawl delays, indexing and bot enforcement are outside this check. The results folder includes summary.json and the output files produced by this input. The sample produces: DECISIONS.csv, REVIEW.csv, SUMMARY.csv. ## Español · Auditor de cambios en políticas robots.txt Compara dos archivos robots.txt y verifica qué rutas pasan a estar permitidas o bloqueadas para cada rastreador, con informes CSV de los cambios y resultados esperados. Extrae el ZIP completo y abre una terminal en esa carpeta con Python 3.12. Crea y activa un entorno virtual e instala las dependencias con el comando anterior. Ejecuta: python run.py example.json resultados Copia example.json para preparar tus datos. Conserva los nombres y tipos de los campos, usa una carpeta de salida nueva y consulta summary.json junto con los archivos generados. La descripción detallada de los campos está en inglés. The included LICENSE covers use and modification for your own personal and commercial projects. This edition includes its source and can run locally after setup.