Most recent entries from all

BREW-scrapy-CVE-2026-55520 — Protego has exponential backtracking ReDoS in robots.txt URL wildcard matching

Affected: Homebrew: scrapy

### Problem description

Protego constructs regular expressions to match URLs against `robots.txt` `Allow:` and `Disallow:` directives, see `protego._urlpattern._URLPattern._prepare_pattern_for_regex()`. Every `*` in the directive value is translated into a lazy `.*?` regex piece, thus a specially crafted directive value with many asterisks may produce a regex that freezes the parser due to exponential backtracking.

### Impact

Parsing a specially crafted `robots.txt` with `protego.Protego.parse()` and then trying to match an URL with `protego.Protego.can_fetch()` results in the latter call not returning for a period dependent on the length of the URL.

### Proof of concept

```python from protego import Protego

robotstxt = f""" User-agent: * Disallow: /{"*1" * 12}*Z """ rp = Protego.parse(robotstxt) url = "/" + "1" * 60 rp.can_fetch(url, "mybot") # freezes ```

EUVD-2026-362093

fkie_cve-2026-55520

Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego._urlpattern._URLPattern._prepare_pattern_for_regex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks creates exponential backtracking. After protego.Protego.parse processes a crafted robots.txt file, protego.Protego.can_fetch can spend an attacker-controlled period matching a near-miss URL and deny service to the crawler. The vulnerable path is src/protego/_urlpattern.py in the _URLPattern match logic. This issue is fixed in version 0.6.2.

GHSA-wjmf-p669-5m5p — Protego has exponential backtracking ReDoS in robots.txt URL wildcard matching

Affected: PyPI: Protego

### Problem description

### Impact

### Proof of concept

```python from protego import Protego

robotstxt = f""" User-agent: * Disallow: /{"*1" * 12}*Z """ rp = Protego.parse(robotstxt) url = "/" + "1" * 60 rp.can_fetch(url, "mybot") # freezes ```

openSUSE-SU-2026:11202-1 — python313-Protego-0.6.2-1.1 on GA media

python313-Protego-0.6.2-1.1 on GA media

PYSEC-2026-3906 — Protego has exponential backtracking ReDoS in robots.txt URL wildcard matching

Affected: PyPI: protego

### Problem description

### Impact

### Proof of concept

```python from protego import Protego

robotstxt = f""" User-agent: * Disallow: /{"*1" * 12}*Z """ rp = Protego.parse(robotstxt) url = "/" + "1" * 60 rp.can_fetch(url, "mybot") # freezes ```

UBUNTU-CVE-2026-55520

Affected: Ubuntu:20.04:LTS: python-protego, Ubuntu:22.04:LTS: python-protego, Ubuntu:24.04:LTS: python-protego, Ubuntu:25.10: python-protego, Ubuntu:26.04:LTS: python-protego