v1.20.0-pre1: Thancred Waters
Anubis gains its biggest feature in this version: the weight system.
The big ticket items are as follows:
- Implement a no-JS challenge method:
[`metarefresh`](./admin/configuration/challenges/metarefresh.mdx)
([#95](https://github.com/TecharoHQ/anubis/issues/95))
- Implement request "weight", allowing administrators to customize the
behaviour of Anubis based on specific criteria
- Implement GeoIP and ASN based checks via
[Thoth](https://anubis.techaro.lol/docs/admin/thoth)
([#206](https://github.com/TecharoHQ/anubis/issues/206))
- Add
[custom weight thresholds](./admin/configuration/thresholds.mdx) via CEL
([#688](https://github.com/TecharoHQ/anubis/pull/688))
- Move Open Graph configuration
[to the policy file](./admin/configuration/open-graph.mdx)
- Enable support for Open Graph metadata to be returned by default
instead of doing lookups against the target
- Add `robots2policy` CLI utility to convert robots.txt files to Anubis
challenge policies using CEL expressions
([#409](https://github.com/TecharoHQ/anubis/issues/409))
- Refactor challenge presentation logic to use a challenge registry
- Allow challenge implementations to register HTTP routes
A lot of performance improvements have been made:
- Replace internal SHA256 hashing with xxhash for 4-6x performance improvement
in policy evaluation and cache operations
- Optimized the OGTags subsystem with reduced allocations and runtime per
request by up to 66%
- Replace cidranger with bart for IP range checking, improving IP matching
performance by 3-20x with zero heap allocations
And some cleanups/refactors were added:
- Remove the unused `/test-error` endpoint and update the testing endpoint
`/make-challenge` to only be enabled in development
- Add `--xff-strip-private` flag/envvar to toggle skipping X-Forwarded-For
private addresses or not
- Bump AI-robots.txt to version 1.37
- Make progress bar styling more compatible (UXP, etc)
- Add `--strip-base-prefix` flag/envvar to strip the base prefix from request
paths when forwarding to target servers
Request weight is one of the biggest ticket features in Anubis. This enables
Anubis to be much closer to a Web Application Firewall and when combined with
custom thresholds allows administrators to have Anubis take advanced reactions.
For more information about request weight, see
[the request weight section](./admin/policies.mdx#request-weight)
of the policy file documentation.
TL;DR when you have one or more WEIGHT rules like this:
```yaml
bots:
- name: gitea-session-token
action: WEIGH
expression:
all:
- '"Cookie" in headers'
- headers["Cookie"].contains("i_love_gitea=")
# Remove 5 weight points
weight:
adjust: -5
```
You can configure custom thresholds like this:
```yaml
thresholds:
- name: minimal-suspicion # This client is likely fine, its soul is lighter than a feather
expression: weight < 0 # a feather weighs zero units
action: ALLOW # Allow the traffic through
# For clients that had some weight reduced through custom rules, give them a
# lightweight challenge.
- name: mild-suspicion
expression:
all:
- weight >= 0
- weight < 10
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
algorithm: metarefresh
difficulty: 1
report_as: 1
# For clients that are browser-like but have either gained points from custom
# rules or report as a standard browser.
- name: moderate-suspicion
expression:
all:
- weight >= 10
- weight < 20
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 2 # two leading zeros, very fast for most clients
report_as: 2
# For clients that are browser like and have gained many points from custom
# rules
- name: extreme-suspicion
expression: weight >= 20
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 4
report_as: 4
```
These thresholds apply when no other `ALLOW`, `DENY`, or `CHALLENGE` rule
matches the request. `WEIGHT` rules add and remove request weight as needed:
```yaml
bots:
- name: gitea-session-token
action: WEIGH
expression:
all:
- '"Cookie" in headers'
- headers["Cookie"].contains("i_love_gitea=")
# Remove 5 weight points
weight:
adjust: -5
- name: bot-like-user-agent
action: WEIGH
expression: '"Bot" in userAgent'
# Add 5 weight points
weight:
adjust: 5
```
Of note: the default "generic browser" rule assigns 10 weight points:
```yaml
- name: generic-browser
user_agent_regex: >-
Mozilla|Opera
action: WEIGH
weight:
adjust: 10
```
Adjust this as you see fit.