Fix: 403 and 429 from the server, host firewall or ModSecurity (nginx, Apache, cPanel, Plesk, hPanel)

If the 403 or 429 doesn’t come from Cloudflare or a WordPress plugin, it comes from the server itself: an nginx or Apache rule, ModSecurity, a host firewall (CSF, Imunify360, fail2ban) or a rate limit. This guide gives copy-paste config for each.

What to allowlist

IPv4 147.93.19.85 and IPv6 2a02:4780:12:d33e::1

User-Agent token SEOAuditSoftware (Site crawl, GEO and Local SEO audits; classic audits use a normal Chrome User-Agent, so allowlist by IP)

Full details: What to allowlist.

Always back up a config file before you edit it, and test the config before reloading (nginx -t, apachectl configtest).

403 from an nginx rule

How we detect it

The home page returns 403 with server: nginx and the plain nginx “403 Forbidden” page. The crawl finds 1 page; classic audits fail with Audit failed: the site blocks automated page downloads (403).

Why it matters

Bot-blocking lists in nginx often match broad words and block useful tools along with bad bots.

Common causes

If there is an if ($http_user_agent ~* ...) rule with return 403 → our User-Agent matched a word in the list.
If there is a map $http_user_agent $bad_bot block (for example the “nginx ultimate bad bot blocker”) → we’re on the list or matched a pattern.
If there are deny lines → our IP or its range is denied.

Fix: basic

Connect to the server with SSH, or open the nginx config in the hosting panel.
Search the config for user agent and deny rules (command in Fix: expert).
Add our exception before the block (snippets below). Reference: nginx access module.
Test with nginx -t and reload with systemctl reload nginx.
Click Crawl again in Site crawl.

Fix: expert

Find the rules:

grep -rnE "http_user_agent|deny |bad_bot|return 403|limit_req" /etc/nginx/

If the site uses a map for bad bots, add an allow line at the top of the map (the first matching regex wins):

map $http_user_agent $bad_bot {
    default                 0;
    ~*SEOAuditSoftware      0;   # SEO Audit Software crawler: allow
    ~*(semrush|mj12bot)     1;   # your existing block list continues here
}

Allow our IPs ahead of any deny rules, and skip the User-Agent check for them:

geo $seoaudit_ip {
    default                 0;
    147.93.19.85            1;
    2a02:4780:12:d33e::1    1;
}

server {
    # ... your existing server settings ...

    set $block_ua 0;
    if ($bad_bot) { set $block_ua 1; }
    if ($seoaudit_ip) { set $block_ua 0; }
    if ($block_ua) { return 403; }

    location / {
        allow 147.93.19.85;
        allow 2a02:4780:12:d33e::1;
        # existing deny lines stay below the allow lines
        # deny 203.0.113.0/24;
        try_files $uri $uri/ /index.php?$args;
    }
}

In nginx, allow and deny are checked in order and the first match wins, so the allow lines must come first.

Platform steps

Plesk

Log in to Plesk, then Websites & Domains › (domain) › Apache & nginx Settings › Additional nginx directives (Plesk docs): paste the if rules there (the geo and map blocks go in the http context, which only root can edit in /etc/nginx/conf.d/).

Hostinger VPS / any VPS

Edit /etc/nginx/sites-available/example.com (or /etc/nginx/conf.d/example.com.conf), then nginx -t && systemctl reload nginx.

Verify the fix

curl -sS -o /dev/null -w "%{http_code}\n" -A "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0 Safari/537.36 SEOAuditSoftware" https://example.com/

Returns 200. Then Crawl again in Site crawl and check the Pages tab.

Automation

Assisted: needs server SSH (root or sudo) to edit nginx config.

403 from Apache or .htaccess

How we detect it

The home page returns 403 with server: Apache (or LiteSpeed, which reads .htaccess the same way) and a plain “Forbidden” page.

Why it matters

Old .htaccess bot lists copied from blogs block many modern tools.

Common causes

If .htaccess has SetEnvIfNoCase User-Agent "..." bad_bot with Deny from env=bad_bot or Require not env bad_bot → a User-Agent match.
If it has RewriteCond %{HTTP_USER_AGENT} ... [NC] and RewriteRule .* - [F] → a User-Agent match.
If it has Deny from / Require not ip lines → an IP or range match.

Fix: basic

Open File Manager in the hosting panel (cPanel: Files › File Manager, tick Show Hidden Files in Settings; Hostinger: hPanel › Websites › Manage › Files › File Manager).
Open public_html/.htaccess and click Edit.
Search for User-Agent, HTTP_USER_AGENT, Deny from and Require.
Add the exception lines from Fix: expert above those rules (Apache reference: mod_authz_core).
Save, then click Crawl again in Site crawl.

Fix: expert

For SetEnvIf bot lists (Apache 2.4):

# SEO Audit Software: never treat as a bad bot
SetEnvIf Remote_Addr "^147\.93\.19\.85$" seoaudit
SetEnvIf Remote_Addr "^2a02:4780:12:d33e::1$" seoaudit
SetEnvIfNoCase User-Agent "SEOAuditSoftware" seoaudit
SetEnvIf seoaudit 1 !bad_bot

For mod_rewrite bot lists, add a condition so the block skips our IPs:

RewriteEngine On
RewriteCond %{REMOTE_ADDR} !^147\.93\.19\.85$
RewriteCond %{REMOTE_ADDR} !^2a02:4780:12:d33e::1$
RewriteCond %{HTTP_USER_AGENT} (badbot|scraper|crawler) [NC]
RewriteRule .* - [F,L]

Keep your own pattern in the HTTP_USER_AGENT line; just add the two REMOTE_ADDR lines above it.

For IP-based access control (Apache 2.4):

<RequireAny>
    Require ip 147.93.19.85
    Require ip 2a02:4780:12:d33e::1
    <RequireAll>
        Require all granted
        Require not ip 203.0.113.0/24
    </RequireAll>
</RequireAny>

Replace 203.0.113.0/24 with your existing blocked ranges.

mod_evasive / mod_qos (server-wide DoS modules) can also return 403. Allow our IPs:

<IfModule mod_evasive20.c>
    DOSWhitelist 147.93.19.85
</IfModule>

Platform steps

cPanel

Files › File Manager › public_html › .htaccess › Edit.

Plesk

In Plesk: Websites & Domains › File Manager, open httpdocs/.htaccess.

Hostinger hPanel

In hPanel: Websites › Manage › Files › File Manager, open public_html/.htaccess.

Verify the fix

The curl command from the nginx section returns 200. Then Crawl again in Site crawl.

Automation

Automatable: needs file access (FTP/SFTP, File Manager or SSH) to edit .htaccess.

403 from ModSecurity

How we detect it

The response is 403 (sometimes 406 “Not Acceptable!”) and the page mentions “Mod_Security” or “This error was generated by Mod_Security”. On some hosts it is a generic 403.

Why it matters

ModSecurity rule sets (OWASP CRS, Comodo, Imunify) flag automated User-Agents or unusual headers, and block every page for us.

Common causes

If the server error log shows ModSecurity: Access denied with code 403 and a rule id → that rule matched our request.
If it happens only on URLs with long query strings → a URL rule, not a bot rule.

Fix: basic

cPanel

Log in to cPanel and go to Security › ModSecurity (cPanel ModSecurity docs).
For a quick test, switch ModSecurity Off for the domain and click Crawl again in Site crawl. If it works, switch it back On and allow our IP instead (expert steps, or send the host the lines below).

Plesk

Log in to Plesk and go to Tools & Settings › Web Application Firewall (ModSecurity) (Plesk docs; domain level: Websites & Domains › (domain) › Web Application Firewall).
Open the Logs link and find the rule ID that blocked 147.93.19.85.
In Switch off security rules, enter that ID and click OK. Or set the mode to Detection only for a test.

hPanel (Hostinger shared hosting)

ModSecurity isn’t user-editable on shared plans. Open a chat from hPanel or the Hostinger help centre and send these exact lines: “Please whitelist 147.93.19.85 and 2a02:4780:12:d33e::1 in ModSecurity for example.com. They are our SEO audit tool’s crawler.”

Fix: expert

Add this rule (root access; for example in /etc/apache2/conf.d/modsec/modsec2.user.conf on cPanel, or /etc/modsecurity/custom.conf):

# SEO Audit Software crawler: no ModSecurity checks for its IPs
SecRule REMOTE_ADDR "@ipMatch 147.93.19.85,2a02:4780:12:d33e::1" \
    "id:1000001,phase:1,pass,nolog,ctl:ruleEngine=Off"

Find the rule that blocked us:

grep -h "147.93.19.85" /usr/local/apache/logs/modsec_audit.log /var/log/apache2/modsec_audit.log /var/log/nginx/error.log 2>/dev/null | grep -oE 'id "[0-9]+"' | sort | uniq -c

Reload the web server after editing: apachectl -t && systemctl reload httpd (or apache2), or nginx -t && systemctl reload nginx.

Platform steps

cPanel / WHM

WHM: Security Center › ModSecurity Tools › Rules List › Add Rule, paste the rule above, tick Deploy and Restart Apache.

Plesk

Tools & Settings › Web Application Firewall (ModSecurity) › Settings › Custom directives (if available on your version), paste the rule.

Verify the fix

curl returns 200 and the ModSecurity audit log shows no new entries for our IP. Then Crawl again in Site crawl.

Automation

Assisted: needs root SSH or WHM/Plesk admin access.

Host firewall blocks our IP (CSF, fail2ban, iptables, IP Blocker)

How we detect it

Usually No response (connection dropped or refused) rather than a 403, sometimes after a few pages when a firewall auto-bans the IP.

Why it matters

A server-level ban blocks every site on the server for that IP, and auto-bans often last hours.

Common causes

If the ban happens after some pages → CSF/LFD connection tracking or a fail2ban jail (for example on many 404s or requests per second).
If it fails from the first request → our IP is in a deny list (csf.deny, cPanel IP Blocker, iptables).
If the site blocks whole countries → our server’s country may be blocked.

Fix: basic

cPanel IP Blocker

Log in to cPanel and go to Security › IP Blocker.
If 147.93.19.85 or a range containing it is listed, click Delete.

Plesk Fail2Ban

Log in to Plesk (Plesk docs) and go to Tools & Settings › IP Address Banning (Fail2Ban).
On Banned IP Addresses, select 147.93.19.85 if listed and click Unban.
On Trusted IP Addresses, click Add Trusted IP, enter 147.93.19.85, click OK. Repeat for 2a02:4780:12:d33e::1.

Hostinger hPanel

In hPanel, go to Websites › Manage › Advanced › IP Manager (on some plans: Security › IP Manager).
Make sure our IP isn’t under Block. Add 147.93.19.85 under Allow if the option is offered.

Fix: expert

CSF (ConfigServer Security & Firewall). Add to both files:

# /etc/csf/csf.allow  (always allow)
147.93.19.85 # SEOAuditSoftware
2a02:4780:12:d33e::1 # SEOAuditSoftware

# /etc/csf/csf.ignore  (LFD never auto-blocks)
147.93.19.85 # SEOAuditSoftware
2a02:4780:12:d33e::1 # SEOAuditSoftware

Then:

csf -dr 147.93.19.85
csf -ra

fail2ban:

# /etc/fail2ban/jail.local
[DEFAULT]
ignoreip = 127.0.0.1/8 ::1 147.93.19.85 2a02:4780:12:d33e::1

fail2ban-client set nginx-botsearch unbanip 147.93.19.85
systemctl restart fail2ban

(Replace nginx-botsearch with the jail that banned us; fail2ban-client status lists them.)

iptables / ufw:

iptables -I INPUT -s 147.93.19.85 -p tcp -m multiport --dports 80,443 -j ACCEPT
ip6tables -I INPUT -s 2a02:4780:12:d33e::1 -p tcp -m multiport --dports 80,443 -j ACCEPT
# or with ufw:
ufw allow from 147.93.19.85 to any port 80,443 proto tcp
ufw allow from 2a02:4780:12:d33e::1 to any port 80,443 proto tcp

Platform steps

cPanel / WHM with CSF

WHM › Plugins › ConfigServer Security & Firewall: use Quick Allow and Quick Ignore with 147.93.19.85, then Restart csf+lfd.

Verify the fix

csf -g 147.93.19.85 shows it in the allow list, or fail2ban-client get nginx-botsearch ignoreip lists it. Crawl again in Site crawl shows real status codes instead of No response.

Automation

Assisted: needs root SSH, or WHM/Plesk/hPanel access.

429 Too Many Requests (rate limits)

How we detect it

Pages return 429, usually after the first few pages. The response may include a Retry-After header with the seconds to wait.

Why it matters

Our crawler requests one page at a time with a 0.25 second pause. A limit that blocks this is strict enough to slow down good bots, including search engines.

Common causes

If the server is nginx → limit_req (requests per second) or limit_conn.
If it is Apache → mod_evasive, mod_qos or a host’s “anti-bot” layer.
If the 429 has server: cloudflare → see the Cloudflare guide.
If it is a WordPress plugin → see the WordPress plugin guide.

Fix: basic

Find which layer sends the 429 (server header and page text).
Exempt our IPs in that layer (snippets below; nginx reference: limit_req module), or raise the limit to at least 5 requests per second with a burst of 20.
Click Crawl again in Site crawl.

Fix: expert

nginx: exempt our IPs from limit_req. Requests with an empty key are not counted.

# http { } context, for example /etc/nginx/conf.d/ratelimit.conf
geo $limit {
    default                 1;
    147.93.19.85            0;
    2a02:4780:12:d33e::1    0;
}

map $limit $limit_key {
    0   "";
    1   $binary_remote_addr;
}

limit_req_zone $limit_key zone=perip:10m rate=5r/s;

# server { } or location { } context
limit_req zone=perip burst=20 nodelay;
limit_req_status 429;

Test and reload:

nginx -t && systemctl reload nginx

Platform steps

Plesk

Rate limits in Plesk usually come from Fail2Ban or ModSecurity: use the ModSecurity and host firewall sections above.

Shopify, Wix, Squarespace

Their limits can’t be changed: see Fix: Crawling Shopify, Wix, Squarespace and Webflow sites.

Verify the fix

Run 10 quick requests and check none return 429:

for i in $(seq 1 10); do curl -sS -o /dev/null -w "%{http_code} " -A "SEOAuditSoftware" https://example.com/; sleep 0.25; done; echo

From your own IP this tests the limit in general; the exemption for our IP shows in the next Crawl again (no 429 rows in the Pages tab).

Automation

Assisted: needs server SSH to edit nginx or Apache config.

Common questions

How do I exempt SEO Audit Software from nginx rate limiting?

Use a geo block that gives our IPs 147.93.19.85 and 2a02:4780:12:d33e::1 the value 0, map it to an empty key, and use that key in limit_req_zone. Requests with an empty key are not counted. The full snippet is in the 429 section of this guide.

How do I stop ModSecurity blocking the crawler?

Add a rule that turns the rule engine off for our IP: SecRule REMOTE_ADDR "@ipMatch 147.93.19.85,2a02:4780:12:d33e::1" "id:1000001,phase:1,pass,nolog,ctl:ruleEngine=Off". In cPanel or Plesk you can instead switch off the single rule ID that matched.

Does the crawler respect Crawl-delay in robots.txt?

No. It reads Allow and Disallow rules but sets its own pace: one request at a time with a short pause, which is a few requests per second at most. Exempt our IP from rate limits instead.

What is the difference between 403 and 429?

403 Forbidden means the server decided to refuse the request (a rule, a ban or a firewall). 429 Too Many Requests means a rate limit was reached; it often comes with a Retry-After header saying how long to wait.

Still need help?

Ask in the chat inside the app, or send us a message on the contact page. You can also browse all help topics.

Fix: 403 and 429 from the server, host firewall or ModSecurity (nginx, Apache, cPanel, Plesk, hPanel)

403 from an nginx rule

How we detect it

Why it matters

Common causes

Fix: basic

Fix: expert

Platform steps

Plesk

Hostinger VPS / any VPS

Verify the fix

Automation

403 from Apache or .htaccess

How we detect it

Why it matters

Common causes

Fix: basic

Fix: expert

Platform steps

cPanel

Plesk

Hostinger hPanel

Verify the fix

Automation

403 from ModSecurity

How we detect it

Why it matters

Common causes

Fix: basic

Fix: expert

Platform steps

cPanel / WHM

Plesk

Verify the fix

Automation

Host firewall blocks our IP (CSF, fail2ban, iptables, IP Blocker)

How we detect it

Why it matters

Common causes

Fix: basic

Fix: expert

Platform steps

cPanel / WHM with CSF

Verify the fix

Automation

429 Too Many Requests (rate limits)

How we detect it

Why it matters

Common causes

Fix: basic

Fix: expert

Platform steps

Plesk

Shopify, Wix, Squarespace

Verify the fix

Automation

Common questions

Related articles

Still need help?