To block specific IP addresses from accessing a specific VirtualHost or your entire webserver, you can set up a RewriteMap with the banned IPs.
RewriteEngine on
RewriteMap banned_ips txt:/etc/httpd/conf/banned_ips.txt
RewriteCond ${banned_ips:%{REMOTE_ADDR}} ^banned$ [NC]
RewriteRule ^/.* – [F,L]
The banned_ips.txt file is in this format:
111.111.111.111 banned
222.222.222.222 banned
