Previously, previously, previously.
Requires: 2.8 or higher
Compatible up to: 3.3.2
Last Updated: 2012-6-1
It appears that the Limit Login Attempt plugin may only be compatible up through WP 3.3.2. If you're current (i.e. on 3.5.1) it may or may not work. Since the botnet is bruteforcing the admin user, you could just change the admin user's userid. This, however, requires directly editing the wp_users db table as WP doesn't allow you to change a username via the web UI.
If you don't want to hack on your wp db, you should just make sure you're using a strong password for your admin account. Something randomly generated and very long ( > 30 characters ) would be best.
The Limit Login Attempts plugin works fine with WordPress 3.5.1.
But such countermeasures are not particularly effective against botnets. Best to simply use strong passwords which can't be brute-forced. Alternatively, use the Google Authenticator plugin with the Google Authenticator app on your phone and get 2-factor authentication.
Um, if it's a botnet, won't it be coming from a wide range of random IP's rendering this plugin less than useful?
Unless you've got "password"/"12345" style passwords, brute-force password cracking requires an order of magnitude more requests than "a wide range".
Consider a botnet that has 20,000 computers. Limiting logins to 10 requests per IP address means the ideal botnet only gets 200,000 login attempts. Assuming you use numbers and letters in your password (36 possible characters), that means a crude brute force botnet will get through all the one, two, and three character passwords, and part of the way through the four character ones.
In other words, "A reasonable countermeasure".
I am very much of the limit IP access to your admin page mindset. Mobile users will say that this makes life difficult for them because their IP is random, but that's what VPN is for.
OpenVPN works well with iPhone and Android built-in VPN, but it can be complicated to configure. Or take a simpler approach like port-knocking or SSL client certificate authentication for the admin endpoint via Apache.
If you don't want to bother with any of that, consider what Duo is offering.
Duo is offering FREE two factor auth for WordPress. It's about 3/4 way down the page...
Also, how can we tell if we're compromised?
They probably won't do anything interesting with your compromised site for a bit, unless they can somehow install a web shell. Later, when there's some interesting 0-day exploit out, they'll plant code on your site that either does the exploit directly or sends browsers off to someplace that does the exploit. The code may look innocuous, it may mostly look like a bunch of random numbers in an array. The code may break your site in some way, or it may have no noticeable effect.
Or you could look at your web server logs or wordpress logs for admin activity that's not you.
The issue with using a WordPress plug-in is that the requests still have to processed through the bloated and expensive WordPress engine. On a server with hundreds of WordPress sites that are being attacked by this botnet, something like this becomes much more effective:
AuthName "Extra WordPress Security"
Thanks for this.