Robots.txt: who is looking for the files you want to keep hidden

When hackers first probe a site for vulnerabilities, there is one thing that is almost always on their list, robots.txt. This is a special file for search engine crawlers telling them which pages or files from your site they should or shouldn’t parse and index. Pay attention to the second part, it tells search engines which files …

Continue reading

Pro Tips for Choosing the Right Web Host

There was a time, early in the history of the web, when choosing a hosting provider was easy. You compared a number of server parameters like CPU, memory size, and bandwidth You considered some less important factors like disk capacity, along with more important factors like price, and, eventually, based on what you found, you …

Continue reading

A Short Custom Rule to Prevent Panic Attacks

We at BotGuard know what it’s like to help our users through moments of panic, but we’d rather help you save those nerve cells even though they probably can regenerate. Here’s a simple rule that might offer some prevention against stress-inducing moments. BotGuard support team engineers sometimes encounter an unfortunate situation when a user, after enabling website protection, …

Continue reading