4images Forum & Community
Welcome, Guest. Please login or register.
Did you miss your activation email?
October 18, 2017, 12:12:01 AM

Login with username, password and session length
Search:     Advanced search
Togle to toolbar
Translate this page with =>
Translate this page >
* Home Help Search Login Register
 
+  4images Forum & Community
|-+  General / Allgemeines
| |-+  Programming
| | |-+  Site slow down due to BOTs...
0 Members and 1 Guest are viewing this topic. « previous next »
Pages: [1] Print
This topic has not yet been rated!
Author Topic: Site slow down due to BOTs...  (Read 4070 times)
MrAndrew
Sr. Member
****
Offline Offline

Posts: 302

Thank You
-Given: 9
-Receive: 13


View Profile WWW
« on: October 18, 2011, 01:34:12 PM »

There are many requests of the any robots on my site. This is really slow down site loading. Anytime i have error "502 Bad Gateway"...

Look at the information from my log file below:

requests    %bytes       Organization
12334   71,78%   googlebot.com
7913   46,40%   crawl-66-249-66-9.googlebot.com
2810   15,85%   crawl-66-249-72-24.googlebot.com


What to do? Any solutions?

Thanks!
Logged

dsc_phi
Newbie
*
Offline Offline

Posts: 16

Thank You
-Given: 0
-Receive: 0


View Profile WWW
« Reply #1 on: January 30, 2016, 04:08:05 AM »

Hello and happy new year.

I am very bothered (bored) with slow dances repeated logs égaement of Malformed Google, MSNBot, Apple Bot, Malformed MJ12 and others.

I gave a more important value of crawl-delay via the file robot txt, but for Google it is necessary to go to tools if your site is registered (recorded) and in the setting of the site, to ask for a slowing down of the indexation. I do not manage to find the good compromise, it is the real bad adventure.
I have slowlogs every day non-stop now and it is painful. I am going to try to reduce the time-out of the session to 5 and I would see if that changes actually something.
I don't know of when dates this post but if that can help or if somebody has a real idea.
The servers mutuality is sometimes a little limited, even with the compression deflates and the putting in mask (hiding place).

Sorry for my english.

Je suis très ennuyé avec les slows logs égaement à répétition de Google Bot, MSNBot, Apple Bot, MJ12 Bot et d'autres.

J'ai donné une valeur de crawl-delay plus importantes via le fichier robot txt, mais pour google il faut aller dans les outils si votre site est enregistré et dans le setting du site, demander un ralentissement de l'indexation.Je n'arrive pas à trouver le bon compromis, c'est une vraie galère.
J'ai des slowlogs tous les jours sans arrêt maintenant et c'est pénible.Je vais tenter de réduire le time-out de la session à 5 et je verrais si ça change effectivement quelque chose

.Je ne sais pas de quand date ce poste mais si ça peut aider ou si quelqu'un a une véritable idée.
Les serveurs mutualités sont parfois un peu limités, même avec la compression deflate et la mise en cache.

Nice weekend, bon weekend ...
Logged
ardosan
Pre-Newbie

Offline Offline

Posts: 1

Thank You
-Given: 0
-Receive: 0


View Profile
« Reply #2 on: July 16, 2017, 03:35:44 PM »

I don't know why there are a lot of bots on your site. Something strange! maybe try to play with anti-bot systems for websites. It should help!
Logged
Pages: [1] Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!
Page created in 0.045 seconds with 19 queries.