• [MOD] Treat bots as users with less rights 3 0 5 1
Currently:  

Author Topic: [MOD] Treat bots as users with less rights  (Read 116888 times)

0 Members and 1 Guest are viewing this topic.

Offline martrix

  • Hero Member
  • *****
  • Posts: 755
    • View Profile
    • overlord.cz
[MOD] Treat bots as users with less rights
« on: July 06, 2005, 02:42:20 PM »
90% of the mod: V@no/Jan

A part of this mod was already published in this forum, but unfortunately it got lost after the hack.

Step 1: Create new users called for example "searchbot" and "googlebot" (or any other aditional you want...)  with password "12345"
(you may use any username or password you want, but then you will need to change the code)
The password 12345 is just for "demonstration" - please use another one for your bots and replace 12345 in the code with your new password...
If you decide not to do that, everybody can visit your page as a "bot"


Step 2: Create a new usergroup "searchbots" and set all bot-users as members of this group.

Step 3: in includes/sessions.php find:

Code: [Select]
$site_sess = new Session();
and replace it with:

Code: [Select]
// mod GoogleBot
$site_sess = new Session();
$robots = array(
"FAST-WebCrawler" => "searchbot",
"Gulliver" => "searchbot",
"AltaVista" => "searchbot",
"Alexibot" => "searchbot",
"asterias" => "searchbot",
"BackDoorBot" => "searchbot",
"Black.Hole" => "searchbot",
"BlowFish" => "searchbot",
"BotALot" => "searchbot",
"NPBot" => "searchbot",
"Pompos" => "searchbot",
"Scooter" => "searchbot",
"SiteSnagger" => "searchbot",
"Slurp" => "searchbot",
"WebmasterWorldForumBot" => "searchbot",
"zyborg" => "searchbot",
"Google" => "googlebot",
"msnbot" => "searchbot",
"ZACATEK_CZ_BOT" => "searchbot",
"EARTHCOM.info" => "searchbot",
"sherlock" => "searchbot",
"Holmes" => "searchbot",
"ia_archiver" => "searchbot",
"lwp-trivial" => "searchbot",
"appie" => "searchbot",
"Scooter" => "searchbot",
"Gigabot" => "searchbot",
"Wget" => "searchbot",
"jyxobot" => "searchbot",
"Xenu Link Sleuth" => "searchbot",
"SeznamBot" => "searchbot",
);
foreach($robots as $key => $val) {
  if(!$_SERVER['HTTP_USER_AGENT'] || preg_match("/".$key."/i", $_SERVER['HTTP_USER_AGENT'])) {
    $site_sess->login($val, "12345");
    $user_bot = 1;
    break;
  }
}

You may add or remove other bots's useragent names as you like or try to put MSIE there for a while, so you may see, how the bot will see the pages... don't forget to delete MSIE from the array aftewards, otherwise ALL your visitors will be logged in as bots ;) (well, or you could just login as a searchbot :) but if you do that and you do hide the userbox for searchbots, you have to delete the cookies, to login back again under your "real" account.)

Step 4.1: If you did not install Conditional User Group template tags yet, follow the next steps
Open includes/page_header.php and search for each
Code: [Select]
if ($user_info['user_level'] >= USER) {add the following code above
Code: [Select]
$usergroups = array();AND add the following code below
Code: [Select]
  $current_time = time();
  $sql = "SELECT group_id 
          FROM ".GROUP_MATCH_TABLE."
          WHERE user_id = ".$user_info['user_id']."
          AND groupmatch_startdate <= $current_time
          AND (groupmatch_enddate > $current_time OR groupmatch_enddate = 0)";
  $result = $site_db->query($sql);

  while ($row = $site_db->fetch_array($result)) {
    $usergroups[$row['group_id']] = true;
  }
Step 4.2 search for
Code: [Select]
//-----------------------------------------------------
//--- User Box ----------------------------------------
//-----------------------------------------------------
and add the following code above
Code: [Select]
$site_template->register_vars(array(
"is_bot" => (isset($usergroups[XXX])) ? 1 : 0
));
Where XXX has to be replaced with the number of the usergroup. (If you don't know, how to find out the number of the usergroup, look here)
and from now on you may use this tag:

{if is_bot}
 This will appear only for the searchbots...
{endif is_bot}

if you want to hide some parts or disable some functions for the searchbots, install [MOD] Adding {ifno tagname1}{tagname2}{endifno tagname1} v1.7/v1.7.1
and you will be able to disable :) thes parts by
Code: [Select]
{ifnot is_bot}
This code will not be shown to searchbots
{endifnot is_bot}

On my site bots may browse the pages, but they can't see links to: download, favorities, user-profile, private messages, logout etc.

That should be everything...
Tell me if it works for you - I did not try it out "again" on a fresh install, so I could forget something...
« Last Edit: December 31, 2013, 06:02:13 AM by Rembrandt »
MAяTRIX


Offline mawenzi

  • Moderator
  • 4images Guru
  • *****
  • Posts: 4.500
    • View Profile
Re: [MOD] Treat bots as users with less rights
« Reply #1 on: July 06, 2005, 03:49:14 PM »
hi martrix,

you are the man ... thanks ... !
that's what I'm searching for some time ...   8O
nice that you re-publish the MOD with your own statements ...  :D
thanks again ... !

mawenzi
Your first three "must do" before you ask a question ! ( © by V@no )
- please read the Forum Rules ...
- please study the FAQ ...
- please try to Search for your answer ...

You are on search for top 4images MOD's ?
- then please search here ... Mawenzi's Top 100+ MOD List (unsorted sorted) ...

Offline JensF

  • Addicted member
  • ******
  • Posts: 1.028
    • View Profile
    • http://www.terraristik-galerie.de
Re: [MOD] Treat bots as users with less rights
« Reply #2 on: August 04, 2005, 10:30:11 AM »
Quote
On my site bots may browse the pages, but they can't see links to: download, favorities, user-profile, private messages, logout etc

Can anyone say me what i must "save" to have this?? Where must i put the tags in??
Mit freundlichem Gruß
Jens Funk



-> Sorry for my bad English <-

Offline martrix

  • Hero Member
  • *****
  • Posts: 755
    • View Profile
    • overlord.cz
Re: [MOD] Treat bots as users with less rights
« Reply #3 on: August 04, 2005, 11:20:45 AM »
If you finish all the changes, then you may use the tags {if is_bot} blabla {endif is_bot} or {ifnot is_bot} blabla {endifnot is_bot}

For example: you would like to show the text "Dear bot, welcome to my page" to the bots, then just insert

Code: [Select]
{if is_bot} "Dear bot, welcome to my page" {endif is_bot}
anywhere you wish in the templates...

or if you want to hide favourites button from bots, so they won't use it, then just replace

Code: [Select]
{lightbox_button}
in details.html with

Code: [Select]
{ifnot is_bot}{lightbox_button}{endifnot is_bot}
or if you don't want the bot to see the control-panel of the userbot-account, just replace

Code: [Select]
<a href="{url_control_panel}">{lang_control_panel}</a>
in user_logininfo.html with

Code: [Select]
{ifnot is_bot}<a href="{url_control_panel}">{lang_control_panel}</a>{endifnot is_bot}
The rest is up to you, your imagination and your needs...

But beware: if you don't want to give anybody the possibility of seing the image-details, don't install this - anybody may change the name of his useragent and browse through your pages without registering (that's the reason, why I made up a lot of restrictions for bots)

Have fun
« Last Edit: December 31, 2013, 06:31:54 AM by Rembrandt »
MAяTRIX


Offline leedzinh

  • Pre-Newbie
  • Posts: 7
    • View Profile
Re: [MOD] Treat bots as users with less rights
« Reply #4 on: August 06, 2005, 05:28:30 AM »
do you know yahoo bot?

Offline V@no

  • If you don't tell me what to do, I won't tell you where you should go :)
  • Administrator
  • 4images Guru
  • *****
  • Posts: 17.849
  • mmm PHP...
    • View Profile
    • 4images MODs Demo
Re: [MOD] Treat bots as users with less rights
« Reply #5 on: August 06, 2005, 05:59:32 AM »
I know two yahoo bots (dont know if they have more).
Its identify itself as:
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
and
ia_archiver

In the code I used this to identify them:
Code: [Select]
      "ia_archiver" => "bot-yahoo",
      "Yahoo! Slurp" => "bot-yahoo",
      "help\.yahoo\.com" => "bot-yahoo",
Your first three "must do" before you ask a question:
Please do not PM me asking for help unless you've been specifically asked to do so. Such PMs will be deleted without answer. (forum rule #6)
Extension for Firefox/Thunderbird: Master Password+    Back/Forward History Tweaks (restartless)    Cookies Manager+    Fit Images (restartless for Thunderbird)

Offline leedzinh

  • Pre-Newbie
  • Posts: 7
    • View Profile
Re: [MOD] Treat bots as users with less rights
« Reply #6 on: August 06, 2005, 06:42:31 AM »
anyone know more bot please post all of them

thanks!

Offline bibabobu

  • Sr. Member
  • ****
  • Posts: 311
  • Technische und künstlerische Gravuren
    • View Profile
    • Technische und künstlerische Gravuren für die gesamte Metall-, Kunststoff- und Papierindustrie
Re: [MOD] Treat bots as users with less rights
« Reply #7 on: August 06, 2005, 08:57:11 AM »
Sorry.
Perhaps i am too stupid to understand. :roll:
What does this MOD do? What are the advantages?
 Do searchbots of search engines find better keywords on my site for a better ranking or what?

Please could someone explain me? :oops:

Offline V@no

  • If you don't tell me what to do, I won't tell you where you should go :)
  • Administrator
  • 4images Guru
  • *****
  • Posts: 17.849
  • mmm PHP...
    • View Profile
    • 4images MODs Demo
Re: [MOD] Treat bots as users with less rights
« Reply #8 on: August 06, 2005, 09:49:43 AM »
its usefull if u have categories not acceptible for guests, but u still want them (categories) to be indexed on search engines.
Your first three "must do" before you ask a question:
Please do not PM me asking for help unless you've been specifically asked to do so. Such PMs will be deleted without answer. (forum rule #6)
Extension for Firefox/Thunderbird: Master Password+    Back/Forward History Tweaks (restartless)    Cookies Manager+    Fit Images (restartless for Thunderbird)

Offline ~Ally~

  • Pre-Newbie
  • Posts: 7
    • View Profile
Re: [MOD] Treat bots as users with less rights
« Reply #9 on: August 07, 2005, 04:37:49 AM »
Thank you SO much. I am hoping that this mod will help with the HUGE amount of RAM used up by my galley. .htaccess bans do not work anymore for bots so I'm hoping this will. thank you!

Offline bibabobu

  • Sr. Member
  • ****
  • Posts: 311
  • Technische und künstlerische Gravuren
    • View Profile
    • Technische und künstlerische Gravuren für die gesamte Metall-, Kunststoff- und Papierindustrie
Re: [MOD] Treat bots as users with less rights
« Reply #10 on: November 04, 2005, 02:35:53 PM »
Please could someone help me because i am little bit confused.

If i want all of my images and pages to be indexed by search engines when my categories are only accessible for members
i will have to install this MOD.

Is this right???

So is there a problem to also install the MOD Registration with security code validation v1.0 :?:

Do the search engines also indexing my pages if i install the security MOD?

I hope i explained clear enough with my broken english :oops:

Offline V@no

  • If you don't tell me what to do, I won't tell you where you should go :)
  • Administrator
  • 4images Guru
  • *****
  • Posts: 17.849
  • mmm PHP...
    • View Profile
    • 4images MODs Demo
Re: [MOD] Treat bots as users with less rights
« Reply #11 on: November 04, 2005, 03:00:02 PM »
The bots will not be registering by themselfs, you as admin will register them, meaning you will need create account(s) for bots, and update names in the mod, so when a bot visit your site 4images will automaticaly log them in as a member you specifyed.
Your first three "must do" before you ask a question:
Please do not PM me asking for help unless you've been specifically asked to do so. Such PMs will be deleted without answer. (forum rule #6)
Extension for Firefox/Thunderbird: Master Password+    Back/Forward History Tweaks (restartless)    Cookies Manager+    Fit Images (restartless for Thunderbird)

Offline impss

  • Sr. Member
  • ****
  • Posts: 382
    • View Profile
    • Cusstom.net
Re: [MOD] Treat bots as users with less rights
« Reply #12 on: December 14, 2005, 05:30:52 PM »
The bots will not be registering by themselfs, you as admin will register them, meaning you will need create account(s) for bots, and update names in the mod, so when a bot visit your site 4images will automaticaly log them in as a member you specifyed.

So you will need to register each bot that you want, in the list above?

Offline mawenzi

  • Moderator
  • 4images Guru
  • *****
  • Posts: 4.500
    • View Profile
Re: [MOD] Treat bots as users with less rights
« Reply #13 on: December 14, 2005, 06:21:36 PM »
So you will need to register each bot that you want, in the list above?

... NO ...
For the list above you only need to register one new member ... "searchbot" !
... but ...
If you want a searchbot as a single member e.g. Google ... than register a new member "googlebot" ... and set in in list
Code: [Select]
"Google" => "googlebot",

... for other single searchbots is to proceed similar ...

mawenzi
Your first three "must do" before you ask a question ! ( © by V@no )
- please read the Forum Rules ...
- please study the FAQ ...
- please try to Search for your answer ...

You are on search for top 4images MOD's ?
- then please search here ... Mawenzi's Top 100+ MOD List (unsorted sorted) ...

Offline impss

  • Sr. Member
  • ****
  • Posts: 382
    • View Profile
    • Cusstom.net
Re: [MOD] Treat bots as users with less rights
« Reply #14 on: December 14, 2005, 06:53:53 PM »
Thanks mawenzi  8)