4images Forum & Community
4images Modifications / Modifikationen => Mods & Plugins (Releases & Support) => Topic started by: V@no on September 10, 2006, 02:50:33 AM
-
This is a very simple check if a uploaded file was previously uploaded.
The method used in this mod is to generate MD5 hash for each image in the database and then compare them with MD5 hash of newly uploaded files.
The name, filename and extension is ignored by this mod, the contents of the file is what used to generate MD5 hash. Though this method has a weak spot: if a file was altered in any way the MD5 hash will be different from the original file.
----------[ Installation ]------------
Step 1
Download this (http://gallery.vano.org/file71dl) package. (File attached on bottom of this post too)
Extract it and upload image_md5_hash.php file into admin/plugins/ folder (if no such folder exists then create it)
Step 2
Open member.php
Find: if (!$error) {
// Start Upload
include(ROOT_PATH.'includes/upload.php');
Insert above:/*
MOD CHECK FOR DUPLICATE IMAGES
START INSERT
*/
##########
# CONFIG #
##########
$check_admin = true; //do check when administrator is uploading? (true/false)
$show_image = true; //show link to the image that was previously uploaded? (true/false)
$show_member = true; //show name and link to profile page of the member who previously uploaded that file? (true/false)
##############
# END CONFIG #
##############
$md5 = "";
unset($HTTP_POST_VARS['image_md5']);
if ($user_info['user_level'] != ADMIN || $check_admin)
{
if (!empty($HTTP_POST_FILES['media_file']['tmp_name']) && $HTTP_POST_FILES['media_file']['tmp_name'] != "none")
{
$md5 = md5_file($HTTP_POST_FILES['media_file']['tmp_name']);
$file = $HTTP_POST_FILES['media_file']['filename'];
}
elseif ($remote_media_file)
{
$md5 = md5($remote_media_file);
$file = $remote_media_file;
}
if ($md5)
{
$sql = "SELECT image_id, image_name, cat_id, user_id
FROM ".IMAGES_TABLE."
WHERE image_md5 = '".$md5."'
LIMIT 1";
if ($row = $site_db->query_firstrow($sql))
{
$row['image_name'] = stripslashes($row['image_name']);
if (function_exists('multilang')) $row['image_name'] = multilang($row['image_name']);
$user_row = get_user_info($row['user_id']);
// $msg .= (($msg != "") ? "<br />" : "")."<b>".$lang['file_upload_error'].": ".$file."</b><br />";
$msg .= (($msg != "") ? "<br />" : "").(($user_info['user_level'] > GUEST && $user_info['user_id'] == $user_row['user_id']) ? $lang['image_md5_duplicate_self'] : sprintf(($show_member ? $lang['image_md5_duplicate_more'] : $lang['image_md5_duplicate_simple']), "<a href=\"".$site_sess->url(ROOT_PATH."member.php?action=showprofile&".URL_USER_ID."=".$user_row['user_id'])."\">".$user_row['user_name']."</a>"));
if ($show_image && (($user_info['user_level'] > GUEST && $user_info['user_id'] != $user_row['user_id']) || (check_permission("auth_viewcat", $row['cat_id'] && check_permission("auth_viewimage", $row['cat_id'])))))
{
$msg .= ": <a href=\"".$site_sess->url(ROOT_PATH."details.php?image_id=".$row['image_id'])."\">".$row['image_name']."</a>";
}
$error = 1;
}
else
{
$sql = "SELECT image_id, image_name, user_id
FROM ".IMAGES_TEMP_TABLE."
WHERE image_md5 = '".$md5."'
LIMIT 1";
if ($row = $site_db->query_firstrow($sql))
{
$user_row = get_user_info($row['user_id']);
// $msg .= (($msg != "") ? "<br />" : "")."<b>".$lang['file_upload_error'].": ".$file."</b><br />";
$msg .= (($msg != "") ? "<br />" : "").(($user_info['user_level'] > GUEST && $user_info['user_id'] == $row['user_id']) ? $lang['image_md5_duplicate_validation_self'] : sprintf(($show_member ? $lang['image_md5_duplicate_validation_more'] : $lang['image_md5_duplicate_validation_simple']), "<a href=\"".$site_sess->url(ROOT_PATH."member.php?action=showprofile&".URL_USER_ID."=".$user_row['user_id'])."\">".$user_row['user_name']."</a>"));
$error = 1;
}
}
$HTTP_POST_VARS['image_md5'] = $md5;
}
}
/*
MOD CHECK FOR DUPLICATE IMAGES
END INSERT
*/
Step 3
Open lang/<your language>/main.php
At the end, above closing ?> insert:
:flag-en: English:$lang['image_md5'] = "Image MD5 hash";
$lang['image_md5_duplicate_self'] = "You have submitted this file before";
$lang['image_md5_duplicate_more'] = "This file has been previously submitted by %s";
$lang['image_md5_duplicate_simple'] = "This file has been previously submitted";
$lang['image_md5_duplicate_validation_self'] = "You have submitted this file before and awaiting validation.";
$lang['image_md5_duplicate_validation_more'] = "This file has been previously submitted by %s and awaiting validation.";
$lang['image_md5_duplicate_validation_simple'] = "This file has been previously submitted and awaiting validation.";
:flag-ru: Russian:$lang['image_md5'] = "MD5 хэш файла";
$lang['image_md5_duplicate_self'] = "Вы уже раньше заливали этот файл";
$lang['image_md5_duplicate_more'] = "%s уже залили этот файл до вас";
$lang['image_md5_duplicate_simple'] = "Этот файл кто-то уже залил до вас";
$lang['image_md5_duplicate_validation_self'] = "Вы уже залили этот файл, и он ожидает подтверждения от администрации";
$lang['image_md5_duplicate_validation_more'] = "%s уже залили этот файл, и файл ожидает подтверждения от администрации";
$lang['image_md5_duplicate_validation_simple'] = "Кто-то уже залил этот файл, и файл ожидает подтверждения от администрации";
Step 4
Open includes/db_field_definitions.php
At the end, above closing ?> insert:$additional_image_fields['image_md5'] = array($lang['image_md5'], "text", 0);
After installation in ACP (Admin Control Panel) in "Plugins" part of menu you should see new link "Image MD5 hash update".
When you click on it for the first time, the database installation process will start.
After new fields in the databse were added, you can again click on the "Image MD5 hash update" link and generate MD5 hash for all images in your gallery.
NOTE:
MD5 hash for remote images generates from the URL and not from the image content, so, if image was changed on the remote server, MD5 will not change.
P.S.
In case the download link doesnt work, I've attached the package, but I can not guarantee, that the attached package will be up-to-date in the future.
P.P.S.
This mod was created for 4images v1.7.3 but should work on v1.7 as well :)
-
Thanks V@no ,
I have been needing something like this for awhile..
And it seems to be working good..
:D
-
Hi,
is this Mod working when i say in ACP Upload Modus "save image with new name"????
-
yes, the filename is irrelevant for this mod.
-
OK, i will test it but i have a respect for this mod.
you say
If your images/files were resized, watermarked or altered in any way after they were uploaded, the MD5 hash will not be accurate, therefore will allow upload again the original files
My images are watermarked.....can anyone say me in german the problem???
-
Great mod!!!!
Thanks a lot.
Simple but yet effective.
A fantastique piece of code.
-
Hi Vano,
I´ve been using this mod since you posted it, and as i saud, it works great.
I´ld like to know if it is dificult to you to add another feature... i explain:
If you have time... :lol:
When i instaled the mod, it generated the MD5 records for all files in the server... Perfect!
When i try to add an existant file, the mod doesnt allow.. Perfect!
What about the files i already had when i first run the script? is there any way to see if there is duplicate entries in the old files?
Is it dificult to add a menu like "Files Check" that will look at the whole database, and show only files with duplicate MD5 records, giving the possibility do delete one of them?
Thanks in advance,
António
-
Yes, I had thought about it, but since the MD5 hash is not indexed in the database, it creates too much load on the database (at least with the queries I've been testing it)...
anyways, untill I get my site back up this not gonna happend...
-
Did anyone test the MD5 hash generation on a database with over 25k images? I wonder how long it would take and if it would succesfully complete with such a large collection.
-
Did anyone test the MD5 hash generation on a database with over 25k images? I wonder how long it would take and if it would succesfully complete with such a large collection.
You tell it how many images to do at once, and it does it in steps.. and it continues by itself
I did 15k pictures at 100 pics at at time .. I didnt have any problems
-
Did anyone test the MD5 hash generation on a database with over 25k images? I wonder how long it would take and if it would succesfully complete with such a large collection.
You tell it how many images to do at once, and it does it in steps.. and it continues by itself
I did 15k pictures at 100 pics at at time .. I didnt have any problems
Just perfect.
Hey, thanks for telling me that!
-
Help!!
in form upload, photo is up. but appears
Warning: md5_file() [function.md5-file]: open_basedir restriction in effect. File(/tmp/phpzCUGaD) is not within the allowed path(s): (/www/docs/ahmira.com.ar/) in /www/docs/ahmira.com.ar/public_html/member.php on line 586
Warning: md5_file(/tmp/phpzCUGaD) [function.md5-file]: failed to open stream: Operation not permitted in /www/docs/ahmira.com.ar/public_html/member.php on line 586
i have v1.7.3, thx
-
Help?
-
Hi V@no,
I Trust you are well.
My site relies mostly on video within 4Images. My file sizes range from 1MB to over 200 or even 300MB files...
would it affect performance if this mod is applied to my site? when it generates a checksum, does it need to stream the entire file before it can generate it?
Kind Regards,
WeZ
-
Hi V@no & The Rest...
How do i get this mod to integrate with "[Plugin] Batch Import" and "[MOD] multiupload". ?
Do i post this request on this topic or the other two?
Ciao
WeZ
-
Can Anybody tell me how i get this mod to integrate with "[Plugin] Batch Import" and "[MOD] multiupload". ?
Ciao
WeZ
-
please! NO GARANTIE
multi look at this: http://www.4homepages.de/forum/index.php?topic=8517.120
batch-import look at this: http://www.4homepages.de/forum/index.php?topic=15429.0
-
I just followed all the instructions and implemented the code, but I'm getting the following error message. Can someone help with this? I'm using version 1.74
DB Error: Bad SQL Query: SELECT i.image_id, i.cat_id, i.user_id, i.image_name, i.image_description, i.image_keywords, i.image_date, i.image_active, i.image_media_file, i.image_thumb_file, i.image_download_url, i.image_allow_comments, i.image_comments, i.image_downloads, i.image_votes, i.image_rating, i.image_hits, i.image_md5, c.cat_name, u.user_name FROM (4images_images i, 4images_categories c) LEFT JOIN 4images_users u ON (u.user_id = i.user_id) WHERE i.image_active = 1 AND c.cat_id = i.cat_id AND i.cat_id NOT IN (0) ORDER BY i.image_date DESC LIMIT 5
Unknown column 'i.image_md5' in 'field list'
Warning: mysql_num_rows(): supplied argument is not a valid MySQL result resource in /home/wannaber/public_html/wallpaper-downloads/includes/db_mysql.php on line 116
Thanks in advance,
Joanne
-
Joanne,
field image_md5 is missing in your image table
you didn't executed the plugin from Admin CP "PlugIns > Image MD5 hash upadate" :mrgreen: (typo by V@no)
execute it.. then fields be added automaticly
then test it again
-
Excellent! Thanks so much - works great!
Joanne
-
The link to the original package doesn't exist anymore.
Can anyone post another source?
-
... but the download link at the bottom of the first post works ... ;)
-
Hi.
I have few problems with this mod with using multiupload mod.
1. When sending for example 10 pictures, and two of them are the same, script don't tell me this and all images are uploaded.
2. Sometimes script don't detect the same picture in base.
3. Sometimes, two (ideal the same) pictures have two different md5 hash code. Why? I don't use any watermark mod.
Please help me, my gallery is very big (over 78800 gif's), and duplicate files are big problem.
Sorry for my bad english.
-
Help?
dein Avatar löst bei mir eine Virenmeldung aus
Tojanisches Pferd TR/Spy.Banker.vk.1
-
Joanne,
field image_md5 is missing in your image table
you didn't executed the plugin from Admin CP "PlugIns > Image MD5 hash upadate" :mrgreen: (typo by V@no)
execute it.. then fields be added automaticly
then test it again
thanks ;)
-
Where can i download this mod?
The downloadlink is broken
-
Welcome to 4images forum.
The file is attached to the first post
-
thank you - it work's
is there also a way to add this by the check images and add images functions in the admincp?
-
Sorry, I don't understand the question.
Duplicate images check when using "Check new images" feature?
-
i wanted to know if there is also a option to find duplciate picture when i add new pictures over the admincp or direct upload in a database folder
-
Hi very nice mod
Is it possible to make a plugin where you can detect double images. So that you run that plugin and generate the md5 hash of all the pictures and than a plugin which search inside the database for double entries in the md5 cell?
Than list double founded images and you can delete it...
Thanks
Crazymodder
-
sure
After installation in ACP (Admin Control Panel) in "Plugins" part of menu you should see new link "Image MD5 hash update".
When you click on it for the first time, the database installation process will start.
After new fields in the databse were added, you can again click on the "Image MD5 hash update" link and generate MD5 hash for all images in your gallery.
;)
-
Yes than all my images have an md5 hash. But I don't know if there are some pictures with the same md5 hash.
So I want to show alle pcitures with the same md5 hash so that i can delete the duplicates.
-
Now I see what you are saying.
currently there is no easy way to do so.
What you can do is run this query in phpmyadmin:
SELECT image_md5, COUNT(*) AS num
FROM 4images_images
GROUP BY image_md5
HAVING num>1(assuming 4images_ is the table prefix set in your config.php)
It should give you a list of duplicate MD5 (if any)
Then having them, you can search for images using [MOD] Batch Copy/Move/Edit Images v4.15.1 (2010-08-14) (http://www.4homepages.de/forum/index.php?topic=6759.0) (make sure at MD5 search field select "default" in dropdown)
-
Now I see what you are saying.
currently there is no easy way to do so.
What you can do is run this query in phpmyadmin:
SELECT image_md5, COUNT(*) AS num
FROM 4images_images
GROUP BY image_md5
HAVING num>1(assuming 4images_ is the table prefix set in your config.php)
It should give you a list of duplicate MD5 (if any)
Then having them, you can search for images using [MOD] Batch Copy/Move/Edit Images v4.15.1 (2010-08-14) (http://www.4homepages.de/forum/index.php?topic=6759.0) (make sure at MD5 search field select "default" in dropdown)
Hi,
i have 628 duplicate files in my galery, is there a method to search for that hashes with your mod at once?
Seems to be much work: pasting each hash into the search page and deleting them after that :)
Hope there is a clou?
Maybe something directly in database? but i don't know what rows have to be infected?
And how to submit the list of 628 hashes to the command that should delete that... uh i don't know
Greetz X23
-
Will this mod work on 1.7.10? :?
-
@Vano, Is it possible to modify this MOD to just simply check for duplicate filename only? Best variant would be to check for duplicate file name and extension while uploading
Thanks!
-
:thumbup: das wäre klasse, schon beim hoch laden müsste eine Meldung kommen "Dateinamen vorhanden" und Unterscheidung groß und Kleinschrift.
-
Hi!
Ich verstehe nicht wozu ihr das braucht, wenn der "Dateiname" in dem selben Kategorie Ordner schon vorhanden ist, hängt 4images sowieso eine Zahl hinter den Namen drann.
Vorrausgesetzt ihr habt im ACP unter Allgemein/Einstellungen/Upload Einstellungen/Upload-Modus auf "Dateien mit neuem Namen speichern" gestellt.
mfg Andi
-
nicht immer Andy, Bild ist nicht gleich bild
OK, im Web wird das unterschieden, also nicht überschrieben, wenn ich aber die Bilder als Backup auf dem Compi lade, geht mir ein Bild verloren.
Dem Compi ist es egal ob ich Bild, bild BiLd oder bILd schreibe, es ist ein Name.
Und mir sind bei einem Serverwechsel schon einige verloren gegangen.
Mach einem User mal begreiflich das er die Bilder umbenennen soll :cry:
Ich wäre generell dafür, das Bilder beim hoch laden als MD5 umgewandelt wird (habe ich im Portal)
z.B. so d99729bcd27fd077d056e9b39.jpg
LG Harald
-
Ok schon klar was du meinst, aber seit der V1.7.10 werden erstens die Dateinamen in Kleinbuchstaben umgewandelt und wenn sie gleichlautend sind, wie schon geschrieben ein Zahl hinten drann gehängt.
Somit gibt es keine Probleme mit einen Betriebssystem oder nach einen Serverwechsel.
mfg Andi
-
@Vano, Is it possible to modify this MOD to just simply check for duplicate filename only? Best variant would be to check for duplicate file name and extension while uploading
Thanks!
This will be helpfull in sitivation where the file name does not change and a user uploads a same file with the same file name, instead of it being renamed and added, can this mod detect and warn that already a file with this name exists somewhere in database and need to proceed with the renaming or not!
-
Ok schon klar was du meinst, aber seit der V1.7.10 werden erstens die Dateinamen in Kleinbuchstaben umgewandelt und wenn sie gleichlautend sind, wie schon geschrieben ein Zahl hinten drann gehängt.
Somit gibt es keine Probleme mit einen Betriebssystem oder nach einen Serverwechsel.
mfg Andi
:oops: stand das wo, war mir nicht bewusst, also mein Posting vergessen :wink:
wünsche dir frohe Ostertage (allen anderen auch)
LG Harald
-
... stand das wo,....
glaube nicht, das wurde in der 1.7.10 gemacht als die Probleme mit den Umlauten bestand.
Wünsche dir auch noch Frohe Ostern
Andi