Programmer: Image Utility Required

[Carlos Camacho]

I'm wondering if someone can whip up a small app for me? This is what I need...

I have a few CDs with images of textures. The other day I was checking my images and found that 5% of them were not saved correctly, so the images are corrupt. For example, if an image is loading, and you save it to to your HD, before it finishes loading.

If I try to use Preview, it reports that the image is bad. Graphic Converter also reports that it is bad, but still opens it. You can of course see the bottom of the image that is corrupted.

Testing each image would take too long. My idea is for a small utility to do the following:

(1) Use QuickTime to display the image (no need to display it to the screen actually, so maybe a toggle to "display" or not)
(2) Do an error check, if QuickTime can't display the image, then chances are, the image is corrupted.
(3) Have a button to select the source folder (and sub-folders option?)
(4) Have a button to select a destination folder
(5) If the image is corrupt, move it to the destination folder, if the image is fine, leave it alone.

This would be a wicked good time saver, and maybe a handy utility to release on versiontracker.com, etc.

Is there anyone that can make this app? Xcode, RB, etc., I don't care. As long as it does the job, and is fairly quick.

If you need more info, let me know!

Cheers,

[Malarkey]

Well, I've been doing a lot of Cocoa programming lately so if no one else steps up to this, I can try my hand at it.

[junyx]

Wow, 30 minutes with my brand new uDG-prize RB Pro! :cool:
download here (http://www.bluepixysw.com/carlos.zip)


select source folder
select dest folder
click run


It will explore all sub-directories opening all files, so don't forget to take somehing to drink before doing the job :p

It will (or should :p) copy to dest folder the files that can't be opened as image by QuickTime.
Note: it will not copy the file if another file with the same name is already in the dest directory. Is this a problem?
If you found something to fix, I think I can work a bit on it tomorrow, so drop me a line.

Cheers,
Ludovico

[Carlos Camacho]

Thanks so much!

* Does it copy or move the file?
* I think it is best to rename the file if another file in the destination folder has the same name. Perhaps add a 1 to the file name, before the .jpg or after the .jpg part. Add 2 if file texture1.jpg already exists. Any other better idea is welcomed. But for sure, some files might have the same name, so this will be a problem.

I tested it on a folder, and thus far it found some links that I had saved to the image folder (texture sites) which I didn't need. So already, it is helping me out!

I assume that because you use QuickTime, your app supports gif, png, etc?

Opps! The app just quit on me as I was typing this.

Anyone else care to try this "challenge" out. I'd like to see some of you Cocoa programmers give it a go.

Cheers

[Malarkey]

Working on one. Give me a couple days, though, since my full-time day job is programming. :)

[Carlos Camacho]

Excellent, thanks!

junyx, good to see you make use of your uDG prize. (Music to my ears.)
You might need to do a bit more testing as the app seemed to quit sometimes on its own.

For the GUI, I think it would be cleaner if you had one window plane, and three buttons on it. "Source", "Target", "Run". Print the folder paths to the left or right of the buttons.
Maybe a progress bar, or the name of the file it is currently examining would be neat.

It seems that I have "links" and "text files" in many of my directories. I need to get rid of them too. Thus, a filter would be nice to have. For example, Checkboxes next to "Ignore non-images." So your program would first check if the file is a non-image, depending on the checkbox, it would either move it to the target or skip over it.

Anyhow, I appreciate anyways help!

Cheers,

[flipflop]

Not to step on any toes but I went ahead and started a solution for this as well. Not quite fully functioning yet but it has most of the features.

A few notes:
- No threads are in place yet. So if you run through a large directory of images the application will show a nice spinning cursor and not respond to any mouse clicks until it's done.
- The stop button does nothing
- The status bar does nothing
- Files are copied to the destination folder but not removed. Just in case there are bugs. (Wouldn't surprise me).
- The image is currently drawn in the window, no toggle switch to turn it off.

Thoughts, suggestions and bug reports are welcome!

Download "ImageBrowser" (yes, I know... stupid name) here:
Download here (http://homepage.mac.com/WebObjects/FileSharing.woa/wa/default?user=jnikolai&templatefn=FileSharing18.html&xmlfn=TKDocument.18.xml&sitefn=RootSite.xml&aff=consumer&cty=US&lang=en)

Malarkey, please continue working on yours. This small project just gave me an excuse to do some Cocoa programming. It has been a while... :)

Thanks!
- John

[Carlos Camacho]

Hey, the more the better. It is good to see how different people attack the same problem.. using their tools of choice. I would love to see the community create more small and usefull utilities -- especially ones to help in game dev.

>The stop button does nothing
:D

>Files are copied to the destination folder but not removed
Ah, that might be a problem for me because I need to get rid of the corrupted files from the folders that SHOULD have good images. How about a toggle to switch between COPY mode and MOVE mode?

>The image is currently drawn in the window
Do you find that displaying images slows the process down?

ImageBrowser --> ImageChecker ;)

I was wondering, if you don't want to MOVE the file, how about this. Once the process is done, have a window with a list of all the files that were copied. Next to each file name, a button to view and delete the original image. That would work for me. :)

Thanks!

[Malarkey]

Bad Image Begone (http://web.newsguy.com/malarkey/software/Bad_Image_Begone.zip)

Whew, finally whipped up my version of this tool. Some things to note:
There's a checkbox for processing QuickTime types only which include at least jpg, gif, png, bmp (there's something like 11 I put in there). Next version will include a table so you can edit the extension list to include mp3, mov, whatever else that QuickTime can open.

Drag-and-dropping onto the application icon is supported for files and folders; however, I suggest running the program first and setting a destination directory because I just realized I forgot to account for that. :blush:

Error checking is pretty minimal for this version so keep a copy handy of whatever directories you decide to process.

Hm, I think that's about it. There's some more things I want to add (like some sort of progress display) but I'll do that tomorrow or the day after. It's bed time for me. :zzz:

[Malarkey]

Malarkey, please continue working on yours. This small project just gave me an excuse to do some Cocoa programming. It has been a while... :)


No worries. :) This is like the time Carlos decided to spontaneously do 72-hour programming contests.

[junyx]

I'm still working on it but in the next days I'll very busy with my exams at university, so I'm pretty sure that another person will complete the task before me :p

What's new:

choose between copy and move
rename with name_1.ext double files


It worked with my tests, but It has still some bugs. I'm a newbie with RB :p

Ooops! I'm at university now and I can't use ftp here to upload :rolleyes: I'll post it tomorrow, maybe with the code.

[flipflop]

ImageBrowser --> ImageChecker Pure genius! :p

Off to try and figure out threading... Whee!

[Carlos Camacho]

>Off to try and figure out threading... Whee!
Yeah, you need that! I set the app to go last night, and when to sleep.

My quick check of the destination folder showed it found about 240 files. I tested them to see if they were indeed corrupt. 5 of the 6 were. 1 though seemed fine.

Keep at it boys!

Cheers,

[flipflop]

Could you please zip up the corrupt images as well as the one that was picked out as being corrupt but was not actually corrupt so that we can have some thing to test? Please? :D

Currently I went into one of my image files with vi and chopped off the first few lines. It's now my corrupt image file for testing. It works but obviously didn't catch all the cases. ;)

A technical question for anyone in the know - The threading code is chugging along slowly. I've been playing with NSThread as well as NSNotification for notifying the UI thread when to update the status bar. Is this the "proper cocoa" way to do this? Or is NSNotification and NSNotificationCenter overkill or not being used properly in this situation?

Thanks!
- John

[Carlos Camacho]

Could you please zip up the corrupt images as well as the one that was picked out as being corrupt but was not actually corrupt so that we can have some thing to test? Please?Yep.

[Carlos Camacho]

I emailed you 3 images.

By the way. If anyone needs a corrupted image for testing, simple do the following. Get an image that you know to be OK, open it up in a text editor. Delete a bunch of the text at the end.

Try to open in Preview... it will report that it can't open. Try with Graphic Converter, and it will say bad file, but still let you open it. Look at the bottom of the image and you will see that there is data missing/bad.

Cheers,

[flipflop]

Thanks for the images. Interesting. With my box running 10.3.8, Preview is able to open parque28.jpg which appears to be the one file with some missing information at the bottom of the image. Are you saying on your box this file is reported to be corrupt by both Preview and Graphic Converter? :???:

Versions ->
OS: 10.3.8
QuickTime: 6.5.2
Preview: 2.1

Thanks!

[Carlos Camacho]

Sorry, I didn't check to see if all three were reported as corrupt, I sent those three as I noticed they have missing data at the bottom.

By the way, not that I want to weasel "more" from you than what I asked for.... ;)

Once your app is running well, could you think about the following.

How to compare to images, and if they are the same, allow the user to send one of them to the target folder? The reason why I mention this is that I noticed that I have duplicate textures. In some cases, the images are exactly the same (different names), but in some cases, one image is a low-resolution version. I'll like to delete dups and low-resolution versions. Another example is the images are the same, but one was saved with heavy compression, so I want to ditch the lower quality version.

For this task, I use a Classic only app called PICTCompare. The dev released a Carbon version but it isn't stable. I have given up on him releasing a newer version.

What his app does is very much like what everyone's submitted utility to me does, but it also compares images. I think he generates a thumbnail for every image at a certain size. Then he compares the thumbnails using some algorithm . There is a value which allows you to increase the ability to find dups over slower speeds.

I'm not sure what type of algorithms he uses. I'm sure a Google can turn up several type of methods. It may be that using QuickTime and some bit of coding could be enough to catch 60% of dups. Not sure.

Well, keep this in the back of your mind. For now, I am happy with a robust mini app to get rid of corrupted textures. I'll ponder the second problem later. :)

Cheers,

[flipflop]

OK, this is a bit odd... On a positive note I finally got a thread working! Who would have thought that talking between 2 threads would be so difficult to implement... :mad: Ah well, chalk it up as a learning experience.

My quicktime code is a little bit off though so I'm trying to get that working better now.

Carlos... I have been toying with the images you sent to me by opening them up in both QuickTime and Preview. Here is the run down of what happens:

QuickTime:
2parq03.jpg -> Fails to open and crashes QT
2parq20.jpg -> opens up just fine. Corrupt data at the bottom of the image.
parque28.jpg -> Opens up (you can view it) but QT notes the file may be corrupt.

Preview:
2parq03.jpg -> Fails to open (Preview displays error that file can't be opened)
2parq20.jpg -> Fails to open (Preview displays error that file can't be opened)
parque28.jpg -> Opens fine... corrupt data at the bottom of the image

The only thing I can think of at the moment is that QT is a Carbon based application while Preview is Cocoa? I figure Preview would use QT API's to display its images...

[flipflop]

New version up at my .mac account (http://homepage.mac.com/WebObjects/FileSharing.woa/wa/default?user=jnikolai&templatefn=FileSharing18.html&xmlfn=TKDocument.18.xml&sitefn=RootSite.xml&aff=consumer&cty=US&lang=en)

Simple directions - Choose your source and choose your destination directories. Hit the start button. The code will recursively run through all the files in the source directory moving any files that QT can not open to the destination directory.

New features:
- Threaded! Whoopie! The start and stop buttons work as they were intended to.
- New QT code in place. Should work better at picking out corrupt images (I hope). Thanks to OneSadCookie for making his QT source available (QT source (http://onesadcookie.is-a-geek.net/svn/repos/QTValuePak/)) and ThemsAllTook for some API clarification.
- Does a copy of the file to the destination folder. If the copy went OK than the file is removed from the source folder. A small safety feature just in case the copy fails. :p
- UI updated. I took out the movie view and added some text fields to show the status

Things to do:
- The code is a mess! Will need to be re-factored.
- Might be nice to have some kind of indicator when a corrupt file was found. Perhaps a counter or something...

Comments and feedback always welcome!

Thanks,
- John

[Carlos Camacho]

Great! Can't wait to get home and try it out!

[Carlos Camacho]

I ran the app in Mac OS X 10.3.7 iMac 800MHz G4.

Nothing happened. Didn't seem the app launch even though I did indeed double click it.

:(

[flipflop]

Darn! :wacko:

OK, I will check the compiler and see what I forgot to include. I can reproduce the problem on my laptop so the next release should solve this problem.

[flipflop]

OK, a new version is back up here (http://homepage.mac.com/WebObjects/FileSharing.woa/wa/default?user=jnikolai&templatefn=FileSharing18.html&xmlfn=TKDocument.18.xml&sitefn=RootSite.xml&aff=consumer&cty=US&lang=en)

Sorry about that... :p

[Carlos Camacho]

Thanks! I will give it a shot tonight.

[Carlos Camacho]

Last night I ran the app. It launched, and seemed to work smoothly. This morning I checked about 10 pictures in the Target Folder and they were all OK -- they were displayed with Preview without any error messages. I'm not sure if it caught any corrupted images. :(

[flipflop]

Well, I'm glad it ran smoothly. :blush:

I will see about saving a text document to the destination folder summarizing why the file was considered corrupt. There are several places the image checking could fail.

Did you try opening these image files up in QT? I have seen discrepancies between QT and Preview and how each handles a corrupt image file.

Could you please send me one or two of the images that the program complained about yet Preview opened up just fine?

How many image files are you running through? I tested the application against my iPhoto gallery (~1500 images) and it pulled out 3 corrupt image files.

[Carlos Camacho]

Did you try opening these image files up in QT? I have seen discrepancies between QT and Preview and how each handles a corrupt image file.
No, just Preview.

How many image files are you running through? I tested the application against my iPhoto gallery (~1500 images) and it pulled out 3 corrupt image files.
About the same.

Let me do some more checking.