Finding and Removing Duplicate Files from Your Digital Photo Library

I needed to sort through a library of 150,000 digital photos that was taking up 150GB of disk space. I knew most were duplicates. (Poor file management practices!)

I needed a tool that would help me find the duplicates so I could delete them.

The problem was, the file were frequently renamed–so two files that were identical often had different file names. I needed a tool that would be able to recognize duplicate digital photos even if the names were different.

I found an awesome Windows program called DoubleKiller that can do just that.

DoubleKiller allows you to find duplicate files by many methods. The method that DoubleKiller offered that worked to solve my problem (duplicate digital photos with differing filenames) was to search the folders I stored my photos in and flag all files that had the same file size AND the same CRC32 signature.

(A CRC32 signature is like a unique serial number that can be calculated from a file. It’s not really unique, but the odds of two files of the same size having the same CRC32 is about 1 in 16,000,000 so it’s a pretty good indicator that the files are identical.)

One you set up the search parameters, tell DoubleKiller what folders to search in and start to run the program, it lists all the duplicate files it finds.

You can then select and delete the duplicates you want to remove.

One suggestion: DoubleKiller has a one-click feature that allows you to select all the duplicates but the first one or all the duplicates but the last one. The result set is ordered in the same order as the folders are entered in the Folders selection section of the Options tab. So when you add folders to search in the Options tab, put the folders where you want to keep files in the order of the priority that you want to keep the files. (Wow. That does not even sound clear to me–but I hope it makes sense once you play with DoubleKiller.)

It took a long time to process 150,000 files (several hours) but I did not have to waste days–or even weeks–doing it by hand and I can be sure I did not accidentally delete every copy destroying valuable work.

And now I have an extra 150GB of disk space. Awesome!

Hurray for DoubleKiller! I wish I have found this gem a long time ago!

Now to tidy up my MP3 collection…

Oh–one more thing, there is also a Pro version of DoubleKiller that runs US$ 19.95. It’s faster and offers more automation options. I think I might have to pick up a copy!

Trek Vapor 3 Helmet Liners and a Tale of Poor Customer Service

Here’s the tale:

I bought a Trek Vapor 3 helmet last year. After using it for a few months, I went to wash the helmet liners because they had accumulated a lot of sweat and grime–as will any piece of cloth that is on athletic gear that comes into contact with skin. The helmet liners fell apart–they must have been held together by water soluble glue–rendering the helmet useless. I complained to Trek customer service and, instead of fixing the problem, they just made me mad.

The Disintegrating Trek Vapor 3 Helmet Liners–After One Wash

My original email to Trek customer service:

I have to say that I am very disappointed in the Vapor 3 helmet I purchased from Trek.

After several months of use, I took out the removable pads inside the helmet to wash the grime and oil out of them. They detach by velcro, so I assumed they were made to be removable and cleanable.

To my surprise, when I washed the pads, the cloth on either side of the foam fell off.

How was the cloth attached to the foam? With water-soluble glue? However you attached it–you made the wrong choice. A helmet liner gets sweaty and needs to be cleaned. Any part of helmet that gets sweaty, should be washable without it falling apart.

This is a complete and utter disappointment. The helmet is now useless without the helmet liner. It was a total waste of money.

I am requesting you send me a new set of helmet pads/ liner to replace the defective ones you shipped with the helmet.

On second thought, better send about 10 sets in case the new ones dissolve when I get them sweaty.

My mailing address is:

John Berns
xxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxx
Din Daeng, Bangkok 10400
Thailand

Trek Vapor 3 Helmet – With Disolving Helmet Liners

To which they replied:

Thanks for writing and riding Trek bikes. Trek warrants each helmet for a period of one year from the date of purchase against manufacturer’s defects in materials and craftsmanship. We also offer a one year crash replacement policy. To exchange your helmet, please send it along with your original sales receipt to:

Trek Bicycle Corporation
Attn: Helmet Crash Replacement
801 W. Madison Street
Waterloo, WI 53594

You can also take your helmet, along with your original sales receipt, to your local Trek dealer and exchange it there.

Well, I think they could have done more. After all, the product should never have been put on the shelves. Why should I have to PROVE that there is a problem with a product they must know has a serious problem?

Here is my rather long and angry response:

That’s a rather ridiculous offer and it angers and offends me that you even suggested that.

To start with, let me tell you why you should care that I am upset.

To begin with, the Vapor 3 is a defective product by design. You should already know that fact and be willing to take a customer at his or her word and fix the problem–a customer that is sold a defective product should not have to make a concerted effort to convince the company that made the product to make right.

Furthermore I am a long-term Trek customer. I have owned 4 Trek bicycles and currently own two bikes; a road bike and hardtail mt. bike. I have a Trek Vapor 3 helmet (that is now unusable), Trek gloves and other assorted Trek gear. I am a customer that spends money buying Trek products. That’s a probably the best reason a company can have to care.

Here’s the problem:

You guys made a seriously flawed product. Athletic headgear that is made to be in contact with a sweaty part of the body and that falls apart when you wash it is just plain flawed. Somebody in product design or manufacturing really screwed up on this product. It never should have been shipped. Period.

Now, you are telling me I should jump through hoops and send you the helmet so you can see the problem for yourself before you fix the problem? That’s not just wrong, it’s stupid.

Why your request is impractical to the point of being ridiculous:

I live in Thailand; you should already know that if you ready my complaint, I put my address in there. Do you know where Thailand is? It’s half-way around the world from the address you gave me to return the products to. Now, I could send you the helmet by air courier–that would cost more than the price of the helmet. I could send you the helmet by surface mail, but that’s still probably half the price of the helmet and it would take months. My head is too valuable to go unprotected in Bangkok traffic for months. And why should I shoulder the costs of fixing a product that was flawed to begin with? I am already pissed that I got a crap product–why insult me more and tell me I should waste more time and money proving to you that the product is flawed? I mean–you guys must know this is a problem already, don’t you? Shipping you the helmet requires a great deal of effort on my part, requiring time and effort to package the helmet and take to a shipper. More of my time than the helmet is worth, I am afraid. Hell, the time I spent writing the original email was time wasted that I should not have had to waste. This email is even more time wasted.

Take to the bike shop I bought it from? It’s impractical, it’s rather far and the time and expense to get there is more than the helmet is worth.

Besides–I have no clue where the receipt is.

What do you think this is? A shady ploy to bilk you out of a set of helmet pad / liners? Sorry, I have more important things to do than to try and swindle Trek out of $0.59 worth of helmet liners.

But if you are still convinced this is a shady scam, see the attached photo: the liners are really falling apart. There, now that I wasted 5 minutes snapping a photo will you believe I am not trying to scam you out of fifty nine cents worth of helmet liners?

My suggestion, that you just ship me a few of the helmet pad / liners was a pretty simple solution to the problem. They weigh about 5 grams and it would have taken somebody 5 minutes to put it in an envelope and email it to my address. Problem solved. Customer happy. And it would have required less effort or expense then you receiving a damaged helmet from me and re-shipping it back.

It would require a little initiative on your behalf and a company that supports service reps that are willing to take some initiative to satisfy an unhappy customer. Well, obviously this is not the case.

Now you have a customer (me) that has a useless product (the helmet) and a very, very poor image of Trek Bikes.

With a little effort this situation could have been avoided. Instead, either through wrong-minded corporate policy or general laziness, you squandered an opportunity to build lasting good will and a give a good customer had one more reason to stay loyal to your brand.

Oh–and you would not have had a customer telling as many other people that he can that he thinks your products and customer service are sub-par.

http://www.johnberns.com/2008/04/16/trek-vapor-3-helmet-liners/

– A Former Trek Customer

That’s this blog post.

I don’t understand most company’s policies on making good on customer problems. They often force the customer to go to great lengths to achieve recourse which, at least in my case, further aggravates me. Not only did I get a crap product, I get a run-around to remedy the situation. Now I have a crap product and a company that I feel is jerking me around. Pffft. Why bother? That’s the last time I will buy their product.

Bye Bye Trek! It’s Been Nice Buyin’ Your Products–Until Now!

Now, it a company hears my complaint and takes immediate action (and I even told Trek what they could do and the cost of it is mere pennies by my estimate), they could have made me very happy with almost zero effort on their part.

But alas, they just alienated me.

Facebook Sez: Party Tonight, Hangover Tomorrow!

Analyzing the data embedded in social networks is always a blast.

The Facebook Lexicon allows you to graph the relative occurrence of words posted to The Wall by date. Even more fun, you can graph more then one term at a time.

So, what happens when you graph the occurrences of “party tonight” and “hangover?”

It’s not often that a graph makes me laugh.

Stuff Your Eyes With Wonder

Image by guppiecat

“Stuff your eyes with wonder … live as if you’d drop dead in ten seconds. See the world. It’s more fantastic than any dream made or paid for in factories. Ask no guarantees, ask for no security, there never was such an animal. And if there were, it would be related to the great sloth which hangs upside down in a tree all day every day, sleeping its life away. To hell with that … shake the tree and knock the great sloth down on his ass!”

Ray Bradbury – Fahrenheit 451

Lonely Planet’s Bad Trip

A bit of a media storm is brewing over insider revelations by ex-Lonely Planet travel writer Thomas Kohnstamm.

Kohnstamm’s soon-to-be-published book “Do Travel Writers Go to Hell?” claims to blow the cover off the travel guide book industry. He claims that much of the content he wrote was plagiarized or fraudulent because the low pay and short deadlines made it simply impossible to do the work ethically.

Lonely Planet has posted a rebuttal on their website.

In Lonely Planet’s Thorn Tree travel forums, there is quite an outrage brewing. The outrage is partially over the content of Kohnstamm’s article, partially over a perception that LP and their parent company (The BBC) has quashed some of the postings on the topic in the South America forum.

Several blogs have written damning criticism of Kohnstamm including “5 reasons to be outraged by the Lonely Planet fraud” at Gadling and one article by Eva Holland predicting that hell will be the destination for Kohnstamm at Brave New Traveler.

But some travel writers, like David Stanley, stand alongside Kohnstamm in saying that the wages to guidebook writers can’t possibly fund the hands-on research that the guidebooks say they provide.

“To save money and maintain full control, Lonely Planet often assigns inexperienced office clerks and interns to update their guides. Little wonder that some of these underpaid novices resort to plagiarism. My books have been copied by Lonely Planet writers time and again … Today Lonely Planet updaters get no royalties and must sign away all rights, even moral rights. Thus it’s no surprise that the quality of Lonely Planet guides is so uneven.”

Here are my thoughts.

Kohnstamm has written or contributed to 12 books. After one or two he would have known that, if, as he claims, the work was impossible to do for the budget allocated, then he could have walked away. He chose to stay and write more–nobody forced him to do it.

He freely admits that he has passed off work that contains fabrications in the past–so any claims he makes now–especially ones that line his pocket (by promoting his tell-all book) should be taken with a great deal of skepticism.

I know several guidebook writers. The ones I know personally are solid people and their work is outstanding. I travel in the places they write about and I have used their books. I have never been disappointed or mislead.

I have never had a really bad experience with LP guidebooks–other than some prices and info being out of date–but that’s a dead-tree publishing issue, not just guidebooks or Lonely Planet.

(By dead-tree publishing issue, I mean that, by the time you print something on a dead tree and distribute it, the information printed on the paper is no longer current. Say you buy a guidebook the day it hits the shelves of your local bookstore; the information contained in it is 6-24 months old (that’s how long it took the writer to compile his notes, the editor to edit it, the printer to print it, the shipper to ship it, etc.); if the book has a 2 year shelf life–the information is quite dated.)

On the flip side, I do know that, in spite of LP’s claim that “we lead the industry in the fees we pay, and are committed to a yearly review of author fees,” the fees to guidebook writers in general have been going down–and the cost of living and traveling has been going up. Lower wages mean that some of the best people will walk away from the work.

I know the time it takes to do the field research (I have done it myself) and I know the volume of research required and the ballpark of the fees paid and Kohnstamm’s assertion that it’s not enough to do thorough research rings quite true.

One thing is for sure: Kohnstamm’s claims will have reverberations in the industry. Guidebooks will have to scrutinize who they hire and how their material is compiled. It will also make travelers think a bit more critically about the books they read.

Having had travel guides on the brain for the better part of 10 years now, I see there are some real problems with the travel content industry in general. The volume of information that must be researched, sifted and sorted is gargantuan. It takes a small army to do that kind of work. Communities can form that sort of small army–but then you run into the problems of credibility and reputation as well. (Without authorship that is credible and reputable you can nver achieve authority–ad guidebooks are nothing without some measure of authority-that’s why this scandal has so much potential to damage Lonely Planet.)

My burning question is: how do you utilize community and still be able to surmount the issues of credibility and reputability of the contributors so you retain a level of quality and authority?

Thai MICT: If You Can’t Beat ‘Em, Hack ‘Em!

The folks over at the Ministry of Information and Computer Technology here in Thailand just keep up the great work.

What can they do to top their random and ridiculous website block lists?

Well, they can hack websites they deem offensive to the Thai character:

The Information and Communications Technology Ministry is to ‘hack and crack’ foreign websites deemed offensive to Thailand’s revered institutions. A March 15 report in Krungthep Turakij newspaper (www.bangkokbiznews.com) quoted a source at the ICT that the ministry could pursue legal proceedings only with websites registered in Thailand, and is now planning a ‘hack and crack’ programme to hack offensive websites hosted abroad and delete their contents, because the legal process would take too long.

Wow. What a twisted mindset.

Via the good folks at Freedom Against Internet Censorship Thailand.

Secure Password Management with Password Safe

I have a lot of passwords. Hundreds. I need some place to store them. But that storage system has to be secure. Having a spreadsheet or text document of all your passwords stored on your computer–or a sheet tucked in your wallet–is an invitation to disaster. If you computer is hacked or your list is found by somebody, if it is NOT ENCRYPTED they have the keys to your personal (online) kingdom and they can reek all manner of havoc.

I have been using Password Safe, a free, open source password manager that stores all my passwords in an encrypted, password protected file. (Yeah–you have to remember that one password, but everybody can remember one password, can’t they?)

Password Safe was created by Bruce Schneier–the guy that, literally, wrote the book on cryptography. The Java version of Password Safe is cross-platform, so you can use it on Linux, Mac and PC. It allows you to store and manage all your passwords from one place. The data files are small and securely encrypted, so you can back them up to a USB drive, send them to your Gmail account or store them wherever and not worry that somebody will open the data file and steal all your passwords. (Assuming, of course, that you have created a good, secure password for the data file itself.)

As more and more of our lives is transacted online, keeping our identity, our bank accounts and our information private and secure becomes more and more important.

Use good passwords and keep them stored in a secure manner. Please.

Are You Tweeting About Me?

Do you have the feeling that people you are not following on Twitter are talking about you or even trying to @message you but the messages are not reaching you because you are not following them?

Do you even care?

Well, if you do care, here is a simple solution to seeing who’s tweeting about you. Use the Twitter Track feature and track your user name! On Twitter I am jfxberns so I would use the following command to track my user name:

track jfxberns

Now, any time somebody’s tweet has jfxberns in it, I see it on my Twitter client. Pretty handy!

If you want to see who’s tweeting about you–just track yourself.

track yourusername

Gradual Engagement: Build Engagement, Not Barriers

I love A List Apart; it’s certainly one of the best websites out there on website design. We are not talking just web design as in “just cool graphics,” but big picture, user-centered design philosophy kinda stuff, presented in a way you can use in practical every day scenarios.

I especially liked their recent article Sign Up Forms Must Die:

I’ll just come out and say this: sign-up forms must die. In the introduction to this book I described the process of stumbling upon or being recommended to a web service. You arrive eager to dive in and start engaging and what’s the first thing that greets you? A form.

Photo by Tuis

Don’t you hate those in-your-face forms? Don’t you hate barriers? I do.

I strongly believe that engaging a user in your site is the main goal of most websites. Registrations are a nice metric and marketers, accountants and other people that have to show reports and be accountable love to show registration numbers. But it might not be the best strategy to build a strong user base.

Throwing a barrier in front of a potential user before they see the value in your site is a lot like asking a customer in a car showroom to sign a sales contract before he or she has test driven a car. WHY would anybody be interested in committing BEFORE they see the value?

But web architects still continue to throw registration forms in the users face well before the user has had a chance to see the value a website / web service provides. (See my recent rant about Kodak Easyshare.)

Better to lure them in, reveal value a piece at a time and make registration part of the unfolding process. Quid pro quo: I show you some value, you give me a little information, OK? It keeps the users interested and \motivated to take the next gradual step of engagement.

Unless of course, you like registration forms.

I love the article. Well written, excellent examples. If you design websites, do read it, please!

Drupal Modules: Rank, Rate, Review at DrupalModules.com

I usually don’t get all gooey and gushy when I write up a website, but this is one I like a lot: DrupalModules.com built by John Forsythe.

DrupalModules.com does a great job of taking the list of Drupal Module (which was nothing more than a raw information dump on the Drupal site) and wrapping it with a smattering of practical community features and thereby turning it into an extremely useful resource for Drupal developers.

It’s not only the site itself that shines, the people that use it are doing a bang-up job with the information they are contributing. The reviews are generally well written, they give examples of how people have used the modules in real life to solve problems and they link to other off site resources (examples of the module in action, tutorials) that help clarify what the module can do and how it does it.

In comparison, Drupal.org lists the modules and has a short blurb–usually written by the maintainer who, even though he knows the module the best, often does a terrible job explaining the modules capabilities and uses. Lots of data, not much information.

DrupalModules.com is a big step forward.

By the way, John has also developed a nifty search engine for sifting through the Drupal code base appropriately called DrupalCodeSearch.com.