DaemonForums  

Go Back   DaemonForums > FreeBSD > FreeBSD General

FreeBSD General Other questions regarding FreeBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 26th July 2009
deadeyes deadeyes is offline
Port Guard
 
Join Date: Jun 2008
Posts: 19
Thanked 0 Times in 0 Posts
Default Find command (Linux options?)

Hi all,

I am trying to do the following:
I want to have a list of all files in a directory and its subdirectories with a md5 hash in its first column.

This is to get a list off all mp3s which I have double.

I though to use
find /musiclocation -printf %f
or something similar (which does not show the path)

But it seems that the command syntax is different.

Is there another way I can accomplish this? or is it possible to install a port that has the linux find command?

Maybe someone knows another approach to this.

Thanks in advance.
Reply With Quote
  #2   (View Single Post)  
Old 26th July 2009
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,640
Thanked 214 Times in 189 Posts
Default

Perhaps this will be sufficient?
Code:
$ mkdir test
$ cd test
$ touch a b c
$ find . -type f -exec md5 {} > /tmp/results \;
$ cat /tmp/results
MD5 (./a) = d41d8cd98f00b204e9800998ecf8427e
MD5 (./b) = d41d8cd98f00b204e9800998ecf8427e
MD5 (./c) = d41d8cd98f00b204e9800998ecf8427e
$
Reply With Quote
  #3   (View Single Post)  
Old 27th July 2009
deadeyes deadeyes is offline
Port Guard
 
Join Date: Jun 2008
Posts: 19
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by jggimi View Post
Perhaps this will be sufficient?
Code:
$ mkdir test
$ cd test
$ touch a b c
$ find . -type f -exec md5 {} > /tmp/results \;
$ cat /tmp/results
MD5 (./a) = d41d8cd98f00b204e9800998ecf8427e
MD5 (./b) = d41d8cd98f00b204e9800998ecf8427e
MD5 (./c) = d41d8cd98f00b204e9800998ecf8427e
$
This shows the (relative) path... and that is not what I want.

Thanks for your time though.
Reply With Quote
  #4   (View Single Post)  
Old 27th July 2009
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,640
Thanked 214 Times in 189 Posts
Default

Quote:
Originally Posted by deadeyes View Post
This shows the (relative) path..
Because that particular find(1) command used a relative directory: "find .".

If you need a full path, use find with /path/to/your/files instead. If you don't want any path information, run the output through awk or perl or python and remove it.
Reply With Quote
  #5   (View Single Post)  
Old 27th July 2009
vermaden's Avatar
vermaden vermaden is offline
Administrator
 
Join Date: Apr 2008
Location: pl_PL.lodz
Posts: 1,052
Thanked 118 Times in 93 Posts
Default

@deadeyes


Code:
~ % cat find.sh
#! /bin/sh

find ${1} -type f \
  | while read LINE
    do
      MD5=$( md5 -q ${LINE} )
      FILE=$( basename ${LINE} )
      echo "${MD5} ${FILE}"
    done

~ % ./find.sh misc/sys/FreeBSD/
3d3991e7f828003c3abe0c7f1e48011a loader.conf
66ac143dc9e53b22df4fd77ae6fa01fa make.conf
01963f0ccdd17822c5ee210bba5df10f xorg.conf
085616e4fc95e7af3771deceab5171c3 sysctl.conf
(...)
~ % 
__________________
religions, worst damnation of mankind
"If 386BSD had been available when I started on Linux, Linux would probably never had happened." Linus Torvalds

Linux is not UNIX! Face it! It is not an insult. It is fact: GNU is a recursive acronym for “GNU's Not UNIX”.
vermaden's: links resources deviantart spreadbsd
Reply With Quote
  #6   (View Single Post)  
Old 27th July 2009
deadeyes deadeyes is offline
Port Guard
 
Join Date: Jun 2008
Posts: 19
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by jggimi View Post
Because that particular find(1) command used a relative directory: "find .".

If you need a full path, use find with /path/to/your/files instead. If you don't want any path information, run the output through awk or perl or python and remove it.
It is just that the linux find has this option available.
So scripting is not necessary.
Python, Perl does make more dependencies.

vermaden: nice piece of shell script (and purely bash ) didn't know the command basename

Thanks for your efforts guys!
Reply With Quote
  #7   (View Single Post)  
Old 27th July 2009
BSDfan666 BSDfan666 is offline
Real Name: N/A, this is the interweb.
Helpful companion
 
Join Date: Apr 2008
Location: Ontario, Canada
Posts: 2,223
Thanked 193 Times in 184 Posts
Default

Quote:
Originally Posted by deadeyes View Post
It is just that the linux find has this option available.
So scripting is not necessary.
Python, Perl does make more dependencies.

vermaden: nice piece of shell script (and purely bash ) didn't know the command basename

Thanks for your efforts guys!
No.. it's purely bourne-compatible.
Reply With Quote
  #8   (View Single Post)  
Old 28th July 2009
vermaden's Avatar
vermaden vermaden is offline
Administrator
 
Join Date: Apr 2008
Location: pl_PL.lodz
Posts: 1,052
Thanked 118 Times in 93 Posts
Default

Quote:
Originally Posted by deadeyes View Post
It is just that the linux find has this option available.
So scripting is not necessary.
Code:
# pkg_add -r findutils
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-7.2-release/Latest/findutils.tbz... Done.
~ % pkg_info -L -x findutils | grep bin
/usr/local/bin/gfind
/usr/local/bin/goldfind
/usr/local/bin/glocate
/usr/local/bin/gupdatedb
/usr/local/bin/gxargs
~ % rehash
~ % gfind --version
find (GNU findutils) 4.4.0
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Eric B. Decker, James Youngman, and Kevin Dalley.
Built using GNU gnulib version e5573b1bad88bfabcda181b9e0125fb0c52b7d3b
Features enabled: D_TYPE O_NOFOLLOW(enabled) LEAF_OPTIMISATION FTS() CBO(level=0) 
~ %
Quote:
Originally Posted by deadeyes View Post
(and purely bash )
Which part of it works only in bash(1) while it works without any problem on POSIX sh(1) (first line indicates parser as /bin/sh ... not bash(1))
__________________
religions, worst damnation of mankind
"If 386BSD had been available when I started on Linux, Linux would probably never had happened." Linus Torvalds

Linux is not UNIX! Face it! It is not an insult. It is fact: GNU is a recursive acronym for “GNU's Not UNIX”.
vermaden's: links resources deviantart spreadbsd
Reply With Quote
  #9   (View Single Post)  
Old 30th July 2009
s0xxx's Avatar
s0xxx s0xxx is offline
Package Pilot
 
Join Date: May 2008
Posts: 194
Thanked 47 Times in 22 Posts
Default

Hm, I don't know what's wrong with suggestion jggimy made? Because the name doesn't come before the hash? Then just:
Code:
$ find . -type f -exec md5 {} + | awk '{ print $4, $2 }'
IMHO, path to file is usefull if you got several directories containing mp3's.
Also, using awk you could filter the duplicates, using something like this:
Code:
$ find . -type f -exec md5 {} + | awk -f find.awk


find.awk:

a[$4]++ { if(match($2, /[^\/]*\)/)) {
           pattern=substr($2, RSTART, RLENGTH-1);
               printf("%s\t%s\n", $4, pattern);
           }
}
I'm sorry for not providing more command output or examples, I'm doing this on a Windows machine.
__________________
The best way to learn UNIX is to play with it, and the harder you play, the more you learn.
If you play hard enough, you'll break something for sure, and having to fix a badly broken system is arguably the fastest way of all to learn. -Michael Lucas, AbsoluteBSD

Last edited by s0xxx; 30th July 2009 at 01:51 PM.
Reply With Quote
Old 30th July 2009
s0xxx's Avatar
s0xxx s0xxx is offline
Package Pilot
 
Join Date: May 2008
Posts: 194
Thanked 47 Times in 22 Posts
Default

But if you have spaces in names of songs that may disturb the regex, so maybe this is better:
Code:
find.awk:


a[$4]++ { 
        regex="\\/[^\\/]*\\)"
        if(match($2, regex)) {
           pattern=substr($2, RSTART+1, RLENGTH-2);
             printf("%s\t%s\n", $4, pattern);
          }
}
__________________
The best way to learn UNIX is to play with it, and the harder you play, the more you learn.
If you play hard enough, you'll break something for sure, and having to fix a badly broken system is arguably the fastest way of all to learn. -Michael Lucas, AbsoluteBSD
Reply With Quote
Old 31st July 2009
BSDKaffee's Avatar
BSDKaffee BSDKaffee is offline
Real Name: Jason Hale
Coffee Addict
 
Join Date: May 2008
Location: Wintersville, Ohio
Posts: 212
Thanked 36 Times in 34 Posts
Default

Quote:
Originally Posted by deadeyes View Post
This is to get a list off all mp3s which I have double.
If you just want to check and see if you have exact duplicate files, I would suggest using something like sysutils/duff. It is very fast, easy to use, and written entirely in C with no other dependencies.
Reply With Quote
Old 23rd August 2009
deadeyes deadeyes is offline
Port Guard
 
Join Date: Jun 2008
Posts: 19
Thanked 0 Times in 0 Posts
Default

I knew there were some tools already for doing this but I wanted to write a script myself (as an exercise)

Thanks for you help guys. Eventually the basename command would do the trick but I kept it out of the script as it would not very usefull if you don't really know where those duplicate files are.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to get port's building options? Sunsawe FreeBSD Ports and Packages 14 9th May 2009 06:35 PM
portupgrade -af, how to submit fetch options? bsdfan FreeBSD Ports and Packages 4 28th December 2008 09:05 PM
Command to find and replace, but not creating a new file 18Googol2 Programming 4 22nd September 2008 10:28 PM
Change Makefile options in ports shep FreeBSD Ports and Packages 5 18th August 2008 07:58 AM
Buildworld make.conf options siffland FreeBSD Installation and Upgrading 4 12th May 2008 12:02 AM


All times are GMT. The time now is 07:56 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick