DaemonForums  

Go Back   DaemonForums > Miscellaneous > Guides

Guides All Guides and HOWTO's.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 10th February 2014
Carpetsmoker's Avatar
Carpetsmoker Carpetsmoker is offline
Real Name: Martin
Old man from scene 24
 
Join Date: Apr 2008
Location: Eindhoven, Netherlands
Posts: 2,074
Thanked 198 Times in 156 Posts
Default Generate passwords from the commandline

I needed to generate a random password from a shell script, I figured that this was solved long ago, so I turned to teh interwebz to quickly copy/paste a working solution.
Inspecting the first few links that turned up, I noticed many of the proposed solutions are dubious at best.

The date ain’t random, buddy
The most obviously wrong are:

Code:
$ date +%s | sha256sum | base64 | head -c32
$ date | md5sum
$ ping -c 1 yahoo.com | md5 | head -c8
To paraphrase a quote from holy scripture The Hitchhikers guide to the Galaxy: “This is obviously some strange usage of the word ‘random’ that I hadn’t previously been aware of”.

Both SHA256 & MD5 also output in hex, so that would limit the total amount of characters to just 16, instead of 92.

tr means translate characters
Most of the other commands suffer from a dubious usage of tr(1).

tr(1) works on characters, not byte streams, /dev/urandom outputs a byte stream, not characters.
If your locale is set to (extended) ASCII or a variant thereof (ISO-8859-1, Windows-1252) this is more or less okay, since every byte is a character or escape code.

However, with UTF-8 or another multibyte character sets, it gets more complicated. Not every random byte stream is a valid set of UTF-8 characters, the chances of a random byte stream also being a valid UTF-8 character stream is quite small.

Yet, it seems to work on Linux with GNU tr. Why? Here’s a clue:

Code:
$ echo 'I løv€ π' | tr '[:lower:]' '[:upper:]' 
I LøV€ π

$ echo 'I løv€ π' | tr øπ€ X
I lXXvXXX XX
We would expect the ø and π to be uppercased, but they’re not, and the ø, π, and € getting replaced by 2 or 3 X’s?

The astute reader will have recognized what this means, GNU tr doesn’t handle multibyte characters, and always assumes an ASCII character set, which is somewhat disappointing, since it’s 2014, not 1974.

FreeBSD, for example, does this correctly, it also gives an error message on invalid UTF-8 sequences:

Code:
$ echo 'I løv€ π' | tr '[:lower:]' '[:upper:]'
I LØV€ Π

$ echo 'I løv€ π' | tr øπ€ X
I lXvX X

$ head -c5 /dev/urandom | tr X Y
tr: Illegal byte sequence

$ setenv LC_CTYPE C

$ head -c5 /dev/urandom | tr X Y
f��!�
The moral here is: byte streams are not character streams, don’t use ’em as such. It may work for now, but whenever someone adds multibyte support to GNU tr, your command will fail. It’s 2014, always assume multibyte by default.

Other problems
While I’m whining anyway…

Code:
$ openssl rand -base64 8 | md5 | head -c8
Using openssl rand is a good idea, but piping it to md5 isn’t. base64 gives me 64 characters, md5 gives me 16, making the password a lot easier to brute force. Also, 8 characters is too short, use at least 15.

Code:
$ curl -s http://sensiblepassword.com/?harder=1
Getting a random from the internet is spectacularly stupid & naive. Someone now knows:
  • A password you are using for some service or site
  • Unique personal details about you (IP address, browser/environment info)
I can now cross-reference the info with other data collected about you. For example, you once posted to a mailing list, your IP address is in the mail’s header, so we now have a password, name, and an email address.
I hope you can finish the scenario from here…

Just don’t do this. Ever. Randomly banging on the keyboard is a lot better.

Good solutions
Code:
$ head -c100 /dev/urandom | strings -n1 | tr -d '[:space:]' | head -c15
$ openssl rand -base64 15
$ gpg2 --armor --gen-random 1 15
The first solution could be considered slightly better, since it includes more characters (92 instead of 64). It also doesn’t require external tools (although openssl is almost always available these days).

Lessons
  • A byte streams is not the same thing as a character stream
  • Use strings(1) to convert a byte stream to a character stream
  • Don’t use the hex output of a hashing algorithm (SHA256, MD5)
  • Don’t trust copy/paste solutions from the internet; always think for yourself
  • BSD > GNU
__________________
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
Reply With Quote
  #2   (View Single Post)  
Old 10th February 2014
Carpetsmoker's Avatar
Carpetsmoker Carpetsmoker is offline
Real Name: Martin
Old man from scene 24
 
Join Date: Apr 2008
Location: Eindhoven, Netherlands
Posts: 2,074
Thanked 198 Times in 156 Posts
Default

j65nko wrote somthing similar a few years ago: Generating passwords with jot(1).

jot is that it's something of a BSD-ism, and not available on any other UNIX or Linux... I prefer to avoid it if I can.
__________________
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
Reply With Quote
  #3   (View Single Post)  
Old 10th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

Or you can use /dev/[u]random and hexdump(1).

Attached is a simple script I was working on recently to make WPA pre-shared keys or passphrases for wpa_supplicant. It could use a bit more investigation of a gray area, but not sure if I'll get around to it. The basic idea is there and since hexdump is in many unices, it should be very portable. You could use it to make other kinds of passwords, etc. BTW, the passphrase part of this assumes ASCII, so some may want to change that part depending on intended use.

The file upload for attachment keeps failing, athough data is transferred. Is it a forum problem or is it me?

Here's a link for the time being: randpass.sh.bz2

Last edited by IdOp; 10th February 2014 at 06:27 PM.
Reply With Quote
  #4   (View Single Post)  
Old 10th February 2014
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,710
Thanked 214 Times in 189 Posts
Default

I've been thinking. Please bear with me as I try to establish my current understanding.
  1. I understand that OpenSSL will use /dev/urandom if available, and only use /dev/random if /dev/urandom is not available, in order to seed its own PRNG.
  2. I understand that /dev/urandom produces low quality streams as it will always output values, even when the entropy pool is exhausted.
However, I am at a loss to understand why anyone would use urandom for password generation, if they are concerned about the quality of random number generation.

The urandom pseudo device is not available on every OS, but on those where it is still available I'd thought it was intended for use in transient functions, such as padding bytes or short lifespan keys (such as one might find in PKI for telecommunications).

I ask because both of you have apparently made the recommendation.

If I assume the quality of OpenSSL PRNG that seeds from /dev/urandom is of equivalent quality to the /dev/urandom stream itself, then $ openssl rand -hex ... produces the same quality of random hex stream as $ dd if=/dev/[u]random ... | hexdump ...

I don't know if the PRNG seeded by urandom is of equivalent quality. But I cannot see it being better than urandom itself, and based only on what I know of urandom, I wouldn't recommend it as a base for static keys such as passwords.

Have I misunderstood?

It's an academic question if the OS doesn't have urandom, of course.




Last edited by jggimi; 10th February 2014 at 07:24 PM.
Reply With Quote
  #5   (View Single Post)  
Old 10th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

jggimi, I think you're asking "How much worse is /dev/urandom than /dev/random ?" and "How suitable is the output of /dev/urandom for various password usages?" Those are great questions and I am not expert enough to answer them in any depth at all. I would suspect the quality of urandom might vary somewhat depending on the OS, of course, and the appropriateness of its usage would vary greatly with the intended application. For example if you wanted, say, an 8- or 10-digit password to sign up for a short-term use forum somewhere, I'll go on a limb and say /dev/urandom is probably good enough, and better than typing "asdklj3f" or something like that.

For that reason my script supports both devices; it's really up to the user to decide which is best for their needs. It doesn't recommend one or the other for everything. The reason urandom is the default is simply that it doesn't block.

When I used it to generate a WPA PSK "for real", I did use /dev/random, and had to type a lot of keystrokes into a console before it finished -- it seemed worth the trouble in that case.

ADDED: I've also used the same technique to generate random wireless MAC addresses:

Code:
00`hexdump -n5 -e'5/1 ":%02x"' /dev/urandom`
In this case cryptographic strength isn't a real issue and non-blocking of /dev/urandom is very helpful as the command is run in a shell script. (Just another application, nothing to do with randpass.sh.)

Last edited by IdOp; 10th February 2014 at 08:25 PM.
Reply With Quote
  #6   (View Single Post)  
Old 10th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

About the differences between /dev/random and /dev/urandom for Linux, there is a good discussion on this man page:

random(4)

Quote:
If you are unsure about whether you should use /dev/random or /dev/urandom, then probably you want to use the latter.

Last edited by IdOp; 10th February 2014 at 08:43 PM. Reason: added (4)
Reply With Quote
  #7   (View Single Post)  
Old 10th February 2014
Carpetsmoker's Avatar
Carpetsmoker Carpetsmoker is offline
Real Name: Martin
Old man from scene 24
 
Join Date: Apr 2008
Location: Eindhoven, Netherlands
Posts: 2,074
Thanked 198 Times in 156 Posts
Default

Quote:
I understand that /dev/urandom produces low quality streams as it will always output values, even when the entropy pool is exhausted.
On FreeBSD, /dev/random uses Yarrow (/dev/urandom is just a link to /dev/random, apparently).
/dev/random should to produce either a quality stream, or no output at all.

From random(4):
Code:
     The kern.random.sys.seeded variable indicates whether or not the random
     device is in an acceptably secure state as a result of reseeding.  If set
     to 0, the device will block (on read) until the next reseed (which can be
     from an explicit write, or as a result of entropy harvesting).  A reseed
     will set the value to 1 (non-blocking).
I don't have a Linux system, so I'll just quote wikipedia:

Quote:
/dev/random should be suitable for uses that need very high quality randomness such as one-time pad or key generation. When the entropy pool is empty, reads from /dev/random will block until additional environmental noise is gathered

[...]

A counterpart to /dev/random is /dev/urandom ("unlimited" /non-blocking random source) which reuses the internal pool to produce more pseudo-random bits. This means that the call will not block, but the output may contain less entropy than the corresponding read from /dev/random. While it is still intended as a pseudorandom number generator suitable for most cryptographic purposes, it is not recommended for the generation of long-term cryptographic keys.
So it would seem that you're right. I had always assumed that /dev/random was an old (insecure) generator, and that /dev/urandom was a new (good) generator. I have no idea where I got this idea from. I think I got told "use /dev/urandom" once many years ago and just made assumptions from there ... :-/


Assuming that /dev/urandom isn't of spectacular low quality (which it doesn't seem to be), I don't think this has to be a problem, necessarily.
Remember, we not generating keys for use in a stream cipher 100 times/minute, where even a small problem in your PRNG would indeed be a major problem. What we're doing is occasionally generate a short password, often with large time intervals (days, weeks) in between two runs.

- An adversary needs n bits of output first to predict anything. Passwords (15 bytes in my example) generally aren't long enough to provide this output. My proposed solution even discards some output (not a security feature as such, obviously).

- You're not generating a lot of passwords, and the passwords you generate have a lot of time in between them. So if an advisory has somehow managed to get 200 of yours passwords, it's extremely unlikely anything useful can be learned about a potential 201st password.

- Even if an adversary examined all output of your PRNG, it needs to be *very* broken if you can learn enough to learn a 120 bit password to be truly useful.

- If the output wouldn't be uniform, you'd still get a fairly good pseudo-random password, unless, of course, your PRNG starts outputing 80% 0's or something ...


A system is only as strong as the weakest link, and there are quite a few weaker links I would worry about, such as:

- Passwords getting stolen (many examples of this), and re-used on other accounts. Obviously easily preventable by using an unique password for every service.

- Passwords not being stored securely, for as plaintext on the server, or as plaintext on your computer (such as your browser's cookie. Yes, people do this, often without the httponly flag set, meaning is ludicrously easy to steal it).

- (Hardware) keyloggers (I wouldn't know how to protect against this, I've seen people glue USB ports, but I don't consider that to be truly secure. You would somehow need to tamper-proof your computer's connections, your keyboard, and the cable).

- Various social engineering attacks (as described here, many other examples to be found).

- Shoulder surfing (through eyeballs, or camera's).


Open questions
- How likely is it that /dev/random is unable to generate high quality randomness, and will block?
- How low is the quality of /dev/urandom, exactly?
- What is the best cross-platform, random source? /dev/{a,u}random isn't defined by any standard. POSIX only offers random(), which isn't considered secure.
__________________
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.
Reply With Quote
  #8   (View Single Post)  
Old 10th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

From the OpenBSD man page random(4):

Quote:
All the random devices are expected to provide high quality pseudo-random output data.
No warnings and no significant quality distinction there.

About the chance of /dev/random of blocking, I guess you could experiment a bit and see what happens on a given system. For a single-user system, at least there will be no other users purposely or accidentally draining the entropy pool, but there may be less entropy being collected too? I've had varied results with blocking on different systems and OSes. I'm not sure I can say anything definitive though.
Reply With Quote
  #9   (View Single Post)  
Old 11th February 2014
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,710
Thanked 214 Times in 189 Posts
Default

Quote:
Originally Posted by IdOp View Post
From the OpenBSD man page random(4)....
and that's because that is an OS which no longer has urandom. See the link:
Quote:
It's an academic question if the OS doesn't have urandom, of course.
Reply With Quote
Old 11th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

Quote:
Originally Posted by jggimi View Post
and that's because that is an OS which no longer has urandom. See the link:
Thanks jggimi, I'd missed following that link! Now this is very weird, because I tested my script, without trouble, at the free OpenBSD host devio.us, which is running 5.4 i386, and there we have:

Code:
$ ls -l /dev/*random
crw-r--r--  1 root  wheel   45,   3 Jan 15 05:12 /dev/arandom
crw-r--r--  1 root  wheel   45,   0 Jan 14 04:03 /dev/random
crw-r--r--  1 root  wheel   45,   1 Jan 14 04:03 /dev/srandom
crw-r--r--  1 root  wheel   45,   2 Jan 14 04:03 /dev/urandom
I don't know how to explain this. Is there some kind of optional support for it?
Reply With Quote
Old 11th February 2014
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,710
Thanked 214 Times in 189 Posts
Default

Looking at the patch, it seems to me that all devices use the same arc4random(3) PRNG.
Reply With Quote
Old 11th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

Ok good, that makes sense. Then it seems there is a urandom, but it isn't different in any way from the others?
Reply With Quote
Old 11th February 2014
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 3,710
Thanked 214 Times in 189 Posts
Default

The man page just takes the reader to random(4), and only /dev/random and /dev/arandom are listed, with no difference described.

Note that the ARC4 PRNG switches to ChaCha with the next -release.
Reply With Quote
Old 11th February 2014
Alphalutra1 Alphalutra1 is offline
Port Guard
 
Join Date: Sep 2008
Posts: 29
Thanked 0 Times in 0 Posts
Default

I use something similar to Carpetsmoker's first "good" option; OpenBSD's head, however, doesn't have the "-c" option; here's my hackish password generator (characters allowed are alphanumerics, underscore, and dash):

Code:
< /dev/urandom strings -n1 | tr -dc _A-Z-a-z-0-9 | dd count=1 bs=16 2>/dev/null;echo
I know it works on Linux and OpenBSD; just change the bs in the dd command to change the password length;

Comments appreciated since I am far from adept at the command line.
Reply With Quote
Old 11th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

I'd like to subject the WPA passphrase part of my script to some needed criticism , as I realized it has a certain weakness, that I think is worth explaining.

The basic idea is that each input byte from /dev/[u]random, thought of as an integer in the domain {0, 1, ..., 255} is scaled into the desired output set { ASCII 32, ..., ASCII 126 }, which is a set of 95 values. (Actually, there is one less because I weed out the double quote " but let's ignore that, it just complicates the discussion.) The scaling idea was appealing to me, but since we're working with discrete sets, not real numbers, there can be problems.

So basically there's a map from 256 values into 95 values. If you think of such maps generally, many of them would be very bad. E.g., if the map was a constant, it would always give the same output, which wouldn't be random at all! The scaling map I used isn't nearly that bad; it does map onto the whole desired output range, in a many-to-one way. The problem is that the "many" is not the same for each output value though.

For example the map starts off like this:

Code:
input output
byte  ASCII
 0    32
 1    32
 2    32
 3    33
 4    33
 5    33
 6    34
 7    34
 8    34
 9    35
10    35
11    36
12    36
13    36
  ...
You can see output 35 has only two pre-images (inputs 9 and 10), while the others shown have three. If you look at the whole map, you find:

1 output has a single pre-image,
27 outputs have two pre-images,
67 outputs have three pre-images.

Note: 1 + 27 + 67 = 95 and 1*1 + 27*2 + 67*3 = 256.

What this means is that even if the inputs from /dev/[u]random are uniformly probable, the outputs are not. E.g., an output of ASCII 32 is 1.5 times as likely to occur as ASCII 35.

I don't think there is a way around this kind of problem if you want to keep all the inputs, because 95 just doesn't divide evenly into 256. You could get around it by throwing out some of the inputs so that the range is uniformly covered, say 2-to-1 everywhere. But this would have the cost of reading more bytes from /dev/[u]random, and in principle the process might never finish!

An alternative would be to restrict the size of the output set so it divides evenly into 256 (say, 64), and choose a map that covers it uniformly. But a smaller output set is a drawback too.

So, all in all, I think there are several reasons to prefer the WPA pre-shared key (PSK) method, which as far as I can see is done very cleanly by my script (criticisms also welcome!). They are:

1) Passphrase has the above problem (in my script). (Though perhaps I'm making a mountain out of a molehill?)

2) Passphrase is at most 63 characters anyway, which doesn't seem to contain as much randomness as a 64-byte PSK chosen with each byte totally random. The passphrase, if used, is mixed with the ESSID to create a PSK, but this probably doesn't help since the ESSID is public knowledge.

3) A good passphrase should be long, contain characters from a large set, and be very random. This means you won't be able to remember it! If you can't remember the passphrase, you might as well use a random PSK (which you can't remember either), since it's better. Either way you'll have to cut-and-paste it into your wireless setups.

Comments etc. also very welcome.

Last edited by IdOp; 11th February 2014 at 11:51 PM. Reason: spelling, and molehill
Reply With Quote
Old 15th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

Interesting thread!

Quote:
Originally Posted by Alphalutra1
Code:
< /dev/urandom strings -n1 | tr -dc _A-Z-a-z-0-9 | dd count=1 bs=16 2>/dev/null;echo
I know it works on Linux and OpenBSD; just change the bs in the dd command to change the password length;

Comments appreciated since I am far from adept at the command line.
At first I thought that having the dd command at the end with bs=16 would make it stop reading from /dev/urandom when it had found 16 (allowed) characters. That would be a nice feature, but then thinking about it more I became less sure that would happen. I thought the first command in the pipeline, strings, might very well read large blocks from /dev/urandom and start feeding it down the pipe, only to have the thing terminate after a small number of bytes had been processed. But how to check this?

After a couple of failed attempts, I think I found something that works. Change the pipeline to this:

Code:
< /dev/urandom tee file.tmp | strings -n1 | tr -dc _A-Z-a-z-0-9 | dd count=1 bs=16 2>/dev/null;echo
(Make sure file.tmp does not already exist!) The tee command is like a cat that also copies everything going through it to the given file. So by checking the size of file.tmp you can see how many bytes were read from /dev/urandom (or at least get a lower bound). (I'm assuming tee reads similarly to how strings would.) Here are the file sizes I observed:

Linux: 49152

OpenBSD: 196608

If this is showing what it seems, that's a lot of data to be reading from /dev/urandom just to get 16 password char's. If entropy is a valuable resource, this could deplete it.

Also a comment and question about Carpetsmoker's suggestion:

Quote:
Originally Posted by Carpetsmoker
Code:
head -c100 /dev/urandom | strings -n1 | tr -d '[:space:]' | head -c15
The head -c100 limits the reading of /dev/urandom, which is good, although it still seems a bit long. If you had bad (u)random luck ( ) you might never get the 15 characters you're after. There's a trade-off here as to how much to read, but if this was used in a script it would be good to check that the result was as desired and if not try again.

Another thing that occurred to me. As I understand it UTF-8 characters can be encoded in 1, 2, 3, ... several bytes. Intuitively it seems to me that this means there will be a strongly non-uniform distribution of UTF-8 characters coming out of /dev/urandom. Consider a two-byte sample from /dev/urandom. A given 1-byte UTF-8 character would be found in this sample with probability 2/256 =1/128 (approximately, ignoring a repeat), since it can be in either byte. But a given 2-byte UTF-8 character will be found only with probability (1/256)^2. Similarly one can consider longer UTF-8 chars.

So a question: When you use this method in a UTF-8 environment, do you find that the 2-byte characters (and longer) are indeed seen extremely rarely in the password?

I am thinking if one could effectively turn the /dev/urandom output into some kind of pointer into a table of password characters, it might help with some of these problems.
Reply With Quote
Old 15th February 2014
Alphalutra1 Alphalutra1 is offline
Port Guard
 
Join Date: Sep 2008
Posts: 29
Thanked 0 Times in 0 Posts
Default

Quote:
Originally Posted by IdOp View Post
Interesting thread!



At first I thought that having the dd command at the end with bs=16 would make it stop reading from /dev/urandom when it had found 16 (allowed) characters. That would be a nice feature, but then thinking about it more I became less sure that would happen. I thought the first command in the pipeline, strings, might very well read large blocks from /dev/urandom and start feeding it down the pipe, only to have the thing terminate after a small number of bytes had been processed. But how to check this?

After a couple of failed attempts, I think I found something that works. Change the pipeline to this:

Code:
< /dev/urandom tee file.tmp | strings -n1 | tr -dc _A-Z-a-z-0-9 | dd count=1 bs=16 2>/dev/null;echo
(Make sure file.tmp does not already exist!) The tee command is like a cat that also copies everything going through it to the given file. So by checking the size of file.tmp you can see how many bytes were read from /dev/urandom (or at least get a lower bound). (I'm assuming tee reads similarly to how strings would.) Here are the file sizes I observed:

Linux: 49152

OpenBSD: 196608
So I tweaked your tee stuff in a way I think allows you to measure how much strings pulls at a time:

Code:
< /dev/urandom strings -n1 | tee file.tmp | tr -dc _A-Z-a-z-0-9 | dd count=1 bs=16 2>/dev/null;echo
which gave a file.tmp size on OpenBSD of 80K; using tee before strings got one on my box of 184K. Similarly on Linux it gave me ~30K while starting with tee gave me ~65K. So strings seems to take smaller chunks than tee I guess?

Anyways, I'm now just taking 500 out of /dev/random which gives me a final file that's ~100 characters on linux, plenty of wiggle room and 500 isn't that bad a pull:

Code:
dd if=/dev/urandom count=1 bs=500 2>/dev/null | strings -n1 | tee file.tmp | tr -dc _A-Z-a-z-0-9 | tee file.tmpp | dd count=1 bs=16 2>/dev/null;echo
I'm excited to use the new silent dd option ("status=noxfer") that just went into openbsd to make it compatible with gnu to stop redirecting std err to null; unfortunately not compatible with freebsd/netbsd though

Last edited by Alphalutra1; 15th February 2014 at 10:02 PM.
Reply With Quote
Old 15th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

Quote:
Originally Posted by Alphalutra1 View Post
So I tweaked your tee stuff in a way I think allows you to measure how much strings pulls at a time:

Code:
< /dev/urandom strings -n1 | tee file.tmp | tr -dc _A-Z-a-z-0-9 | dd count=1 bs=16 2>/dev/null;echo
which gave a file.tmp size on OpenBSD of 80K; using tee before strings got one on my box of 184K. Similarly on Linux it gave me ~30K while starting with tee gave me ~65K. So strings seems to take smaller chunks than tee I guess?
Good idea, so I took it one step further and let tee also be in the 3rd position after the tr command. On Linux these are the file sizes:

Code:
49152  file-1
20480  file-2
 4096  file-3
Basically similar to what you observed, but it gets smaller still in the 3rd position. I'm not sure we can conclude one program is reading in bigger chunks than the other. It might just indicate how much data has been passed down the chain by the time everything terminates. That's a bit vague because I don't know exactly how that works.

I also don't know if tee is getting a chance to write out everything that passed through it or not. On one of my failed ideas (before hitting on tee) I wrote a simple C program that is like cat but counts how much data passes through it and writes the total to a file at the very end. When I put it in the pipeline, it never wrote the file. This suggests the pipe's programs may be terminated before they're "done" in the usual sense; they probably just get killed off, in turn, as soon as the final one finishes. If that happens to tee, it may be showing less than the actual amount read.

In any event this suggests strongly that lots of data is being read from /dev/urandom by this method, even if it's hard to tell exactly how much.

Quote:
Anyways, I'm now just taking 500 out of /dev/random which gives me a final file that's ~100 characters on linux, plenty of wiggle room and 500 isn't that bad a pull:

Code:
dd if=/dev/urandom count=1 bs=500 2>/dev/null | strings -n1 | tee file.tmp | tr -dc _A-Z-a-z-0-9 | tee file.tmpp | dd count=1 bs=16 2>/dev/null;echo
500 is much better but it still seems like more than needed. On Linux you can see the entropy pool size in bits with:

$ cat /proc/sys/kernel/random/entropy_avail

(See random(4).) This shows at most 4096 bits = 512 bytes. So reading 500 will nearly drain it, temporarily. For comparison, in a simple modification of my script you only need to read one byte out of /dev/urandom for every character in the password, with a uniformly distributed 64-character set. (The same size as your [:alnum:]_- set by coincidence!)

Note: I edited the above to correct the filename from "poolsize" to "entropy_avail" ... oops.

Last edited by IdOp; 15th February 2014 at 11:49 PM.
Reply With Quote
Old 24th February 2014
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 564
Thanked 14 Times in 13 Posts
Default

Just for fun, I wrote a little shell script that generates pseudo-random passwords along the lines hinted at above. The features are:

* Uses /dev/random by default, or /dev/urandom with the -u flag.

* Passwords can be from 1 to 63 characters long.

* The characters come from a set of 64 ASCII characters, uniformly distributed (assuming output from /dev/[u]random is so).

* The use of entropy is quite efficient in that every 3 bytes read from /dev/[u]random can generate 4 characters. This is done by saving unused bits and reassembling them.

Code:
$ randpw.sh
Usage> randpw.sh [-u] LENGTH
        LENGTH : Length of password (1,...,63)
          -u   : Use /dev/urandom (default: /dev/random)
Code:
$ randpw.sh -u 10
%_9EL^;PO[
It seems to work on sh, ksh, bash. Suggestions, etc., welcome.

randpw.sh.gz
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Generating passwords with jot(1) J65nko Guides 9 29th August 2014 01:03 PM
Security LinkedIn passwords in circulation J65nko News 1 8th June 2012 04:49 AM
Experts: We're stuck with passwords – and maybe they're best J65nko News 1 17th January 2012 03:08 AM
Passwords zhorik OpenBSD General 5 14th January 2011 12:51 AM
Generate xorg.conf.new and black screen aleunix OpenBSD Packages and Ports 2 4th June 2008 10:49 AM


All times are GMT. The time now is 04:19 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick