DaemonForums  

Go Back   DaemonForums > Miscellaneous > Programming

Programming C, bash, Python, Perl, PHP, Java, you name it.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 10th July 2008
na5m na5m is offline
New User
 
Join Date: Jul 2008
Posts: 4
Default sed and hex

I'm using OpenBSD 4.3. I'd like to be able to insert an arbitrary hexcode into a text file. For example, I'd like to hyphenate the word "gratefull" using sed so that "gratefull" becomes "grate-full". I've tried
Code:
sed 's/gratefull/grate\x2dfull/' document.txt
to no avail. I know I could use Ctrl-v, but I'd really like to be able to insert any hex value into my text file using the backslash construct. I'm sure you can do this with GNU sed, but as I've stated, I'm using OpenBSD. Any help is appreciated. Thanks!
Reply With Quote
  #2   (View Single Post)  
Old 10th July 2008
robbak's Avatar
robbak robbak is offline
Real Name: Robert Backhaus
VPN Cryptographer
 
Join Date: May 2008
Location: North Queensland, Australia
Posts: 366
Default

Is there anything wrong with just '-'? you may have to escape it '\-', but that is easy enough, isn't it?

One sed feature does use 3-digit octal for non-printable characters '\055' for your example - maybe that will work. The shortcut for hex is often (C-style) \0x2d - does that work? Of course, for readability's sake, use '\-' if it works!!!
__________________
The only dumb question is a question not asked.
The only dumb answer is an answer not given.
Reply With Quote
  #3   (View Single Post)  
Old 10th July 2008
ocicat ocicat is offline
Administrator
 
Join Date: Apr 2008
Posts: 3,318
Default

Quote:
Originally Posted by na5m View Post
I'd like to hyphenate the word "gratefull" using sed so that "gratefull" becomes "grate-full".
It is unclear as to whether you are expecting the original file to be overwritten. ie.
Code:
$ sysctl kern.version
kern.version=OpenBSD 4.3-current (GENERIC) #0: Wed Jun  4 12:43:45 PDT 2008
    me@mymachine:/usr/src/sys/arch/i386/compile/GENERIC

$ cat file
first line gratefull
second gratefull line
third line
gratefull fourth line
$ sed 's/gratefull/grate-full/' file > file1                                                                             
$ cat file1
first line grate-full
second grate-full line
third line
grate-full fourth line
$
Reply With Quote
  #4   (View Single Post)  
Old 10th July 2008
lvlamb's Avatar
lvlamb lvlamb is offline
Real Name: Louis V. Lambrecht
Spam Deminer
 
Join Date: May 2008
Location: .be
Posts: 221
Default

On OpenBSD sed, I use octal: \o174
For tab and newline, this is sometimes tricky.
Fwiw:
cat /usr/share/misc/ascii
__________________
da more I know I know I know nuttin'
Reply With Quote
  #5   (View Single Post)  
Old 10th July 2008
mfaridi's Avatar
mfaridi mfaridi is offline
Spam Deminer
 
Join Date: May 2008
Location: Afghanistan
Posts: 320
Default

another problem
when I change some words in txt file with Sed I open it and see words changed
but when I change another words and open file again I see new change but last change is not save

for example I run this commnad
Code:
sed 's/torrent/rtGUIHOT/' rtgui.access.log > rtgui.access1.log
I see this

Code:
192.168.0.42 - rtGUIHOT [05/Jul/2008:10:17:48 +0430] "GET / HTTP/1.1" 200 7444
192.168.0.42 - rtGUIHOT [05/Jul/2008:10:17:48 +0430] "GET /submodal/subModal.css HTTP/1.1" 304 -
192.168.0.42- rtGUIHOT [05/Jul/2008:10:17:49 +0430] "GET /submodal/common.js HTTP/1.1" 304 -
192.168.0.42- rtGUIHOT [05/Jul/2008:10:17:49 +0430] "GET /submodal/subModal.js HTTP/1.1" 304 -
192.168.0.42- rtGUIHOT [05/Jul/2008:10:17:49 +0430] "GET /style.css HTTP/1.1" 304 -
192.168.0.42- rtGUIHOT [05/Jul/2008:10:17:49 +0430] "GET /images/downarrow.gif HTTP/1.1" 304 -
80
and I run this command again

Code:
 sed 's/2008/20300/' rtgui.access.log > rtgui.access1.log                                                                  12:10
I see this

Code:
192.168.0.42 - torrent [05/Jul/20300:09:53:21 +0430] "GET /?setview=complete HTTP/1.0" 200 7443
192.168.0.42 - torrent [05/Jul/20300:09:53:23 +0430] "GET /submodal/subModal.css HTTP/1.0" 304 -
192.168.0.42 - torrent [05/Jul/20300:09:53:23 +0430] "GET /submodal/common.js HTTP/1.0" 304 -
192.168.0.42 - torrent [05/Jul/20300:09:53:23 +0430] "GET /submodal/subModal.js HTTP/1.0" 304 -
192.168.0.42- torrent [05/Jul/20300:09:53:24 +0430] "GET /style.css HTTP/1.0" 304 -
192.168.0.42- torrent [05/Jul/20300:09:53:24 +0430] "GET /images/downarrow.gif HTTP/1.0" 304 -

before I change rtorrent to rtGUI
and now change 2008 to 20300
but rtorrent is not rtGUI
Reply With Quote
  #6   (View Single Post)  
Old 10th July 2008
robbak's Avatar
robbak robbak is offline
Real Name: Robert Backhaus
VPN Cryptographer
 
Join Date: May 2008
Location: North Queensland, Australia
Posts: 366
Default

I'm sure if you thought about it, you'd work this one out.

accesslog |sed > accesslog1 Change one made
accesslog |sed > accesslog1 Change two made.

i think you wanted

accesslog |sed > accesslog1 Change one made
accesslog1 |sed > accesslog2 Change two made to file that has change 1 made.

[wrong]Oh, by the way, take a look at the -i option - it allows you to edit the file in-place: it changes the original file.[/wrong] apparently, OpenBSD's sed does not have this. Caught out again.
__________________
The only dumb question is a question not asked.
The only dumb answer is an answer not given.

Last edited by robbak; 10th July 2008 at 09:00 AM. Reason: Once again, caught out by BSD differences.
Reply With Quote
  #7   (View Single Post)  
Old 10th July 2008
ocicat ocicat is offline
Administrator
 
Join Date: Apr 2008
Posts: 3,318
Default

Quote:
Originally Posted by robbak View Post
Oh, by the way, take a look at the -i option - it allows you to edit the file in-place: it changes the original file.
I suspect you are thinking of GNU sed. sed(1) as available in (Open)BSD does not have a -i switch as can be found in the GNU variant:

http://unixhelp.ed.ac.uk/CGI/man-cgi?sed
Reply With Quote
  #8   (View Single Post)  
Old 11th July 2008
na5m na5m is offline
New User
 
Join Date: Jul 2008
Posts: 4
Default

I tried
Code:
$ sed 's/gratefull/grate\055full/' file1 > file2
$ sed 's/gratefull/grate\o055full/' file1 > file2
$ sed 's/gratefull/grate\0x2dfull/' file1 > file2
and many other permutations. They don't work. OpenBSD's sed doesn't seem to allow the programmatic insertion of an abitrary hex character. Not even '\t' or '\n', etc. like GNU sed does. Kind of a bummer. Maybe there's a binary sed available?
Reply With Quote
  #9   (View Single Post)  
Old 11th July 2008
ocicat ocicat is offline
Administrator
 
Join Date: Apr 2008
Posts: 3,318
Default

Quote:
Originally Posted by na5m View Post
They don't work.
I will need a few days before I can get back to this.

In the meantime, GNU sed is available in packages & ports:

http://www.openbsd.org/4.3_packages/....tgz-long.html
Reply With Quote
Old 11th July 2008
robbak's Avatar
robbak robbak is offline
Real Name: Robert Backhaus
VPN Cryptographer
 
Join Date: May 2008
Location: North Queensland, Australia
Posts: 366
Default

$ sed 's/gratefull/grate-full/' file1 > file2
$ sed 's/gratefull/grate\-full/' file1 > file2

??
__________________
The only dumb question is a question not asked.
The only dumb answer is an answer not given.
Reply With Quote
Old 11th July 2008
lvlamb's Avatar
lvlamb lvlamb is offline
Real Name: Louis V. Lambrecht
Spam Deminer
 
Join Date: May 2008
Location: .be
Posts: 221
Default

Ooops! Right, checked my scripts and it is gsed I use to write a postgresql INSERT. VALUES must be single-quoted and comma-separated.

Code:
gsed 's/^/INSERT INTO metals (date,ticker,bid_am,ask_pm,stock,oi) VALUES (\o047/g ; s/$/\o047);/g ; s/\t/\o047,\o047/g' lme_tmp > ../sql/lme-$mydate.sql
With OpenBSD sed, the trick to use the newline character is to ... escape the new line as
Code:
sed 's/|/\
/g' infile > outfile
Similarly the tab can be escaped. Works on the command line, but not when cut'n pasted or mulled through an editor (darn things translate the tabs into spaces).

Well ... call me a traitor, but y scripts work fine with gawk and gsed.
Once in a while on rainy days, I review some scripts and try which lines work with the OpenBSD version, and which one's are better kept on the GNU version. Must have nothing else to do as if it ain't broke, why fix it?
Worse, I use gdate to parse international dates.
Never had the time to read Perl's Date::Manip
__________________
da more I know I know I know nuttin'
Reply With Quote
Old 11th July 2008
na5m na5m is offline
New User
 
Join Date: Jul 2008
Posts: 4
Default

robbak, thank you for your suggestions. But I'd like to be able to look at an ASCII table and use a hex or octal value to insert the character of my choice. Especially 0x09 (horizontal tab).
Reply With Quote
Old 11th July 2008
TerryP's Avatar
TerryP TerryP is offline
Arp Constable
 
Join Date: May 2008
Location: USofA
Posts: 1,547
Default

Well, usually I wouldn't mention it but since it comes pre installed last I looked, you might always try Perl instead of having to install the GNU Stream EDitor.


Code:
Terry@dixie$ echo 'gratefull' | perl -pe 's/gratefull/grate\x2dfull/'      4:54
grate-full
Terry@dixie$                                                               4:55
Which I assume is what your trying to do but upon document.txt instead of stdin with sed.
__________________
My Journal

Thou shalt check the array bounds of all strings (indeed, all arrays), for surely where thou typest ``foo'' someone someday shall type ``supercalifragilisticexpialidocious''.
Reply With Quote
Old 11th July 2008
na5m na5m is offline
New User
 
Join Date: Jul 2008
Posts: 4
Default

thanks for all of the helpfull replies, everyone
and ocicat, i'm looking forward to whatever
else you can provide ...

PS- sorry lvlamb, escaping the newline worked, but escaping the tab did not

Last edited by na5m; 11th July 2008 at 12:11 PM. Reason: additional information
Reply With Quote
Old 11th July 2008
lvlamb's Avatar
lvlamb lvlamb is offline
Real Name: Louis V. Lambrecht
Spam Deminer
 
Join Date: May 2008
Location: .be
Posts: 221
Default

Sed does, but ksh will beep and not accept the tab:

Code:
$ cat dummy
#!/bin/sh
exit
$  sed 's/\//\/g' dummy
sed: 1: "s/\//\/g": unterminated substitute in regular expression
$ csh
celeron:bin {1}  sed 's/\//\	/g' dummy
#!	bin	sh
exit
celeron:bin {2}
Only trust man pages. Trust? As long as you now under which shell the author wrote his code or examples.

-----
Dumb me!
tab is the autocompletion key in bash, ksh, .... 'course it beeps!
__________________
da more I know I know I know nuttin'

Last edited by lvlamb; 11th July 2008 at 03:39 PM.
Reply With Quote
Old 13th July 2008
ocicat ocicat is offline
Administrator
 
Join Date: Apr 2008
Posts: 3,318
Default

Quote:
Originally Posted by na5m View Post
i'm looking forward to whatever
else you can provide ...
Octal sequences are only discussed in the sed(1) manpage only with respect to addressing, not substitutions. Therefore, if you want to embed non-printable characters within a substitution, stay with GNU sed.
Reply With Quote
Old 15th July 2008
IdOp's Avatar
IdOp IdOp is offline
Too dumb for a smartphone
 
Join Date: May 2008
Location: twisting on the daemon's fork(2)
Posts: 1,027
Default

Coming back to na5m's original question, perhaps something like this might do the trick. The idea is to use printf(1), which does support octal/hex notations, to write the sed command, and then use eval to execute the command:

Code:
eval $(printf "sed s/gratefull/grate\xYYfull/") document.txt
Here, YY represents the hex digits of the char you want, so you can specify it by number rather than some keysboard combination.
Reply With Quote
Reply

Tags
hex, horizontal, sed, tab

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 07:13 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick