|
OpenBSD Security Functionally paranoid! |
|
Thread Tools | Display Modes |
|
|||
Solution for updating file safely?
Hi,
Can anyone let me know if OpenBSD has any solution to prevent file damaged if the system crashed while updating large files? |
|
|||
At this moment there is not a journalling filesystem for OpenBSD yet.
You could do something like this:
From rename(2): Code:
int rename(const char *from, const char *to); DESCRIPTION The rename() function causes the link named from to be renamed as to. If to exists, it is first removed. Both from and to must be of the same type (that is, both directories or both non-directories), and must reside on the same file system. rename() guarantees that if to already exists, an instance of to will always exist, even if the system should crash in the middle of the operation.
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump Last edited by J65nko; 5th February 2016 at 11:01 PM. |
|
|||
I think it's possible to only use the steps 1,2 and 4.
In case of a crash, based on the information you provided, we may end up with only the original file. But within a same file system, a rename operation is very fast. Last edited by albator; 5th February 2016 at 11:58 PM. |
|
|||
Thank you all
|
|
|||
There is a difference in behaviour if the target file already exists. rename(2) will overwrite an already existing file , while link(2) will refuse to create a new link (directory entry), if that filename/directory entry already exists.
From link(2): Code:
NAME link - make hard link to a file int link(const char *name1, const char *name2); DESCRIPTION The link() function atomically creates the specified directory entry (hard link) name2 with the attributes of the underlying object pointed at by name1. If the link is successful: the link count of the underlying object is incremented; name1 and name2 share equal access and rights to the underlying object. If name1 is removed, the file name2 is not deleted and the link count of the underlying object is decremented. name1 must exist for the hard link to succeed and both name1 and name2 must be in the same file system. As mandated by POSIX.1 name1 may not be a directory. ERRORS [snip] [EEXIST] The link named by name2 does exist Code:
$ echo Shere Khan >new/animal $ echo Simba >temp/lion $ ln temp/lion new/animal ln: new/animal: File exists $ ls -li new/animal temp/lion 571653 -rw-r--r-- 1 adriaan wheel 11 Feb 6 16:22 new/animal 571654 -rw-r--r-- 1 adriaan wheel 6 Feb 6 16:22 temp/lion $ ln -f temp/lion new/animal $ ls -li new/animal temp/lion 571654 -rw-r--r-- 2 adriaan wheel 6 Feb 6 16:22 new/animal 571654 -rw-r--r-- 2 adriaan wheel 6 Feb 6 16:22 temp/lion $ stat -s new/animal st_dev=0 st_ino=571654 st_mode=0100644 st_nlink=2 st_uid=1001 st_gid=0 st_rdev=2292638 st_size=6 st_atime=1454772179 st_mtime=1454772179 st_ctime=1454772265 st_blksize=16384 st_blocks=4 st_flags=0 $ stat -s temp/lion st_dev=0 st_ino=571654 st_mode=0100644 st_nlink=2 st_uid=1001 st_gid=0 st_rdev=2292638 st_size=6 st_atime=1454772179 st_mtime=1454772179 st_ctime=1454772265 st_blksize=16384 st_blocks=4 st_flags=0 $ cat new/animal Simba $ cat temp/lion Simba $ rm temp/lion $ stat -s new/animal st_dev=0 st_ino=571654 st_mode=0100644 st_nlink=1 st_uid=1001 st_gid=0 st_rdev=2292638 st_size=6 st_atime=1454772179 st_mtime=1454772179 st_ctime=1454772265 st_blksize=16384 st_blocks=4 st_flags=0
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump |
|
|||
This subject presents an opportune moment to again advocate the importance of backups. If the data is important and/or it cannot be easily recreated, back it up, and back it up often.
|
|
|||
Paranoid user may consider to add sync(8) command between steps 1 and 2, 4 and 5 described in post #2.
Sync can be redundant if sync is used as parameter for mounted filesystem, as jggimi suggested. Keep in mind that sync synchronizes all file systems. Last edited by e1-531g; 6th February 2016 at 03:58 PM. Reason: Added further informations |
|
||||
A sync(8) at the end of each of each command step will flush any pending write cache buffers to the filesystems. But, a filesystem mounted with the sync option will cause each writes to wait until the drive confirms a pending write operation has completed. That's a huge difference ... in process performance and I/O throughput particularly. Synchronous mounts are not used as a standard practice. Applications that need to ensure data integrity, such as database management systems, typically issue fsync(2) at the point where a transaction needs to be committed.
Of course, having data written to the drive does not ensure it is readable. The drive may not be able to read the sectors written. The drive may have confirmed the write while sectors remain in the drive's write cache. The data written may not be a discrete entity, such as adding sectors to the end of a file but then missing the metadata update that notes the extension. The drive may fail later. RAID systems may also improve data integrity, by providing redundant write operations. They can also reduce integrity and add risk, and should be used with care. And they are never replacements for backup. A redundant write is written to at least two locations. So are redundant errors. |
|
|||
It's good to hear that are some sane people on the Earth.
|
|
||||
Quote:
...prevent file damaged if the system crashed while updating large files? Part of the sentence (above) is past tense. Did something already happen regarding "updating large files"? BTW - What does "updating large files" mean?And part of the sentence suggests a future tense. Are you preparing for, or speculating about, a potential "crash"? |
|
|||
I now think that "paranoid user" was not the best phrase.
Anyway inspired by that question and answers: https://unix.stackexchange.com/quest...-one-partition I have created a script to update file more safely, than with default filesystem options, but still not sacrifice performance for long period of time. Code:
/sbin/mount -u -o sync,noatime e2687744d2198a2e.a / && echo "filesystem in sync" && mount 2>&1 | grep ' / ' #redundancy for updating file, but if you are going to just copy, it is not required cp /testSync/there_will_be_copied_file1 /testSync/there_will_be_copied_file1.tmp #copying fresh data, but one should update file insted of copy to it cp /testSync/file1 /testSync/there_will_be_copied_file1.tmp mv /testSync/there_will_be_copied_file1 /testSync/there_will_be_copied_file1.old mv /testSync/there_will_be_copied_file1.tmp /testSync/there_will_be_copied_file1 rm /testSync/there_will_be_copied_file1.old /sbin/mount -u -o noatime e2687744d2198a2e.a / && echo "filesystem not in sync" && mount 2>&1 | grep ' / ' Code:
./copy.sh filesystem in sync /dev/sd1a on / type ffs (local, noatime, synchronous) filesystem not in sync /dev/sd1a on / type ffs (local, noatime) Of course everybody should do backups, but... maybe this is backup script? Last edited by e1-531g; 22nd February 2016 at 05:02 PM. |
|
|||
By defaults OpenBSD mounts are already mounted with the "sync" option. See 14.19 - Why aren't we using async mounts?
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump |
|
||||
Not qute. sync and async are mount options. The definitive guidance is in mount(8).
async Metadata and file data are written asynchronously. sync Metadata and file data are written synchronously. Neither async nor sync File data is written asynchronously, metatdata is written synchronously. |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
doing without X .. what files to safely remove ? | daemonfowl | OpenBSD General | 17 | 22nd August 2012 12:07 AM |
Windows solution? | dzoni | Other OS | 3 | 11th July 2010 07:29 AM |
Any Solution for Xorg high CPU usage in Ubuntu 9.04? | Mantazz | Other BSD and UNIX/UNIX-like | 11 | 14th July 2009 06:10 AM |
Which is the best solution from start bsd.mp? | aleunix | OpenBSD General | 18 | 4th May 2009 06:33 PM |
consider OpenVPN production-grade solution? | nimnod | Off-Topic | 1 | 26th March 2009 12:22 AM |