DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD Installation and Upgrading

OpenBSD Installation and Upgrading Installing and upgrading OpenBSD.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 15th February 2018
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default Upgrade -> disk full!

I have three machines running -current, and I apply new snapshots fairly recently. Two of them run through the upgrade process without a hitch. On the third machine, the disk fills up every time!

I boot bsd.rd, hit "upgrade", select sd0 as the root disk, etc etc. As soon as it starts extracting sets, /dev/rd0a fills up. It's like the upgrader is extracting them to the ramdisk, rather than sd0. What could cause this?

Last edited by beavers; 15th February 2018 at 08:32 PM. Reason: typo
Reply With Quote
  #2   (View Single Post)  
Old 15th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

We don't have enough information to answer the question. $ df -h on your running system should tell you what partitions are mounted, and how full each is.
Reply With Quote
  #3   (View Single Post)  
Old 16th February 2018
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Quote:
Originally Posted by jggimi View Post
$ df -h on your running system should tell you what partitions are mounted, and how full each is.
That's exactly where I discovered that /dev/rd0a is the partition that's filling up. All of the partitions on sd0 are mounted correctly on /mnt during the upgrade, it's just that the sets aren't extracting to them -- they're writing to the ramdisk partition (/dev/rd0a), which is only lik e 3.6mb or something, so it fills up almost immediately. When it happens, df shows the other partitions haven't moved at all.
Reply With Quote
  #4   (View Single Post)  
Old 16th February 2018
PapaParrot's Avatar
PapaParrot PapaParrot is offline
parrot
 
Join Date: Jul 2015
Location: Durango, Mx.
Posts: 472
Default

It still would make things easier if you can show us, some real details,
the output of 'df -h' would be a good starting point, there just is not enough details to say anything.
__________________
My best friends are parrots
Reply With Quote
  #5   (View Single Post)  
Old 16th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

Quote:
Originally Posted by beavers View Post
...they're writing to the ramdisk partition (/dev/rd0a)...
My apologies, I should have been more clear. Please show us the output of the df(1) command from your previously installed system that you cannot successfully upgrade.
Reply With Quote
  #6   (View Single Post)  
Old 16th February 2018
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Sure thing. Here's `df -h` right before extracting the sets:

Code:
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/rd0a      3.5M    3.1M    317K    91%    /
/dev/sd0a     1005M    462M    492M    48%    /mnt
/dev/sd0k     40.0G    266M   37.8G     1%    /mnt/home
/dev/sd0d      3.9G   12.0K    3.7G     0%    /mnt/tmp
/dev/sd0f      2.0G    1.0G    865M    55%    /mnt/usr
/dev/sd0g     1005M    184M    770M    19%    /mnt/usr/X11R6
/dev/sd0h      8.6G    676M    7.5G     8%    /mnt/usr/local
/dev/sd0j      2.0G    2.0K    1.9G     0%    /mnt/usr/obj
/dev/sd0i      2.0G    2.0K    1.9G     0%    /mnt/usr/src
/dev/sd0e      8.7G   15.9M    8.3G     0%    /mnt/var
/dev/sd4a      5.3T    799G    4.3T    15%    /mnt/storage
...and this is `df -h` after the partition fills up. It happens in the middle of extracting base62.tgz, starts throwing "device is full" errors.

Code:
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/rd0a      3.5M    3.5M    1.0K   100%    /
/dev/sd0a     1005M    462M    492M    48%    /mnt
/dev/sd0k     40.0G    266M   37.8G     1%    /mnt/home
/dev/sd0d      3.9G   12.0K    3.7G     0%    /mnt/tmp
/dev/sd0f      2.0G    1.1G    797M    58%    /mnt/usr
/dev/sd0g     1005M    184M    770M    19%    /mnt/usr/X11R6
/dev/sd0h      8.6G    676M    7.5G     8%    /mnt/usr/local
/dev/sd0j      2.0G    2.0K    1.9G     0%    /mnt/usr/obj
/dev/sd0i      2.0G    2.0K    1.9G     0%    /mnt/usr/src
/dev/sd0e      8.7G   15.9M    8.3G     0%    /mnt/var
/dev/sd4a      5.3T    799G    4.3T    15%    /mnt/storage
Reply With Quote
  #7   (View Single Post)  
Old 16th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

Thank you. I'm going to guess that it is not the tiny ramdisk that is filling up, as there isn't ever much space within -- but rather, your relatively small /usr partition.

You are attempting to upgrade -current. And, in the 2017/11/05 entry of the Following -current FAQ, it says:
Quote:
2017/11/05 - make room in /usr/share before upgrading

The relinking of libraries and kernels at boot time now happens in new directory /usr/share/relink. If you have little room left on the filesystem containing /usr/share, you may want to run
# rm -r /usr/share/compile
before rebooting into bsd.rd.
This won't hurt, and may solve the problem.
Reply With Quote
  #8   (View Single Post)  
Old 16th February 2018
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

I would have suspected the same thing, except after receiving all the "device is full" messages, df shows /usr still has 797M free. It's definitley writing to the ramdisk, rather than /usr.

For the past few weeks I have instead been following the (not-recommended) directions at http://www.openbsd.org/faq/upgrade62.html#NoInstKern, and that works fine. No disk space issues whatever.

(Also, fwiw, /usr/share/compile does not exist on this system.)
Reply With Quote
  #9   (View Single Post)  
Old 16th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

What is the -release of the system you are attempting to upgrade to -current? I ask, because /usr/share/compile exists on my 6.2 systems.

Last edited by jggimi; 16th February 2018 at 12:15 PM. Reason: reason for asking added
Reply With Quote
Old 16th February 2018
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Code:
~$ dmesg | head -1
OpenBSD 6.2-current (GENERIC.MP) #6: Tue Feb 13 20:16:11 MST 2018
I don't know at what point /usr/share/compile gets created (or populated), but I do include a line in my "un-recommended upgrade" script to delete it.
Reply With Quote
Old 16th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

The directory structure is included in-base and is necessary for Kernel ASLR, known as "KARL".

I don't know for certain if this customization it is the root cause of the bsd.rd upgrade script breakage, but I am certain it is either this or another customization you have made to the system.
Reply With Quote
Old 16th February 2018
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Quote:
Code:
# rm -r /usr/share/compile
Wouldn't this remove the compile directory? Does it get re-created at some point before or during an upgrade?

Other than this missing directory (which would affect KARL certainly, but I can't imagine how it could affect extracting the sets), I honestly have made very few modifications to any of these systems. Each has a single disk, auto-partitioned the way the disklabel(8) suggests.
Reply With Quote
Old 16th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

I am unable to recreate the problem. If the system-to-be-upgraded has a /tmp filesystem, mounted as /mnt/tmp, then sets are downloaded to /mnt/tmp/sets*.

I am now unsure why -- or even HOW -- your problem is occurring.
Reply With Quote
Old 16th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

Quote:
Originally Posted by beavers View Post
Wouldn't this remove the compile directory? Does it get re-created at some point before or during an upgrade?
It would be recreated during the loading of the base*.tgz fileset, as it is included in base.
Reply With Quote
Old 16th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

Thinking about this some more, I am now of the opinion that you have a symbolic link in your filesystems pointing back into "/" where it shouldn't. When your ramdisk root partition fills up, what is it filling with?
Reply With Quote
Old 16th February 2018
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Quote:
Originally Posted by jggimi View Post
Thinking about this some more, I am now of the opinion that you have a symbolic link in your filesystems pointing back into "/" where it shouldn't. When your ramdisk root partition fills up, what is it filling with?
Hmmmmmmm....ran a `find / -type l -ls` and here are the interesting ones. None are pointing directly to /, but I wonder about the tmp?

Code:
/storage/www/htdocs/pub/OpenBSD/snapshots/amd64 -> /6.2/amd64
/sys -> usr/src/sys
/var/tmp -> ../tmp
I believe the root is filling up with the contents of base62.tgz, but I'll confirm that when I get in front of the machine.
Reply With Quote
Old 16th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

There should be no issues with /sys and /var/tmp, those are standard and in-base. /storage, I'll guess, is your own structure.
Reply With Quote
Old 16th February 2018
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Where can I find a copy of the upgrade script? (I don't know how to pull it out of bsd.rd. )
Reply With Quote
Old 16th February 2018
jggimi's Avatar
jggimi jggimi is offline
More noise than signal
 
Join Date: May 2008
Location: USA
Posts: 7,977
Default

The install and upgrade scripts are both symbolic links to install.sub, the source to that script is in src/distrib/miniroot/install.sub. There are also architecture stubs with machine dependent bits located in src/distrib/<arch>/<varies>/install.md.

Here's a link to the -current install.sub at this time, revision 1.1060:

http://cvsweb.openbsd.org/cgi-bin/cv...ype=text/plain
Reply With Quote
Old 17th February 2018
beavers beavers is offline
Shell Scout
 
Join Date: Nov 2017
Posts: 85
Default

Quote:
Originally Posted by jggimi View Post
I am now of the opinion that you have a symbolic link in your filesystems pointing back into "/" where it shouldn't. When your ramdisk root partition fills up, what is it filling with?
(emphasis changed) You were absolutely right. I ran another upgrade, and as it fills up the device, the last few files written were:

Code:
/: write failed, file system is full
tar: Failed write to file ./var/www/bin/ping: No space left on device
tar: Failed write to file ./var/www/bin/traceroute: No space left on device
tar: Could not create: ./var/www/cache: No space left on device
tar: Could not create: ./var/www/cgi-bin: No space left on device
tar: Unable to create ./var/www/conf: No space left on device
...and the bells go off in my head. I reboot and check a few things:

Code:
$ ls -AlF /var
lrwxr-xr-x  1 www  www  12 Mar  8  2017 /var/www@ -> /storage/www

$ grep storage /etc/fstab
/dev/sd4a /storage ffs rw,nodev,nosuid 1 2
So, it's failing to write to /mnt/var/www, either because it's a symlink, or because the target isn't mounted, I guess? Does that explain why it would write to the root of the ramdisk?

At any rate, I rebooted bsd.rd, created /storage on the ramdisk, then mounted sd4a there, and the upgrade proceeded as expected. Woohoo!

TL;DR -- I am a newb, and jggimi very astutely diagnosed my goofy mistake. Thank you immensely for your assistance!
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Full Disk Encryption jjstorm OpenBSD Installation and Upgrading 3 30th May 2016 08:19 PM
Full disk encryption in NetBSD 7.0 RC2 bashrules NetBSD Installation and Upgrading 1 18th August 2015 02:11 AM
Reliability concerns on full disk encryption virtuvoos OpenBSD Security 4 31st October 2013 11:06 PM
FreeNAS 8.3.1 introduces full disk ZFS encryption J65nko News 0 22nd March 2013 02:54 AM
Full disk encryption with Loop-AES deviant085 OpenBSD Security 9 23rd November 2009 12:51 AM


All times are GMT. The time now is 08:52 AM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick