DaemonForums  

Go Back   DaemonForums > Miscellaneous > Guides

Guides All Guides and HOWTO's.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 13th April 2021
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 3,779
Default Checking OpenBSD snapshot package mirror status

A Makefile to check which mirrors have the latest OpenBSD snapshot packages. You can use it together with the script from Checking OpenBSD snapshot mirror status

It is simple to adapt and configure for the mirrors in your country or region:
Code:
# Mirror sites definitions
# You do not have to use all of them
# List the ones (or others) you want to use in the line:
#     ". for site in Alberta ..... PlanetUnixCDN"
# Keep in mind that the Alberta site is the main one
# To alleviate the Alberta site please use a synchronized mirror

Alberta         =  https://ftp.openbsd.org/pub/OpenBSD
Nluug           =  https://ftp.nluug.nl/OpenBSD
Ede             =  https://ftp.bit.nl/pub/OpenBSD
Aachen          =  https://ftp.halifax.rwth-aachen.de/pub/OpenBSD
Stockholm       =  https://ftp.eu.openbsd.org/pub/OpenBSD
Copenhagen      =  https://mirror.one.com/pub/OpenBSD
CloudfareCDN    =  https://cloudflare.cdn.openbsd.org/pub/OpenBSD
LeaseWebCDN     =  https://mirror.leaseweb.com/pub/OpenBSD
PlanetUnixCDN   =  https://mirror.planetunix.net/pub/OpenBSD

status: init
. for site in Alberta Stockholm Nluug Ede Copenhagen Aachen CloudfareCDN LeaseWebCDN PlanetUnixCDN
.  for X in ${FILE}
        @eval SITE=\${${site}} ; printf "\n%s: -- %s --\n" ${site} $${SITE} ;\
        ftp -4 -o ${STORAGE}/${X}-${site} $${SITE}/${DIR}/${X}
.  endfor
. endfor
This predefined, but configurable list of mirrors is used to retrieve the SHA-256, SHA256.sig and index.txt files and these are saved with the name of the site appended like SHA256-Alberta, SHA256.sig-PlanetUnixCDN.

This is done with the following command:
Code:
$ make status

Alberta: -- https://ftp.openbsd.org/pub/OpenBSD --
Trying 129.128.5.191...
Requesting https://ftp.openbsd.org/pub/OpenBSD/snapshots/packages/amd64/SHA256
100% |*****************************************************************************|   890 KB    00:02    
911661 bytes received in 2.50 seconds (354.96 KB/s)

Alberta: -- https://ftp.openbsd.org/pub/OpenBSD --
Trying 129.128.5.191...
Requesting https://ftp.openbsd.org/pub/OpenBSD/snapshots/packages/amd64/SHA256.sig
100% |*****************************************************************************|   890 KB    00:02    
911812 bytes received in 2.47 seconds (360.18 KB/s)

Alberta: -- https://ftp.openbsd.org/pub/OpenBSD --
Trying 129.128.5.191...
Requesting https://ftp.openbsd.org/pub/OpenBSD/snapshots/packages/amd64/index.txt
100% |*****************************************************************************|   846 KB    00:02    
866609 bytes received in 2.46 seconds (343.96 KB/s)

[snip]

PlanetUnixCDN: -- https://mirror.planetunix.net/pub/OpenBSD --
Trying 51.105.228.26...
Requesting https://mirror.planetunix.net/pub/OpenBSD/snapshots/packages/amd64/SHA256
100% |*****************************************************************************|   890 KB    00:00    
911661 bytes received in 0.89 seconds (998.34 KB/s)

PlanetUnixCDN: -- https://mirror.planetunix.net/pub/OpenBSD --
Trying 51.105.228.26...
Requesting https://mirror.planetunix.net/pub/OpenBSD/snapshots/packages/amd64/SHA256.sig
100% |*****************************************************************************|   890 KB    00:00    
911812 bytes received in 0.99 seconds (896.17 KB/s)

PlanetUnixCDN: -- https://mirror.planetunix.net/pub/OpenBSD --
Trying 51.105.228.26...
Requesting https://mirror.planetunix.net/pub/OpenBSD/snapshots/packages/amd64/index.txt
100% |*****************************************************************************|   846 KB    00:00    
866609 bytes received in 0.91 seconds (925.96 KB/s)
BTW: if you redirect this output you also can find out which mirror is the fastest. Compare the download speed of index,txt of Alberta (343.96 KB/s) with the 925.96 KB/s of "planetUnix". So another reason not to use the main Alberta site for the actual downloads/pkg_add's.

Use $ make show to see the result.

Code:
$ make show
866463 Apr 10 23:46 index.txt-Nluug
911658 Apr 10 23:46 SHA256.sig-Nluug
911507 Apr 10 23:46 SHA256-Nluug
866609 Apr 12 12:29 index.txt-Stockholm
866609 Apr 12 12:29 index.txt-PlanetUnixCDN
866609 Apr 12 12:29 index.txt-LeaseWebCDN
866609 Apr 12 12:29 index.txt-Ede
866609 Apr 12 12:29 index.txt-Copenhagen
866609 Apr 12 12:29 index.txt-CloudfareCDN
866609 Apr 12 12:29 index.txt-Alberta
866609 Apr 12 12:29 index.txt-Aachen
911812 Apr 12 12:29 SHA256.sig-Stockholm
911812 Apr 12 12:29 SHA256.sig-PlanetUnixCDN
911812 Apr 12 12:29 SHA256.sig-LeaseWebCDN
911812 Apr 12 12:29 SHA256.sig-Ede
911812 Apr 12 12:29 SHA256.sig-Copenhagen
911812 Apr 12 12:29 SHA256.sig-CloudfareCDN
911812 Apr 12 12:29 SHA256.sig-Alberta
911812 Apr 12 12:29 SHA256.sig-Aachen
911661 Apr 12 12:29 SHA256-Stockholm
911661 Apr 12 12:29 SHA256-PlanetUnixCDN
911661 Apr 12 12:29 SHA256-LeaseWebCDN
911661 Apr 12 12:29 SHA256-Ede
911661 Apr 12 12:29 SHA256-Copenhagen
911661 Apr 12 12:29 SHA256-CloudfareCDN
911661 Apr 12 12:29 SHA256-Alberta
911661 Apr 12 12:29 SHA256-Aachen
This reveals that the Dutch Nluug mirror has not synced yet.

A more recent example where we temporarily use the UTC time zone:
Code:
$ env TZ=UTC make show
866609 Apr 12 10:29 index.txt-Stockholm
866609 Apr 12 10:29 index.txt-PlanetUnixCDN
866609 Apr 12 10:29 index.txt-Nluug
866609 Apr 12 10:29 index.txt-LeaseWebCDN
866609 Apr 12 10:29 index.txt-Ede
866609 Apr 12 10:29 index.txt-Copenhagen
866609 Apr 12 10:29 index.txt-CloudfareCDN
866609 Apr 12 10:29 index.txt-Alberta
866609 Apr 12 10:29 index.txt-Aachen
911812 Apr 12 10:29 SHA256.sig-Stockholm
911812 Apr 12 10:29 SHA256.sig-PlanetUnixCDN
911812 Apr 12 10:29 SHA256.sig-Nluug
911812 Apr 12 10:29 SHA256.sig-LeaseWebCDN
911812 Apr 12 10:29 SHA256.sig-Ede
911812 Apr 12 10:29 SHA256.sig-Copenhagen
911812 Apr 12 10:29 SHA256.sig-CloudfareCDN
911812 Apr 12 10:29 SHA256.sig-Alberta
911812 Apr 12 10:29 SHA256.sig-Aachen
911661 Apr 12 10:29 SHA256-Stockholm
911661 Apr 12 10:29 SHA256-PlanetUnixCDN
911661 Apr 12 10:29 SHA256-Nluug
911661 Apr 12 10:29 SHA256-LeaseWebCDN
911661 Apr 12 10:29 SHA256-Ede
911661 Apr 12 10:29 SHA256-Copenhagen
911661 Apr 12 10:29 SHA256-CloudfareCDN
911661 Apr 12 10:29 SHA256-Alberta
911661 Apr 12 10:29 SHA256-Aachen
This is handy to compare the package build date with the UTC date of the installation sets in the BUILDINFO file.
Code:
$ cat tmp/BUILDINFO_Alberta
Build date: 1618247121 - Mon Apr 12 17:05:21 UTC 2021 - Alberta
The Makefile, but use the attachment to download because you will lose tabs (ASCII 09) characters when you copy and paste, and without tabs in the right places it just won't work)
Code:
# ---------------------------------------------- 
# Retrieve snapshot package mirror sites indices
# ----------------------------------------------
# (c) 2021 j65nko daemonforums.org 
# OpenBSD License: https://cvsweb.openbsd.org/src/share/misc/license.template?rev=HEAD
 
STORAGE = ./pkg

ARCH    = amd64
DIR     = snapshots/${ARCH}
DIR     = snapshots/packages/${ARCH}
FILE    = SHA256 SHA256.sig index.txt

# Mirror sites definitions
# You do not have to use all of them 
# List the ones (or others) you want to use in the line:
#     ". for site in Alberta ..... PlanetUnixCDN"
# Keep in mind that the Alberta site is the main one 
# To alleviate the Alberta site please use a synchronized mirror 

Alberta         =  https://ftp.openbsd.org/pub/OpenBSD
Nluug           =  https://ftp.nluug.nl/OpenBSD
Ede             =  https://ftp.bit.nl/pub/OpenBSD
Aachen          =  https://ftp.halifax.rwth-aachen.de/pub/OpenBSD
Stockholm       =  https://ftp.eu.openbsd.org/pub/OpenBSD
Copenhagen      =  https://mirror.one.com/pub/OpenBSD
CloudfareCDN    =  https://cloudflare.cdn.openbsd.org/pub/OpenBSD
LeaseWebCDN     =  https://mirror.leaseweb.com/pub/OpenBSD
PlanetUnixCDN   =  https://mirror.planetunix.net/pub/OpenBSD

status: init
. for site in Alberta Stockholm Nluug Ede Copenhagen Aachen CloudfareCDN LeaseWebCDN PlanetUnixCDN
.  for X in ${FILE}
        @eval SITE=\${${site}} ; printf "\n%s: -- %s --\n" ${site} $${SITE} ;\
        ftp -4 -o ${STORAGE}/${X}-${site} $${SITE}/${DIR}/${X} 
.  endfor
. endfor

show:
        @ls -ltr ${STORAGE} | awk '{print $$5, $$6, $$7, $$8, $$9}'

init:
.if !exists(${STORAGE})
        mkdir -p ${STORAGE}
.endif

start:
        @date "+%Y_%m%d_%H%M_%Z" >RUNDATE

# --- end of Makefile
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 13th April 2021 at 03:02 AM.
Reply With Quote
  #2   (View Single Post)  
Old 13th April 2021
J65nko J65nko is offline
Administrator
 
Join Date: May 2008
Location: Budel - the Netherlands
Posts: 3,779
Default

The Makefile for downloading. Please rename it from Makefile.txt to Makefile
Attached Files
File Type: txt Makefile.txt (1.6 KB, 13 views)
__________________
You don't need to be a genius to debug a pf.conf firewall ruleset, you just need the guts to run tcpdump

Last edited by J65nko; 13th April 2021 at 03:11 AM.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Checking OpenBSD snapshot mirror status J65nko Guides 3 4th April 2021 11:57 PM
Package mirror for NetBSD 9 Beta? notooth NetBSD Package System (pkgsrc) 2 8th November 2019 09:19 PM
OpenBSD 5.5 snapshot/Gnome 3.10.2 config issues JWJones OpenBSD Packages and Ports 8 18th March 2014 11:50 AM
OpenBSD i386 snapshot & PAE on VirtualBox aleunix OpenBSD Installation and Upgrading 2 1st March 2012 06:44 PM
Apached checking its own status? Mantazz FreeBSD Ports and Packages 5 4th August 2010 04:11 AM


All times are GMT. The time now is 12:39 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick