Monday, April 26, 2010

Complete local copy (mirror) of pkg.opensolaris.org

As you are aware with every official release of OpenSolaris, comes a full download of all the IPS packages (7GB), but what happens if you want to work on the pkg.opensolaris.org/dev releases and you want a local mirror. I have found this blog which does it. I have modified it slight since I think it had a couple of errors.

1. Create partition to store the packages. I have been told a complete will be around 65GB, but I am using a cut down version.
$ pfexec zfs create -o atime=off rpool/export/pkg
2. Create repository
pfexec pkgsend -s file:///export/pkg create-repository -–set-property publisher.prefix=opensolaris.org
3. Obtain list of packages you want.  In this case everything currently install and the release (134) I am interested in.
$ cd /export/pkg
$ pfexec pkg list -Hva | egrep “134:|install” > /tmp/pkglist.txt
$ head -1 /tmp/pkglist.txt
pkg://opensolaris.org/SUNWcs@0.5.11,5.11-0.134:20100302T005446Z

$ pfexec cat /tmp/pkglist.txt | sed 's/pkg:\/\/opensolaris\.org\///' \
       | cut -f 1 -d ' ' > /tmp/packages.txt
$ head -1 /tmp/packages.txt
SUNWcs@0.5.11,5.11-0.134:20100302T005446Z

4.Now download IPS package and place it into local repository.
$ for i in $(cat /tmp/packages.txt); 
do 
pfexec pkgrecv -s http://pkg.opensolaris.org/dev -d file:///export/pkg "$i"
done
5. You can test it to see all is there (http://localhost:80)

$ pfexec /usr/lib/pkg.depotd -d /export/pkg -p 80

6. To make sure the repository is always available.

$ pfexec svccfg -s application/pkg/server setprop pkg/inst_root=/export/pkg
$ pfexec svcadm refresh pkg/server
$ pfexec svcadm enable pkg/server

4 comments:

Anonymous said...

Bound to come in handy. Thanks.

Anonymous said...

I would like to get 129 as well. As that is the current version I am running, not being totally familliar with the repository/package structure what about your post would I need to change, is it just the 134 to 129?

$ cd /export/pkg
$ pfexec pkg list -Hva | egrep “129:|install” > pkglist.txt
$ head -1 pkglist.txt
pkg://opensolaris.org/SUNWcs@0.5.11,5.11-0.129:20100302T005446Z

$ pfexec cat pkglist.txt | sed 's/pkg:\/\/opensolaris\.org\///' \
| cut -f 1 -d ' ' > packages.txt
$ head -1 packages.txt
SUNWcs@0.5.11,5.11-0.129:20100302T005446Z


Thanks for the clean explination and instructions.

Onno Molenkamp said...

If you want to receive the entire 134 distribution, you could also just use

pkgrecv -s http://pkg.opensolaris.org/dev/ -d file:///... pkg:/*

This will include renamed and obsoleted packages, which your commands probably skip because most of them have version 133 instead of 134.

You can use the -n option of pkgrecv to see what packages will be received by this command.

Anonymous said...

Andrew, I am using OpenSolaris snv_134 that's running in a VMware Server 2.0.1 virtual machine. I am trying to create a local repository with a complete copy of IPS packages from pkg.opensolaris.org/release for OpenSolaris.0906 (build 111). I've loosely followed your procedure for building the release 134 local repository, everything goes well until pkgrecv tries to download sunstudio12u1@12.1,5.11-0.111 package:

Retrieving manifests for package evaluation ...
Retrieving package content ...
DOWNLOAD PKGS FILES XFER (MB)
sunstudio12u1@12.1,5.11-0.111:2009... 0/1 4806/5853 453.8/597.5
pkgrecv: Unable to extract file: 46ace7fe165a55e31c125cf996a55878621bb7bd

As it turns out, not even 2.2 GB of RAM was enough for pkgrecv to download this (fairly big) package, /tmp would every time run out of space close to the end of it, as it can be seen from the log. I wonder how much memory you had when you were building a local repository for the development release 134? Is there perhaps a way around this as I am not able to dedicate any more memory to this virtual machine, I would have to take another route?

Regards