File _service:obs_scm:fai-toolkit-1591179917.2894a88... of Package fai-toolkit

Overview Repositories Revisions Requests Users Attributes Meta

File _service:obs_scm:fai-toolkit-1591179917.2894a88.obscpio of Package fai-toolkit (Revision 36077ce53d3a609566d5bb9cb3bfcd8a)

Currently displaying revision 36077ce53d3a609566d5bb9cb3bfcd8a , Show latest

07070100000000000081A4000001C3000001CA000000015ED77A8D00000016000000000000000000000000000000000000002B00000000fai-toolkit-1591179917.2894a88/.kateconfigkate: indent-width 2;
07070100000001000081A4000001C3000001CA000000015ED77A8D000004C3000000000000000000000000000000000000002700000000fai-toolkit-1591179917.2894a88/LICENSEMost of scripts here are derivative of original scripts from FAI:

* create-nfsroot.rb is derivate of fai-make-nfsroot
* create-base.rb is derivate of fai-make-nfsroot
* create-update-cd.rb is derivate of fai-cd.

The original scripts are licensed under GPL-2.0-or-later and hence these
scripts are also available under GPL-2.0-or-later. See more at
https://spdx.org/licenses/GPL-2.0-or-later.html.

Additionally, for all parts that legally can be licensed under ISC,
are licensed under ISC license:

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH REGARD
TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT,
OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
OF THIS SOFTWARE.07070100000002000081A4000001C3000001CA000000015ED77A8D000001F4000000000000000000000000000000000000002900000000fai-toolkit-1591179917.2894a88/README.md# FAI update toolkit

FAI’s default utilities were not useful enough for me, so I went and converted them to Ruby in order to modify them as needed.

For many parts, feature sets of the scripts are smaller than original scripts’, though there is some subtle changes and one feature addition: ability to use own custom initrd in a FAI CD. There is also separate base archive creation script.

# License

See LICENSE. In short, GPL-2.0-or-later and ISC.

# Contributions

Contributions are welcome.07070100000003000041ED000001C3000001CA000000025ED77A8D00000000000000000000000000000000000000000000002300000000fai-toolkit-1591179917.2894a88/bin07070100000004000081A4000001C3000001CA000000015ED77A8D0000010C000000000000000000000000000000000000003800000000fai-toolkit-1591179917.2894a88/bin/create-base-image.rb#!/usr/bin/env ruby

# Creates a base image for FAI to use.

raise "create-base-image must be run as root." unless Process.uid == 0

require "fai-toolkit/create-base-image/base-image"

base_image = CreateBaseImage::BaseImage.new
base_image.options

base_image.perform
07070100000005000081A4000001C3000001CA000000015ED77A8D000001B7000000000000000000000000000000000000003200000000fai-toolkit-1591179917.2894a88/bin/create-base.rb#!/usr/bin/env ruby

# Creates FAI’s nfsroot.

raise "create-nfsroot must be run as root." unless Process.uid == 0

require "fai-toolkit/base-image/create-base"

base = BaseImage::CreateBase.new
base.options

$fai_config_dir = "/etc/fai"
raise "Fai config dir is not a directory" unless Dir.exist? $fai_config_dir

load "#{$fai_config_dir}/nfsroot.conf"
#base.nfsroot = NFSROOT
$FAI_DEBOOTSTRAP_OPTS = FAI_DEBOOTSTRAP_OPTS

base.perform
07070100000006000081A4000001C3000001CA000000015ED77A8D00000119000000000000000000000000000000000000003800000000fai-toolkit-1591179917.2894a88/bin/create-fai-mirror.rb#!/usr/bin/env ruby

# A script to create a mirror FAI uses for installation purposes.

#raise "create-update must be run as root." unless Process.uid == 0

require "fai-toolkit/create-fai-mirror/fai-mirror"

mirror = CreateFaiMirror::FaiMirror.new
mirror.options

mirror.perform

07070100000007000081A4000001C3000001CA000000015ED77A8D000002A4000000000000000000000000000000000000003500000000fai-toolkit-1591179917.2894a88/bin/create-nfsroot.rb#!/usr/bin/env ruby

# Creates FAI’s nfsroot.

raise "create-nfsroot must be run as root." unless Process.uid == 0

require "fai-toolkit/create-nfsroot/create-nfsroot"

nfsroot = Nfsroot::CreateNfsroot.new
nfsroot.options

puts "Using configuration files from #{$fai_config_dir}" if $_verbose
load "#{$fai_config_dir}/fai.conf"
FAI ||= "/var/lib/fai/config"
MNTPOINT ||= "/media/mirror"
$FAI = FAI
$MNTPOINT = MNTPOINT
ENV["FAI"] = $FAI

load "#{$fai_config_dir}/nfsroot.conf"
nfsroot.nfsroot = NFSROOT
#$FAI_CONFIGDIR = FAI_CONFIGDIR
$TFTPROOT = TFTPROOT
$FAI_ROOTPW = FAI_ROOTPW
$FAI_DEBOOTSTRAP_OPTS = FAI_DEBOOTSTRAP_OPTS
$NFSROOT_HOOKS = NFSROOT_HOOKS

nfsroot.perform
07070100000008000081A4000001C3000001CA000000015ED77A8D00000425000000000000000000000000000000000000003700000000fai-toolkit-1591179917.2894a88/bin/create-update-cd.rb#!/usr/bin/env ruby

# A script to create a cd that can be used to boot to update mode.
#
# fai-make-nfsroot doesn’t put xfs to initrd, which is useful for real installations.
# Instead of hackery over a bash script, let’s just create the image by ourselves.

raise "create-update must be run as root." unless Process.uid == 0

require "fai-toolkit/create-update-cd/create-update-cd"

updatecd = Updatecd::CreateUpdateCd.new
updatecd.options

$liveos = "squashfs-root/LiveOS/"

$hidedirs = [ "/usr/share/locale", "/usr/share/doc", "/var/lib/apt", "/var/lib/aptitude", "/var/cache/debconf", "/var/cache/apt", "/var/cache/man", "/usr/share/man", "/var/lib/dpkg/info", "/media/mirror/aptcache" ]

fai_config_dir = "/etc/fai"
raise "#{fai_config_dir} is not a directory" unless Dir.exist? fai_config_dir

#. $fai_config_dir/fai.conf

# This is hackery in order to save some lines and unnecessary parsing; the values in config file must be quoted.
load "#{fai_config_dir}/nfsroot.conf"
updatecd.nfsroot = NFSROOT
$FAI_CONFIGDIR = FAI_CONFIGDIR

updatecd.perform
07070100000009000081A4000001C3000001CA000000015ED77A8D000000CD000000000000000000000000000000000000003C00000000fai-toolkit-1591179917.2894a88/bin/perform-system-update.rb#!/usr/bin/env ruby

# Performs automated system update.

require "fai-toolkit/media/mount"
require "fai-toolkit/system-update/perform"

perform = SystemUpdate::Perform.new
perform.options
perform.perform
0707010000000A000081A4000001C3000001CA000000015ED77A8D00000060000000000000000000000000000000000000003500000000fai-toolkit-1591179917.2894a88/bin/update-from-cd.rb#!/usr/bin/env ruby

require "fai-toolkit/media/mount"

mount = Media::Mount.new
mount.mount_cd
0707010000000B000041ED000001C3000001CA000000095ED77A8D00000000000000000000000000000000000000000000002300000000fai-toolkit-1591179917.2894a88/lib0707010000000C000041ED000001C3000001CA000000025ED77A8D00000000000000000000000000000000000000000000002E00000000fai-toolkit-1591179917.2894a88/lib/base-image0707010000000D000081A4000001C3000001CA000000015ED77A8D00000016000000000000000000000000000000000000003C00000000fai-toolkit-1591179917.2894a88/lib/base-image/base-image.rb
module BaseImage
end
0707010000000E000081A4000001C3000001CA000000015ED77A8D00000609000000000000000000000000000000000000003D00000000fai-toolkit-1591179917.2894a88/lib/base-image/create-base.rb
require "fileutils"
require "tmpdir"

require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "base-image"
require_relative "methods"

class BaseImage::CreateBase
  include Common
  include Output
  include BaseImage::Methods

def perform
    tmpdir = Dir.mktmpdir "create-base_"
    at_exit do
      FileUtils.remove_entry_secure tmpdir
    end
    self.nfsroot= tmpdir

# I find it strange that the cleanup is necessary, but eh... I have a mistake somewhere.
    #umount_dirs

resolve_sources
    create_base

# Wait for compression to be done.
    Process.wait

path = "#{nfsroot}/var/tmp/base.tar.xz"

# Move the file to the location one wants it to be.
    FileUtils.mv path, @base_path
  end

def options
    options = Optimist::options do
      opt :filename,          "Path to the file to be written.",                              type: :string
      opt :force,             "Overwrite existing base image.",                                                   default: false
      opt :debug,             "Show debug output"
      opt :verbose,           "Show verbose output"
    end

raise "--filename is required." if options[:filename].nil?
    @base_path = options[:filename]
    raise "File #{@base_path} already exists. You can use --force to overwrite existing file." if File.exist? @base_path and options[:force] != true

$logfile = File.open("/var/log/fai/create_base.log", "w")
    at_exit do
      $logfile.close
    end

set_opts options
  end
end
0707010000000F000081A4000001C3000001CA000000015ED77A8D00001ABB000000000000000000000000000000000000003900000000fai-toolkit-1591179917.2894a88/lib/base-image/methods.rb
require_relative "base-image"
require_relative "../common"

module BaseImage::Methods
  include Common

# Creates a temporary base archive that is supplied with generated nfsroot.
  def create_base
    raise "Can't find debootstrap command. Aborting." unless command_available? "debootstrap"
    call_debootstrap
    execute "#{rootcmd} apt-get clean"
    execute "rm -f #{nfsroot}/etc/resolv.conf #{nfsroot}/etc/hostname #{nfsroot}/etc/udev/rules.d/70-persistent-net.rules"
    verb "Creating base.tar.xz"
    execute "tar --xattrs --acls --one-file-system -C #{nfsroot} -cf #{nfsroot}/var/tmp/base.tar --exclude etc/machine-id --exclude var/tmp/base.tar --exclude 'var/lib/apt/lists/*_*' ."
    execute "nice xz -q #{nfsroot}/var/tmp/base.tar", wait: false
  end

# put contents of ramdisk back to original location on harddisk
  def umount_ramdisk path

disk = execvalue "readlink -f #{path}"
    ram = "#{disk}-in-ramdisk"

# test, if a ramdisk is mounted to this directory
    if `mount | grep -q " #{disk} type tmpfs"` && !$?.success?
        puts "#{disk} is not a mounted ramdisk."
        exit 1
    end

# Overly cautious check to be sure I don’t accidentally delete root.
    raise "Totally invalid disk var." if disk.strip.size < 3

# There may be a leftover ramdisk from a buggy run.
    if Dir.exist? ram
      # Sometimes there are zombie programs that still are running over the nfsroot.
      # Probably there is better place to kill them than here.
      execute "kill -TERM $(lsof -t /srv/fai/nfsroot)", fail: false, log: false

execute "umount #{ram}", log: false
      FileUtils.rmdir ram
    end

Dir.mkdir ram
    # move current ramdisk to a new location
    execute "mount --bind #{disk} #{ram}", log: false
    execute "umount #{disk}", log: false

FileUtils.rm_rf Dir.glob("#{disk}/*")
    FileUtils.rm_rf Dir.glob("#{disk}/.??*")

# copy ramdisk contents to disk
    execute "cp -a  #{ram}/. #{disk}/.", log: false

execute "umount #{ram}", log: false
    debug "Ramdisk on #{disk} umounted"
    FileUtils.rmdir ram
  end

def umount_dirs
    puts "Performing cleanup..." if $_verbose

Dir.chdir "/"
    execute "umount #{nfsroot}/proc", log: false if Dir.exist? "#{nfsroot}/proc/self"
    execute "umount #{nfsroot}/sys", log: false if Dir.exist? "#{nfsroot}/sys/class"
    raise "/proc still mounted inside the nfsroot." if Dir.exist? "#{nfsroot}/proc/self"
    execute "umount #{nfsroot}/dev/pts", log: false, fail: false

if `mount | grep -q "on #{nfsroot}/var/lib/dpkg type tmpfs"` and $?.success?
      #execute "/usr/lib/fai/mkramdisk -u #{nfsroot}/var/lib/dpkg #{nfsroot}/var/cache", log: false, fail: false
      #execute "/usr/lib/fai/mkramdisk -u #{nfsroot}/var/lib/dpkg #{nfsroot}/var/cache", log: false
      umount_ramdisk "#{nfsroot}/var/lib/dpkg"
      umount_ramdisk "#{nfsroot}/var/cache"
    end
    # A leftover from when this script has had a bug in wrong place.
    if `mount | grep -q "/srv/fai/.will-now-be-deleted/var/lib/dpkg type tmpfs"` and $?.success?
      execute "/usr/lib/fai/mkramdisk -u /srv/fai/.will-now-be-deleted/var/lib/dpkg /srv/fai/.will-now-be-deleted/var/cache", log: false
    end

if !$FAI_DEBMIRROR.nil? and !$FAI_DEBMIRROR.empty?
      if Dir.exist? "#{nfsroot}/#{$MNTPOINT}"
        #execute "umount #{nfsroot}/#{$MNTPOINT}", log: false, fail: false
        execute "umount #{nfsroot}/#{$MNTPOINT}", log: false
      end
      #execute "rmdir #{nfsroot}/#{$MNTPOINT}", fail: false, log: false
      execute "rmdir #{nfsroot}/#{$MNTPOINT}", log: false
    end

# show directories still mounted on nfsroot
    mounted = execvalue %{mount | grep " on #{nfsroot} " || true}
    puts "[WARNING]: Still mounted: #{mounted}" unless mounted.empty?
  end

# Reads target and repository url from /etc/fai/apt/sources.list.
  # Compared to fai-make-nfsroot, this avoids having to pass extra data just for base creation.
  def resolve_sources
    path = "#{$fai_config_dir}/apt/sources.list"
    raise "#{path} does not exist" unless File.exist? path

lines = File.readlines path

# By convention, first real repo is main repo. Do some extra checks anyway to be sure.
    suite = nil
    repourl = nil
    lines.each do |line|
      next unless line[0..2] == "deb"
      splitted = line.split " "
      suite = splitted[2]
      repourl = splitted[1]

break unless suite.include? "/"
      suite = nil
      repourl = nil
    end

raise "Failed to parse #{path}" if suite.nil? or repourl.nil?

@repository_suite = suite
    @repository_url = repourl
  end

def call_debootstrap
    # check if NFSROOT directory is mounted with bad options
    fs = execvalue "df -P #{nfsroot} | tail -1 | awk '{print $6}'"
    if execvalue("mount | grep \"on #{fs} \" |  awk '{print $6}'").match?(/nosuid|nodev/)
      raise "NFSROOT directory #{nfsroot} is mounted using nosuid or nodev. Aborting"
    end

debootstrap_version = execvalue("debootstrap --version").split(" ")[1]
    verb "Creating base system using debootstrap version #{debootstrap_version}"

# Check if we need cross architecture debootstrap
    targetarch = execvalue %{echo "$(expr "#{$FAI_DEBOOTSTRAP_OPTS}" : '.*--arch[=[:space:]]$[^[:space:]]*$' || true)" | tail -n 1}
    hostarch1 = execvalue "dpkg --print-architecture"
    hostarch2 = execvalue "dpkg --print-foreign-architectures"
    hostarch2 = hostarch1 if hostarch2.empty?

debootstrap = "debootstrap"

=begin
    _debootstrap=qemu-debootstrap
    if [ -z "$targetarch" ]; then
        _debootstrap=debootstrap
    else
        if [ $targetarch = $hostarch1 -o $targetarch = $hostarch2 ]; then
            _debootstrap=debootstrap
        fi
    fi
    if [ $_debootstrap = "qemu-debootstrap" ]; then
        if ! command -v $_debootstrap >&/dev/null; then
            die 1 "qemu-debootstrap not found. Please install the package qemu-user-static."
        fi
    fi
=end
    # suite target mirror
    # Calling debootstrap --exclude=wget --include=ca-certificates,gnupg buster /srv/fai/nfsroot http://ryza.smar.fi:3142/debian/debian

verbose = $_verbose ? "--verbose" : ""
    command = "#{debootstrap} #{verbose} #{$FAI_DEBOOTSTRAP_OPTS} #{@repository_suite} #{nfsroot} #{@repository_url}"
    verb "Calling #{command}"
    execute command, fail: false
    unless $?.success?
      puts "ERROR: #{debootstrap} did not complete successfully."
      puts "This is mostly caused by a broken mirror."
      puts "Please check your mirror or use an official mirror."
      puts "Exit code was: #{$?.exitstatus}"
      exit 10
    end

unless File.exist? "#{nfsroot}/usr/bin/apt-get"
      puts "No apt-get executable available inside the NFSROOT."
      puts "Maybe #{debootstrap} did not finish successfully. Aborting."
      exit 11
    end
  end

end
07070100000010000081A4000001C3000001CA000000015ED77A8D00000A94000000000000000000000000000000000000002D00000000fai-toolkit-1591179917.2894a88/lib/common.rb
require_relative "output"

# Common utilities

module Common
  include Output

class CommandError < StandardError
  end

class ExecutionFailed < CommandError
  end

@@opts = nil
  @@nfsroot = nil
  @@rootcmd = nil

def opts
    @@opts
  end

def set_opts options
    @@opts = options

if options[:verbose]
        $verbose = "-v"
        $_verbose = true
    else
        $verbose = ""
    end

$_debug = true if options[:debug]
  end

def rootcmd
    raise "rootcmd is not set." if @@rootcmd.nil?
    @@rootcmd
  end

def rootcmd= cmd
    @@rootcmd = cmd
  end

def nfsroot
    raise "nfsroot is not set." if @@nfsroot.nil?
    @@nfsroot
  end

def nfsroot= path
    set_nfsroot path
  end

def set_nfsroot path
    @@nfsroot = path
    @@rootcmd = %{chroot #{@@nfsroot} env LC_ALL="C"}
  end

# Check if requested program can be found.
  #
  # Should work cross-platform and in restricted environents pretty well.
  def command_available? command
    exts = ENV["PATHEXT"] ? ENV["PATHEXT"].split(';') : ['']
    ENV["PATH"].split(File::PATH_SEPARATOR).each do |path|
      exts.each do |ext|
        exe = File.join(path, "#{command}#{ext}")
        return true if File.executable?(exe) && !File.directory?(exe)
      end
    end
    false
  end

# Returns pid of executed command if wait is false.
  # Returns command output if wait is true.
  #
  # fail == If true, an error is raised in case the command returns failure code.
  def execute command, fail: true, wait: true, log: true, direct_output: false
    debug "Executing “#{command}” at #{caller_locations(1,1)[0]}" if log == true

r, w = IO.pipe
    if direct_output
      pipe_w = w
      w = STDOUT
    end
    pid = spawn(command, err: [:child, :out], out: w)
    w = pipe_w if direct_output
    w.close
    unless wait
      r.close
      return pid
    end
    buffer = String.new capacity: 1024
    output = ""
    while not r.eof?
      r.read_nonblock 1024, buffer
      if log
        print buffer if $_verbose
        Output.logfile.write buffer unless Output.logfile.nil?
      end
      output << buffer
    end
    r.close

# This is instant return, but allows us to have $? set.
    Process.wait pid

raise ExecutionFailed, "Command execution failed.", caller if fail == true and !$?.success?

output.strip
  end

# Same as ::execute, but uses log: false to avoid unnecessary output when we’re just getting a value for internal needs.
  #
  # Defaults to fail: false, since when one just wants to get the value, there is not that much need to unconditionally die if getting bad exit code.
  def execvalue *args, fail: false
    execute *args, fail: fail, log: false
  end
end
07070100000011000041ED000001C3000001CA000000025ED77A8D00000000000000000000000000000000000000000000003500000000fai-toolkit-1591179917.2894a88/lib/create-base-image07070100000012000081A4000001C3000001CA000000015ED77A8D00000E3A000000000000000000000000000000000000004300000000fai-toolkit-1591179917.2894a88/lib/create-base-image/base-image.rbrequire "fileutils"
require "tmpdir"
require "psych"

require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "methods"

# Main class.
class CreateBaseImage::BaseImage
  include Common
  include Output
  include CreateBaseImage::Methods

def perform
    startup_checks
    real_perform
  end

def options
    options = Optimist::options do
      opt :list_distributions,    "Show list of supported distributions"
      opt :foreign_arch,          "Build for given foreign architecture",                           type: :string
      opt :distribution,          "Distribution to build the image from",                           type: :string
      opt :preserve_tmpdir,       "If given, preserves temporary files after script has finished.", type: :boolean, default: false
      opt :fai_config_space,      "Path to FAI’s config space.",                                    type: :string,  default: "/srv/fai/config"
      opt :debug,                 "Show debug output"
      opt :verbose,               "Show verbose output"
    end

if options[:list_distributions]
      print_distributions
      exit 1
    end

if options[:distribution].nil? or options[:distribution].size < 1
      raise "--distribution must be given."
    end

unless Dir.exist? options[:fai_config_space]
      raise "FAI’s config space is used for base image configuration, but config space is missing. Please provide correct config space or create a new one."
    end

@create_base_image_config_file = "#{options[:fai_config_space]}/basefiles/create_base_image.yaml"
    unless File.exist? @create_base_image_config_file
      raise "FAI’s config space is used for base image configuration, but configuration file #{@create_base_image_config_file} is missing. Please add correct config file to config path. For now, you need to investigate this script how to do that."
    end
    lines = File.readlines(@create_base_image_config_file).join("\n")
    @create_base_image_config = Psych.safe_load lines

set_opts options
  end

private

def real_perform
    # These could be used to use other compression algos.
    @ext = "tar.xz"
    @zip = "xz -8"
    @tar_attributes = "--xattrs --selinux --acls"

@tmpdir = Dir.mktmpdir "create-base-image_"
    FileUtils.chmod 755, @tmpdir

# also the name of the output file
    target = opts[:distribution]

case target
    when "OPENSUSE_15_1_32"
      opensuse target, "i386", "15.1"
    when "OPENSUSE_15_1_64"
      opensuse target, "amd64", "15.1"
    when "CENTOS5_32"
      centos target, "i386", 5
    when "CENTOS5_64"
      centos target, "amd64", 5
    when "CENTOS6_32"
      centos target, "i386", 6
    when "CENTOS6_64"
      centos target, "amd64", 6
    when "CENTOS7_32"
      centos target, "i386", 7
    when "CENTOS7_64"
      centos target, "amd64", 7
    when "CENTOS8_64"
      centos target, "amd64", 8
    when "SLC5_32"
      slc target, "i386", 5
    when "SLC5_64"
      slc target, "amd64", 5
    when "SLC6_32"
      slc target, "i386", 6
    when "SLC6_64"
      slc target, "amd64", 6
    when "SLC7_64"
      slc target, "amd64", 7
    when /TRUSTY*|XENIAL*|BIONIC*/
      debgeneric target, @create_base_image_config["MIRROR_UBUNTU"]
    when /SQUEEZE*|WHEEZY*|JESSIE*|STRETCH*|BUSTER*|SID*/
      # TODO: arch
      debgeneric target, @create_base_image_config["MIRROR_DEBIAN"], $ARCH
    else
      puts "Unknown distribution. Aborting."
      exit 1
    end

# cleanup
    if not opts[:preserve_tmpdir] == true
      FileUtils.rm_rf @tmpdir
    end
  end

def startup_checks
    # Nya.
  end

end

07070100000013000081A4000001C3000001CA000000015ED77A8D0000001D000000000000000000000000000000000000004A00000000fai-toolkit-1591179917.2894a88/lib/create-base-image/create-base-image.rb
module CreateBaseImage
end

07070100000014000081A4000001C3000001CA000000015ED77A8D000017D2000000000000000000000000000000000000004000000000fai-toolkit-1591179917.2894a88/lib/create-base-image/methods.rb
require_relative "create-base-image"

module CreateBaseImage::Methods

def setarch arch
    arch == "i386" ? "linux32" : ""
  end

def check
    execute "mknod #{@tmpdir}/test-dev-null c 1 3"
    unless $?.success?
      puts "Cannot create device files on #{@tmpdir}, aborting."
      puts "Perhaps this directory is mounted with option nodev."
      FileUtils.rm_rf @tmpdir
      exit 1
    end

testfile = "#{@tmpdir}/test-dev-null"
    File.open testfile, "w" do |file|
      file.write "test"
    end

unless File.exist? testfile
      puts "Cannot create device files on #{@tmpdir}, aborting."
      puts "Perhaps this directory is mounted with option nodev."
      FileUtils.rm_rf @tmpdir
      exit 1
    end

FileUtils.rm_f testfile
  end

def mkpost_opensuse
    raise "MIRROR_OPENSUSE missing." if @create_base_image_config["MIRROR_OPENSUSE"].empty?

TODO
=begin
    postfile = "#{@tmpdir}/post"
    File.open postfile, "w" do |file|
      file.write <<~EOF
      #!/bin/sh
      mkdir -p #{@tmpdir}/etc/yum.repos.d/orig
      cp -p #{@tmpdir}/etc/yum.repos.d/*.repo #{@tmpdir}/etc/yum.repos.d/orig
      perl -pi -e 's,mirrorlist=,#mirrorlist=,; s,#baseurl=http://mirror.centos.org,baseurl=#{@create_base_image_config["MIRROR_CENTOS"]},;' #{@tmpdir}/etc/yum.repos.d/CentOS-Base.repo
      EOF
    end

FileUtils.chmod 555, postfile
=end
  end

def mkpost_centos
    raise "MIRROR_CENTOS missing." if @create_base_image_config["MIRROR_CENTOS"].empty?

postfile = "#{@tmpdir}/post"
    File.open postfile, "w" do |file|
      file.write <<~EOF
      #!/bin/sh
      mkdir -p #{@tmpdir}/etc/yum.repos.d/orig
      cp -p #{@tmpdir}/etc/yum.repos.d/*.repo #{@tmpdir}/etc/yum.repos.d/orig
      perl -pi -e 's,mirrorlist=,#mirrorlist=,; s,#baseurl=http://mirror.centos.org,baseurl=#{@create_base_image_config["MIRROR_CENTOS"]},;' #{@tmpdir}/etc/yum.repos.d/CentOS-Base.repo
      EOF
    end

FileUtils.chmod 555, postfile
  end

def mkpost_slc version
    raise "MIRROR_SLC missing" if @create_base_image_config["MIRROR_SLC"].empty?

postfile = "#{@tmpdir}/post"
    File.open postfile, "w" do |file|
      file.write <<~EOF
      #! /bin/sh
      mkdir -p #{@tmpdir}/etc/yum.repos.d/orig
      cp -p #{@tmpdir}/etc/yum.repos.d/*.repo #{@tmpdir}/etc/yum.repos.d/orig
      perl -pi -e 's,baseurl=http://linuxsoft.cern.ch,baseurl=#{@create_base_image_config["MIRROR_SLC"]},;' #{@tmpdir}/etc/yum.repos.d/slc#{version}-os.repo
      perl -pi -e 's,baseurl=http://linuxsoft.cern.ch,baseurl=#{@create_base_image_config["MIRROR_SLC"]},;' #{@tmpdir}/etc/yum.repos.d/slc#{version}-updates.repo
      EOF
    end

FileUtils.chmod 555, postfile
  end

def cleanup_deb
    execute "chroot #{@tmpdir} apt-get clean"
    FileUtils.rm_f [ "#{@tmpdir}/etc/hostname", "#{@tmpdir}/etc/resolv.conf",
          "#{@tmpdir}/var/lib/apt/lists/*_*", "#{@tmpdir}/usr/bin/qemu-*-static",
          "#{@tmpdir}/etc/udev/rules.d/70-persistent-net.rules" ]
    File.truncate "#{@tmpdir}/etc/machine-id", 0
  end

def cleanup_rinse

# check if chroot works
    puts "Installed packages in chroot:"
    puts execvalue "chroot #{@tmpdir} rpm -qa|sort"
    print "CHROOT rpm -qa: "
    puts execvalue "chroot #{@tmpdir} rpm -qa|wc -l"

FileUtils.rm_f [ "#{@tmpdir}/etc/resolv.conf", "#{@tmpdir}/post" ]
    if Dir.exist? "#{@tmpdir}/etc/yum.repos.d/orig"
        execute "mv #{@tmpdir}/etc/yum.repos.d/orig/* #{@tmpdir}/etc/yum.repos.d/"
        FileUtils.rm_rf "#{@tmpdir}/etc/yum.repos.d/orig"
    end
  end

def tarit filename
    execute "tar #{@tar_attributes} --numeric-owner --one-file-system -C #{@tmpdir} -cf - . | #{@zip} > #{filename}.#{@ext}"
  end

def opensuse distribution, arch, version
    check
    l32 = setarch arch
    #mkpost_opensuse # TODO: It looks like something not useful... Could do same feature with --mirror directly?
    execute "#{l32} rinse --directory #{@tmpdir} --distribution opensuse-#{version} --arch #{arch} --before-post-install #{@tmpdir}/post"
    cleanup_rinse
    tarit distribution
  end

def centos distribution, arch, version
    domain = execvalue "domainname"

check
    l32 = setarch arch
    mkpost_centos
    execute "#{l32} rinse --directory #{@tmpdir} --distribution centos-#{version} --arch #{arch} --before-post-install #{@tmpdir}/post"
    execute "domainname domain" # workaround for #613377
    cleanup_rinse
    tarit distribution
  end

def slc distribution, arch, version

check
    l32 = setarch arch
    mkpost_slc version
    execute "#{l32} rinse --directory #{@tmpdir} --distribution slc-#{version} --arch #{arch} --before-post-install #{@tmpdir}/post"
    cleanup_rinse
    tarit distribution
  end

def debgeneric distribution, mirror, arch

raise "Not done. You probably should use fai’s example script for Debian instead."

# TODO: no idea what to do with this dist thing.

#dist=${DIST%%[0-9][0-9]}
    #local exc="EXCLUDE_$dist"
    #[ -n "${!exc}" ] && exc="--exclude=${!exc}" || unset exc
    #dist=${dist,,}

check

#if [ -n "$INCLUDE_DEBIAN" ]; then
    #  local inc="--include=$INCLUDE_DEBIAN"
    #fi

unless arch.empty?
      execute "qemu-debootstrap --arch #{arch} #{exc} #{inc} #{distribution} #{@tmpdir} #{mirror}"
      # If this gets implemented, this target needs to be handled.
      target = "#{target}_#{arch}"
    else
      arch = "i386"
      arch = "amd64" if distribution.include? "64"
      execute "debootstrap --arch #{arch} #{exc} #{inc} #{distribution} #{@tmpdir} #{mirror}"
    end
    cleanup_deb
    tarit distribution
  end

def print_distributions

puts <<EOF
Available:

CENTOS5_32   CENTOS5_64
    CENTOS6_32   CENTOS6_64
    CENTOS7_32   CENTOS7_64
                 CENTOS8_64
    SLC5_32      SLC5_64
    SLC6_32      SLC6_64
                 SLC7_64
    TRUSTY32     TRUSTY64
    XENIAL32     XENIAL64
                 BIONIC64
    SQUEEZE32    SQUEEZE64
    WHEEZY32     WHEEZY64
    JESSIE32     JESSIE64
    STRETCH32    STRETCH64
    BUSTER32     BUSTER64
    SID32        SID64
EOF
  end

end
07070100000015000041ED000001C3000001CA000000025ED77A8D00000000000000000000000000000000000000000000003500000000fai-toolkit-1591179917.2894a88/lib/create-fai-mirror07070100000016000081A4000001C3000001CA000000015ED77A8D0000001C000000000000000000000000000000000000004A00000000fai-toolkit-1591179917.2894a88/lib/create-fai-mirror/create-fai-mirror.rb
module CreateFaiMirror
end
07070100000017000081A4000001C3000001CA000000015ED77A8D000018C7000000000000000000000000000000000000003F00000000fai-toolkit-1591179917.2894a88/lib/create-fai-mirror/debian.rbrequire "fileutils"

require_relative "create-fai-mirror"
require_relative "../common"

class CreateFaiMirror::Debian
  include Common

# Installs packages to the mirror.
  def install_packages package_list
    execute "apt-get -d #{@aptoptions} -y --fix-missing install #{package_list}"
  end

def refresh_repositories
    execute "apt-get #{@aptoptions} update", log: false
  end

def _post_initialize mirror_directory

# TODO: root is only needed when FAI_DEBMIRROR is defined. Then we must mount a directory.

# holds the package cache data
    @aptcache = "#{mirror_directory}/aptcache"

@archivedir = "#{@aptcache}/var/cache/apt/archives"
    statefile = "#{@aptcache}/statefile"

# also used in install_packages.conf
    @aptoptions = <<~EOF
      -o Aptitude::Log=/dev/null
      -o Aptitude::CmdLine::Ignore-Trust-Violations=yes
      -o APT::Get::AllowUnauthenticated=true
      -o Acquire::AllowInsecureRepositories=true
      -o DPkg::force-conflicts::=yes
      -o Dir::State=#{@aptcache}/var/lib/apt
      -o Dir::Log=#{@aptcache}/var/log/apt
      -o Dir::State::extended_states=#{@aptcache}/var/lib/apt/lists/extended_states
      -o Dir::State::status=#{statefile}
      -o Dir::Cache=#{@aptcache}/var/cache/apt
      -o Dir::State=#{@aptcache}/var/cache/apt
      -o Dir::Cache::Archives=#{@archivedir}
      -o Dir::Etc=#{@aptcache}/etc/apt/
      -o Dir::State::Lists=#{@aptcache}/var/lib/apt/lists/
    EOF
    @aptoptions.gsub! "\n", " "

# Used by install_packages.
    ENV["FAI_ARCHIVE_DIR"] = @archivedir

# Add architecture.
    unless $arch.empty?
      @aptoptions += " -o APT::Architecture=#{$arch}"
    end

# we only need some empty dirs
    FileUtils.mkdir_p [ "archivedir/partial", "#{@aptcache}/etc/apt/preferences.d", "#{@aptcache}/var/lib/apt/lists/partial", "#{@aptcache}/var/log/apt" ]
    File.open statefile, "w" do |f|
      # Empties statefile.
    end
  end

def startup_checks
    raise "reprepro not found. Please install the package." unless command_available? "reprepo"
  end

# Hooks to make it visually easier to understand the flow.

def _pre_real_perform fai_config_dir, mirror_location
    # if we are using nfs mounts for Debian mirror, this may fail here, since inside a chroot environment different dir are used
    # if sources.list includes file AND FAI_DEBMIRROR is defined we Mhave to mount
    # otherwise mounting is not needed. call task_mirror
    unless $FAI_DEBMIRROR.empty?
        FileUtils.mkdir_p @mirror_location
        execute "mount -r #{$FAI_DEBMIRROR} #{@mirror_location}"
    end

# TODO: use -p to preserve sources.list
    execute "sed -e 's/file:/copy:/' #{fai_config_dir}/apt/sources.list > #{@aptcache}/etc/apt/sources.list"
    test = execvalue "ls -A #{fai_config_dir}/apt/sources.list.d/ 2> /dev/null"
    unless test.empty?
      execute "sed -e 's/file:/copy:/' #{fai_config_dir}/apt/sources.list.d/* >> #{@aptcache}/etc/apt/sources.list"
    end

# aptpref comes from cli arg on original script. Not implemented AFAIR
    #if File.exist? $aptpref
    #  FileUtils.cp $aptpref, "#{@aptcache}/etc/apt/preferences"
    #end

# use the trusted keys from the host
    execute "cp -a /etc/apt/trusted.gpg.d #{@aptcache}/etc/apt"
  end

def _post_real_perform mirror_directory
    # in still undefined, use host architecture
    if $arch.empty?
      $arch = execvalue "dpkg  --print-architecture"
      $arch = "#{$arch} #{execvalue "dpkg --print-foreign-architectures"}"
    end

# create mirror directory structure
    verb "Calling reprepro"
    FileUtils.mkdir "#{mirror_directory}/conf"
    File.open "#{mirror_directory}/conf/distributions", "w" do |f|
      f.write <<~EOF
      # generate config file for reprepro
      Codename: cskoeln
      Architectures: #{$arch}
      Components: main non-free contrib
      DebIndices: Packages Release . .xz
      Description: Package reprository created by fai-mirror
      Label: fai-project.org
      Origin: fai-mirror
      EOF
    end

# check if backports are used
    bponame = execvalue "grep -E '^deb ' #{@aptcache}/etc/apt/sources.list | grep -P -o '\S+-backports'"

# add entry for backports
    unless bponame.empty?
      File.open "#{mirror_directory}/conf/distributions", "w" do |f|
        f.write <<~EOF

Codename: #{bponame}
        Architectures: #{$arch}
        Components: main non-free contrib
        DebIndices: Packages Release . .xz
        Description: Package reprository created by fai-mirror
        Label: fai-project.org
        Origin: fai-mirror
        EOF
      end

# maybe using reprepro pulls it's possible to move instead of copy the packages
      execvalue = "ls #{@archivedir}/*~bpo[0-9]*+[0-9-]*_*.deb"
      if $?.success and !execvalue.empty?
        execute "reprepro -b #{mirror_directory} includedeb #{$bponame} #{@archivedir}/*~bpo[0-9]*+[0-9-]*_*.deb"
        FileUtils.rm_f Dir.glob("#{@archivedir}/*~bpo[0-9]*+[0-9-]*_*.deb")
      end
    end

# allow package without the .deb suffix. At least they must start with a char and contain an underscore or hyphen
    execute "reprepro --ignore=extension -b #{mirror_directory} includedeb cskoeln #{@archivedir}/[a-zA-z]*[_-]*"
    FileUtils.rm_r "#{@archivedir}/[a-zA-z]*[_-]*"
  end

# Quasi-hooks

def download_packages_for_classes classes, fai_config_space
    verb "Downloading packages for classes: #{classes}"
    execute "FAI=#{fai_config_space} install_packages -N -d #{$vflag}"
    unless $?.success?
        raise "ERROR when downloading packages. Your mirror probably is broken."
    end
  end

def use_temporary_sources_list
    # Save original sources.list
    FileUtils.mv "#{@aptcache}/etc/apt/sources.list", @aptcache

# now use different sources.list for debootstrap packages
    execute %{echo "$FAI_DEBOOTSTRAP" | awk '{printf "deb %s %s main\n",$2,$1}' | sed -e 's/file:/copy:/' > #{@installer.aptcache}/etc/apt/sources.list}
  end

def restore_sources_list
    FileUtils.mv "#{@aptcache}/sources.list", "#{@aptcache}/etc/apt/sources.list"
    refresh_repositories
  end

def try_base_package_delete p, arch
    if Dir.glob("#{@archivedir}/#{p}_*#{arch}.deb").size > 0
      debug "deleting package #{p} #{arch}"
      FileUtils.rm Dir.glob("#{@archivedir}/#{p}_*#{arch}.deb")
    end
  end
end07070100000018000081A4000001C3000001CA000000015ED77A8D00001961000000000000000000000000000000000000004300000000fai-toolkit-1591179917.2894a88/lib/create-fai-mirror/fai-mirror.rb
require "optimist"

require_relative "create-fai-mirror"
require_relative "methods"
require_relative "debian"
require_relative "suse"

class CreateFaiMirror::FaiMirror
include Common
  include Output
  include CreateFaiMirror::Methods

def perform
    startup_checks
    real_perform
  end

# TODO: add some way to update existing mirror.
  def options
    options = Optimist::options do
      opt :fai_config_dir,    "FAI’s system configuration directory",                   type: :string,    default: "/etc/fai",            short: "c"
      opt :fai_config_space,  "FAI’s config space",                                     type: :string,    default: "/srv/fai/config",     short: "s"
      opt :mirror_location,   "Directory where mirror will be generated to",            type: :string,    default: "/srv/fai/mirror",     short: "m"
      opt :force,             "Overwrite existing mirror",                              type: :boolean,   default: false,                 short: "f"
      opt :target,            "Target distribution. Supported values: debian, suse",    type: :string,                                    short: "t"
      opt :exclude_class,     "Excludes given class if it otherwise would be present",  type: :strings,                                   short: "x"
      #opt :set_variables,     "Reads and sets variables from class/*.var if used in package_config/<classes>",                            short: "V"
      opt :nfsroot,           "FAI nfsroot location",                                   type: :string,    default: "/srv/fai/nfsroot",    short: "n"
      opt :update,            "Update instead of creating new mirror",                  type: :boolean,   default: false,                 short: "u"
      opt :debug,             "Show debug output",                                                                                        short: "d"
      opt :verbose,           "Show verbose output",                                                                                      short: "v"
    end

unless Dir.exist? options[:fai_config_dir]
      raise "#{options[:fai_config_dir]} does not exist. Please create it or specify alternative path with --fai-config-dir."
    end
    @fai_config_dir = options[:fai_config_dir]

unless Dir.exist? options[:nfsroot]
      raise "#{options[:nfsroot]} does not exist. Please create it or specify alternative path with --nfsroot."
    end
    set_nfsroot options[:nfsroot]

@target_distribution = options[:target]
    case @target_distribution
    when "debian"
      raise "apt-get is missing. Are you sure you are on Debian?" unless command_available? "apt-get"
      @installer = CreateFaiMirror::Debian.new
    when "suse"
      raise "zypper is missing. Are you sure you are on SUSE?" unless command_available? "zypper"
      @installer = CreateFaiMirror::Suse.new
    else
      raise "Invalid target distribution: #{@target_distribution.inspect}"
    end

if options[:force] and options[:update]
      raise "Only --force or --update can be given."
    end

if options[:mirror_location].size < 3
      raise "Please specify proper mirror location. Root is not allowed for safety."
    end
    if Dir.exist? options[:mirror_location]
      if options[:force]
        FileUtils.rm_r "#{options[:mirror_location]}/#{@target_distribution}"
      elsif options[:update]
        # Updating, so there is nothing else to do.
      else
        raise "Mirror #{options[:mirror_location]} already exists. If you want to redo it, use --force."
      end
    else
      if options[:update]
        raise "For --update, mirror dir #{options[:mirror_location]} needs to exist."
      else
        FileUtils.mkdir options[:mirror_location]
      end
    end
    @mirror_location = options[:mirror_location]

@excluded_classes = options[:exlude_class]

unless Dir.exist? "#{options[:fai_config_space]}/class"
      raise "FAI config space seems to be missing or invalid. Create a valid config space or specify path to one with --fai-config-space."
    end

set_opts options
  end

private

def real_perform

@installer._pre_real_perform @fai_config_dir, @mirror_location

verb "Getting package information"
    @installer.refresh_repositories

# This is basic operation, i.e. no -B or -b flags in fai-mirror.
    # TODO: if those flags are added, this needs to be guarded.
    #[ $add -eq 1 ] && add_base_packages
    add_base_packages

@installer.download_packages_for_classes @classes, @fai_config_space

umount_dirs

# TODO: implement?
    #[ $add -eq 0 ] && delete_base_packages

@installer._post_real_perform @mirror_directory

verb "create-fai-mirror finished."
    if opts[:debug]
      debugdf = execvalue "du -sh --exclude aptcache --exclude db #{@mirror_directory}"
      debug "Mirror location: #{@mirror_directory.inspect}"
      debug "Mirror size: #{debugdf}"
    end
    cleandirs
  end

def startup_checks
    at_exit do
      umount_dirs
    end

#verb "Using configuration files from #{@fai_config_dir}"
    #[ $bcount -gt 1 ] && die 7 "You can't use -b and -B simultaneously."
    #. #{@fai_config_dir}/fai.conf
    #. #{@fai_config_dir}/nfsroot.conf

@installer.startup_checks

#[ -n "$exclasses" -a -n "$cclasses" ] && die 3 "Options -x and -c not allowed at the same time."

# use first argument if given, use variable mirrordir if not argument was given
    #[ -n "$1" ] && mirrordir=$1
    #[ -z "@mirror_directory" ] && die 2 "Please give the absolute path to the mirror."
    #{ echo @mirror_directory | grep -E -q '^/'; } || die 4 "Mirrordir must start with a slash /."

@mirror_directory = "#{@mirror_location}/#{@target_distribution}"
    FileUtils.mkdir @mirror_directory unless opts[:update]

"package_config/ missing in #{@fai_config_pace}" unless Dir.exist? "#{@fai_config_space}/package_config"

# set default if undefined
    # TODO: handle MAXPACKAGES
    #: ${MAXPACKAGES:=1}
    #export MAXPACKAGES

set_classes

# TODO: implementing this is not easy, since it requires some conversion from bash to ruby.
    # Probably those files could be read and parsed manually, but not doing that now.
=begin
    if opts[:set_variables]
      variable_files = mk_class_var_list
      set -a
      for f in $flist; do
        echo Sourcing variable definitions in $f
        source $flist
      end
      set +a
    end
=end

@installer._post_initialize @mirror_directory
  end
end07070100000019000081A4000001C3000001CA000000015ED77A8D00001284000000000000000000000000000000000000004000000000fai-toolkit-1591179917.2894a88/lib/create-fai-mirror/methods.rb
require "fileutils"

require_relative "create-fai-mirror"

module CreateFaiMirror::Methods

# removes/excludes all classes in $* from @classes
  def exclude_classes excluded_classes
    @classes.delete_if do |c|
      excluded_classes.include? c
    end
  end

def umount_dirs
    return if $FAI_DEBMIRROR.nil? or $FAI_DEBMIRROR.empty?
    execute "umount #{@mirror_location}", fail: false, log: false
  end

def cleandirs
    return # currently nothing to do
  end

# now delete all packages that are already included in base.tar.xz
  def delete_base_packages
    base_packages_file = "#{nfsroot}/var/tmp/base-pkgs.lis"
    unless File.exist? base_packages_file
        debug "#{base_packages_file} not available."
        verb "Can't remove wasteful packages that are already in base.tar.xz."
        return
    end

verb "Removing packages that are already included in base.tar.xz"

File.readlines(base_packages_file).each do |p|
        all = p.gsub(":", " ").split(" ")
        p = all[0]
        arch = all[1]

@installer.try_base_package_delete p, arch
    end
  end

# add packages from base.tar.xz and additional packages in nfsroot
  def add_base_packages

# arch dependent packages defined in fai-make-nfsroot
    verb "Adding packages of #{@fai_config_dir}/NFSROOT."
    if File.exist? "#{nfsroot}/var/tmp/packages.nfsroot"
        plist = File.readlines("#{nfsroot}/var/tmp/packages.nfsroot")[0].split(" ")
        @installer.install_packages plist
    else
        puts.STDERR "WARNING: #{nfsroot}/var/tmp/packages.nfsroot does not exist."
        puts.STDERR "Can't add those packages. Maybe the nfsroot is not yet created."
        exit 1
    end

@installer.use_temporary_sources_list

base_packages_file = "#{nfsroot}/var/tmp/base-pkgs.lis"
    verb "Adding packages from #{base_packages_file}"
    if File.exist? base_packages_file
        plist = File.readlines base_packages_file
        plist.map! do |package|
          package.strip
        end
        @installer.refresh_repositories
        @installer.install_packages plist
    else
        puts.STDERR "WARNING: #{base_packages_file} does not exist."
        puts.STDERR "Can't add those packages. Maybe the nfsroot is not yet created."
        exit 1
    end

@installer.restore_sources_list
  end

def set_classes

# if -c is given, ignore -x
    unless $cclasses.nil? or $cclasses.empty?
      @classes = $cclasses
      return
    end

# all available file names are classes
    classes = execvalue %{cd #{@fai_config_space}/package_config; ls -1 | grep -E -i "^[a-zA-Z0-9_-]+$"}
    addclasses = execvalue %{grep -h PACKAGES #{@fai_config_space}/package_config/* | sed -e 's/#.*//' | awk '{printf $3"\n"$4"\n"$5"\n"$6"\n"}'}
    @classes = execvalue %{echo -e "#{classes}\n#{addclasses}\n" | sort | uniq}

exclude_classes @excluded_classes unless @excluded_classes.nil? or @excluded_classes.empty?
  end

# create a list of files named class/*.var that contain a definition of a  variable that is
  # used in package_config/<classes>

# we assume, that variables used in package_config are defined in class/*.var.
  # For all classes search if there's a variables used in package_config/CLASS
  # For all variables found look which files class/CLASS.var define these variables.
  def mk_class_var_list

filelist = ""

#filelist: all files in package_config which are also a defined class
    @classes.each do |c|
      if File.exist? "#{@fai_config_space}/package_config/#{c}"
        filelist+=" $c"
      end
    end

return if filelist.empty?

# make the list of variables that are used
    # regex: $abc or ${abc}
    variables = execvalue(%{($(cd #{@fai_config_space}/package_config;grep -h -P -o '\$[A-Za-z0-9_]+|\$\{[A-Za-z0-9_]+\}' #{$filelist}|sort|uniq))}).split("\n")

# debug
    #variables.each do |variable|
    #  debug "Variable found: #{variable}"
    #end

# create a regex like varname= for all variables

variables.map! do |variable|
      variable.gsub! /[{}$]/, ""
      variable += "="
    end

classlist = []

# classlist: make list of files in class, which are also a class
    @classes.each do |c|
      if File.exist? "#{@fai_config_space}/class/#{c}.var"
        classlist << "#{@fai_config_space}/class/#{$c}.var"
      end
    end
    # look for variable definition in this list

sourcefiles = []

classlist.each do |f|
      variables.each do |variable|
        execute %[grep -q -P "#{variable}" #{f}], log: false, fail: false
        if $?.success
          sourcefiles << f
          break
        end
      end
    end

# these files contain those variable definitions which are needed
    sourcefiles
  end
end0707010000001A000081A4000001C3000001CA000000015ED77A8D000004DB000000000000000000000000000000000000003D00000000fai-toolkit-1591179917.2894a88/lib/create-fai-mirror/suse.rbrequire "fileutils"

require_relative "create-fai-mirror"
require_relative "../common"

class CreateFaiMirror::Suse
  include Common

# Installs packages to the mirror.
  def install_packages package_list
    execute %[zypper #{@zypper_options} download #{package_list.join(" ")}], fail: false
    if $?.exitstatus == 106
      puts "WARNING: a repository sync failed. It may mean that this command needs to be run as root."
    end
  end

def refresh_repositories
    #TODO
  end

def _post_initialize mirror_directory
    #TODO

raise "For some reason mirror_directory is invalid..." if mirror_directory.empty?

@zypper_options = <<~EOF
    --non-interactive
    --cache-dir=#{mirror_directory}
    EOF
    @zypper_options.gsub! "\n", " "
  end

def startup_checks
    #TODO
  end

# Hooks to make it visually easier to understand the flow.

def _pre_real_perform fai_config_dir, mirror_location
    #TODO
  end

def _post_real_perform mirror_directory
    #TODO
  end

# Quasi-hooks

def download_packages_for_classes classes, fai_config_space
    #TODO
  end

def use_temporary_sources_list
    #TODO
  end

def restore_sources_list
    #TODO
  end

def try_base_package_delete p, arch
    #TODO
  end

end
0707010000001B000041ED000001C3000001CA000000025ED77A8D00000000000000000000000000000000000000000000003200000000fai-toolkit-1591179917.2894a88/lib/create-nfsroot0707010000001C000081A4000001C3000001CA000000015ED77A8D0000082D000000000000000000000000000000000000004400000000fai-toolkit-1591179917.2894a88/lib/create-nfsroot/create-nfsroot.rb
require "fileutils"
require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "methods"
require_relative "execute"

# Main class.
class Nfsroot::CreateNfsroot
  include Common
  include Output
  include Nfsroot::Methods

def perform
    startup_checks
    execute = Nfsroot::Execute.new
    execute.perform
  end

def options
    options = Optimist::options do
      opt :fai_config_dir,    "FAI’s main config files.",                                     type: :string,      default: "/etc/fai"
      opt :all_packages,      "Installs full set of packages to nfsroot.",                                        default: true
      opt :refresh,           "Installs new packages from NFSROOT and updates all packages."
      opt :force,             "Removes existing nfsroot before installing a new one.",                            default: false,         short: :f
      opt :base_archive,      "Use existing base archive instead of creating a new one."
      opt :debug,             "Show debug output"
      opt :verbose,           "Show verbose output"
    end

raise "Given --fai-config-dir is not valid." if not File.exist? options[:fai_config_dir]
    $fai_config_dir = options[:fai_config_dir]

#raise "--mirror-dir is required." if opts[:mirror_dir].nil?
    #raise "--mirror-dir needs to be a valid directory." unless Dir.exist? opts[:mirror_dir]
    #$mirrordir = opts[:mirror_dir]

$all_packages = options[:all_packages]

$logfile = File.open("/var/log/fai/fai-make-nfsroot.log", "w")
    at_exit do
      $logfile.close
    end

set_opts options

$do_log = false
  end

private

def startup_checks
    raise "NFSROOT needs to be defined in nfsroot.conf" if nfsroot.empty?
    raise "TFTPROOT needs to be defined in nfsroot.conf" if $TFTPROOT.empty?

raise "#{$fai_config_dir}/apt does not exist." unless Dir.exist? "#{$fai_config_dir}/apt"

$oldnfsroot = nfsroot
    $deldir = nfsroot

# Required so that apt-get won’t try to run debconf.
    ENV["DEBIAN_FRONTEND"] = "noninteractive"
  end

end
0707010000001D000081A4000001C3000001CA000000015ED77A8D00000E43000000000000000000000000000000000000003D00000000fai-toolkit-1591179917.2894a88/lib/create-nfsroot/execute.rb
require_relative "nfsroot"
require_relative "methods"

class Nfsroot::Execute
  include Nfsroot
  include Nfsroot::Methods

def perform
    at_exit do
      $do_log = false
      $logfile.close
      umount_dirs
    end

real_perform
  end

private

def perform_refresh
    verb "Upgrading nfsroot and installing new packages into the nfsroot."

execute "#{rootcmd} apt-get update"
    execute "#{rootcmd} apt-get -y dist-upgrade"
    add_packages_nfsroot
    #setup_tftp
    #run_hooks
    #umount_dirs
    #trap "true" EXIT
    #echo "fai-make-nfsroot finished $merror."
    #exit $exitcode
    exit
  end

def initialize_logging
    # repeat message, so it will appear in the log file
    FileUtils.rm_f "/var/log/fai/fai-make-nfsroot.log"
    first_line = "Using configuration files from #{$fai_config_dir}"
    verb first_line
    execute "echo #{first_line} > /var/log/fai/fai-make-nfsroot.log"

# Commands to be logged follows
    $do_log = true
  end

def initialize_nfsroot_dir

if Dir.exist? nfsroot and !opts[:force]
      STDERR.puts "-----------------------------------------------------------------"
      STDERR.puts "Error: #{nfsroot} exists already."
      STDERR.puts "Neither force nor update option present, exiting."
      STDERR.puts "-----------------------------------------------------------------"
      STDERR.puts "NOTE: Use -f option to force overwriting an existing nfsroot."
      STDERR.puts "      Use -k option to update/install packages defined in NFSROOT config."
      exit 1
    end

verb "Creating FAI nfsroot in #{nfsroot}"

if Dir.exist? "#{nfsroot}/proc"
      # Since the umount seems to be unfinished in some cases, let’s do it here too...
      umount_dirs

execute "umount #{nfsroot}/dev/pts >&/dev/null", fail: false
      execute "umount #{nfsroot}/proc", fail: false if File.symlink? "#{nfsroot}/proc/self"
      raise "/proc is still mounted inside the nfsroot." if File.symlink? "#{nfsroot}/proc/self"
      execute "umount #{nfsroot}/#{$MNTPOINT}  2>/dev/null", fail: false # it's safer to try to umount
      # remove old nfsroot using a background process
      execute "rm -rf #{$deldir}/../.will-now-be-deleted", fail: false
      execute "mv #{$deldir} #{$deldir}/../.will-now-be-deleted", fail: false
      unless $?.success?
        execute "rm -rf #{$deldir}"
      else
        execute "nice rm -rf #{$deldir}/../.will-now-be-deleted", wait: false
      end
    end
  end

def real_perform

if opts[:refresh]
      return perform_refresh
    end

initialize_logging
    begin
      initialize_nfsroot_dir
    rescue ExecutionFailed
      # There is some bug that requires a cycle. Let’s just emulate that to see if it helps.
      umount_dirs
      initialize_nfsroot_dir
    end

# These are from Methods.
    resolve_sources
    create_nfsroot
    #setup_tftp
    setup_ssh
    #run_hooks
    check_nfsroot
    write_variables

=begin
    if [ $generic = 1 ]; then
        # remove fai.conf which comes from package fai-client
        rm -f $NFSROOT/etc/hosts $NFSROOT/etc/resolv.conf* $NFSROOT/etc/fai/fai.conf $NFSROOT/var/lib/apt/lists/*_*
        ainsl -as $NFSROOT/etc/hosts "127.0.0.1 localhost"
    fi
=end

umount_dirs

=begin
    # Wait for background jobs to finish
    BGJOBS=$(jobs -p)
    if [ -n "$BGJOBS" ]; then
        echo "Waiting for background jobs to finish"
        jobs
        for job in $BGJOBS; do
            wait $job
        done
    fi
=end
    Process.waitall

verb "fai-make-nfsroot finished."
    opts[:verbose] and puts "Log file written to /var/log/fai/fai-make-nfsroot.log"
  end
end
0707010000001E000081A4000001C3000001CA000000015ED77A8D00003348000000000000000000000000000000000000003D00000000fai-toolkit-1591179917.2894a88/lib/create-nfsroot/methods.rb
require_relative "nfsroot"
require_relative "../common"
require_relative "../base-image/methods"

module Nfsroot::Methods
  include Common
  include BaseImage::Methods

def setclasses

#local iarch distro c
    # Export all the information we have about the system
    iarch = execvalue "#{rootcmd} dpkg --print-architecture|tr /a-z/ /A-Z/"
    distro = execvalue ". #{nfsroot}/etc/os-release; echo ${ID} ${ID}_${VERSION_ID} | tr '[:lower:].' '[:upper:]_'"

# do we want to install the full list of packages?
    c = "FULL" if $all_packages

$classes="NFSROOT #{c} #{iarch} #{distro} #{$nonfree}"
    ENV["classes"] = $classes
    verb "Classes are set to #{$classes}"
  end

def add_packages_nfsroot

ENV["FAI_ROOT"] = nfsroot

setclasses
    execute "#{rootcmd} apt-cache dumpavail > #{nfsroot}/var/lib/dpkg/available", log: false
    execute %{env LC_ALL="C" install_packages -l -p#{$fai_config_dir} > #{nfsroot}/var/tmp/packages.nfsroot}, log: false
    puts "Adding additional packages to #{nfsroot}:"
    puts execvalue "cat #{nfsroot}/var/tmp/packages.nfsroot"
    execute %{env LC_ALL="C" install_packages -v -p#{$fai_config_dir}}

FileUtils.rm "#{nfsroot}/var/lib/dpkg/available"
  end

def add_etc_hosts_entries

# some default entries
    execute %{ainsl -as #{nfsroot}/etc/hosts "127.0.0.1 localhost"}
    execute %{ainsl     #{nfsroot}/etc/hosts "# Following entries are specific to your environment"}
    execute %{ainsl #{$verbose}  #{nfsroot}/etc/hosts "#{$NFSROOT_ETC_HOSTS}"}

# add entries for all network devices
    ips = execvalue("ip addr show up| grep -P -o '(?<=inet )[^/]+' | grep -v 127.0.0.1").split("\n")

ips.each do |eth|
      line = execvalue "getent hosts #{eth}", fail: false
      unless line.empty?
        execute %{ainsl #{$verbose} #{nfsroot}/etc/hosts "#{$line}"}
      else
        puts "Warning: no hostname for #{$eth} found, not adding to /etc/hosts."
      end
    end

if File.exist? "/etc/resolv.conf"
      execute "cp -Lp #{$verbose} /etc/resolv.conf #{nfsroot}/etc/resolv.conf-installserver"
      execute "cp -Lp #{$verbose} /etc/resolv.conf #{nfsroot}/etc/resolv.conf # this is needed during fai-make-nfsroot"
    end
  end

# Calls dpkg-divert to mark a file not to be overridden when a package updates.
  def fdivert *args
    for item in args
      execute "#{rootcmd} dpkg-divert --quiet --add --rename #{item}"
    end
  end

def upgrade_nfsroot

FileUtils.mkdir_p "#{nfsroot}/dev/pts"
    execute "mount -t proc   /proc  #{nfsroot}/proc"
    execute "mount -t sysfs  /sys   #{nfsroot}/sys"
    execute "mount -t devpts devpts #{nfsroot}/dev/pts"
    execute "/usr/lib/fai/mkramdisk #{nfsroot}/var/lib/dpkg #{nfsroot}/var/cache"

# stop mdadm from calling mkconf
    FileUtils.mkdir "#{nfsroot}/etc/mdadm"
    FileUtils.touch "#{nfsroot}/etc/mdadm/mdadm.conf"

# add any 3rd party repository keys that you may need installed into the NFSROOT
    debug "Copying apt keys"
    if Dir.exist? "#{$fai_config_dir}/apt/keys"
      for f in `find #{$fai_config_dir}/apt/keys -type f -name '*.asc' -o -name '*.gpg'`.split("\n")
        debug "Copying #{f} to #{nfsroot}/etc/apt/trusted.gpg.d"
        FileUtils.cp f, "#{nfsroot}/etc/apt/trusted.gpg.d"
      end
    end

# if we have the keyring on the host, install the key of the fai-project.org package repository
    execute <<MULTILINE
if [ ! -f #{nfsroot}/etc/apt/trusted.gpg.d/fai-project.gpg -a -n "$(apt-key list 2BF8D9FE074BCDE4 2>/dev/null)" ]; then
    HOME=/root apt-key export 2BF8D9FE074BCDE4 > #{nfsroot}/etc/apt/trusted.gpg.d/fai-project.asc
fi
MULTILINE

pkgs = $all_packages ? "nfs-common" : ""

execute "#{rootcmd} apt-get update"

# check if we need more dracut packages (dracut >=049)
    execute "#{rootcmd} apt show dracut-live dracut-squash >/dev/null 2>/dev/null"
    pkgs += " dracut-live dracut-squash" if $?.success?

# disable generating the initrd
    fdivert "/etc/kernel/postinst.d/dracut"
    execute "#{rootcmd} ln -s /bin/true /etc/kernel/postinst.d/dracut"

# fai-nfsroot must be sucessfully installed before anything else
    execute "#{rootcmd} apt-get -yf --no-install-recommends install fai-nfsroot dracut-network dracut-config-generic #{$pkgs}"
    execute "#{rootcmd} apt-get -y dist-upgrade"
    # patch dracut. Do not fail if network is not available
    fn = execvalue "find #{nfsroot}/usr/lib/dracut/modules.d/ -name net-genrules.sh"
    execute "sed -i -e 's/exit 1/exit 0/' #{fn}"
    fdivert "/usr/sbin/update-grub"
  end

def set_root_pw
    # NOTE: $generic is not implemented for now.
    return if $generic # do nothing here if we generate a generic nfsroot

# set root pw inside the nfsroot
    execute %{#{rootcmd} usermod -p "#{$FAI_ROOTPW}" root}
  end

def create_nfsroot

#local fslist

execute "mkdir -p #{nfsroot}/#{$FAI}"
    Dir.chdir nfsroot

if opts[:base_archive]
      extract_base
    else
      create_base
    end

FileUtils.touch ".THIS_IS_THE_FAI_NFSROOT"
    # save the list of all packages in the base.tar.gz
    execute "#{rootcmd} dpkg --get-selections | awk '/install$/ {print $1}' > var/tmp/base-pkgs.lis"

if !$FAI_DEBMIRROR.nil? and !$FAI_DEBMIRROR.empty?
      verb "Mounting #{$FAI_DEBMIRROR} to #{nfsroot}/#{$MNTPOINT}."
      FileUtils.mkdir_p "#{nfsroot}/#{$MNTPOINT}"
      execute "mount -o ro,noatime $FAI_DEBMIRROR nfsroot/$MNTPOINT"
      raise "Can't mount #{$FAI_DEBMIRROR} to #{nfsroot}/#{$MNTPOINT}." unless $?.success?
    end

# liloconfig, dump needs this file
    FileUtils.touch "etc/fstab"

# we need these option before installing the first package. So we
    # can't put this into fai-nfsroot /etc/apt/apt.conf.d/90fai
    execute <<MULTILINE
cat >#{nfsroot}/etc/apt/apt.conf.d/10fai <<EOF
APT::Install-Recommends no;
APT::Get::AllowUnauthenticated true;
Acquire::AllowInsecureRepositories "true";
Aptitude::CmdLine::Ignore-Trust-Violations yes;
Acquire::Languages none;
Acquire::Check-Valid-Until false;
EOF
MULTILINE

# currently we do not want to use systemd, but CD booting, also we
    # do not need all those filesystems inside the initrd
    FileUtils.mkdir "#{nfsroot}/etc/dracut.conf.d"
    execute %{echo 'omit_dracutmodules+=" dracut-systemd systemd systemd-initrd systemd-networkd systemd-initrd "' > #{nfsroot}/etc/dracut.conf.d/12-no-systemd.conf}
    if $all_packages
      execute %{echo 'add_dracutmodules+=" livenet "' > #{nfsroot}/etc/dracut.conf.d/13-add-cd-support.conf}
      fslist = "nfs lockd ext4 xfs"
    else
      execute %{echo 'add_dracutmodules+=" dmsquash-live "' > #{nfsroot}/etc/dracut.conf.d/13-add-cd-support.conf}
      fslist = "ext4 xfs"
    end

execute "echo 'hostonly=\"no\"' > #{nfsroot}/etc/dracut.conf.d/20-generic-image.conf"
    execute <<MULTILINE
cat <<EOF > #{nfsroot}/etc/dracut.conf.d/01-omit.conf
omit_dracutmodules+=" btrfs crypt dash lvm resume usrmount modsign mdraid shutdown dracut-systemd systemd systemd-initrd systemd-networkd systemd-initrd "
filesystems="#{fslist}"
EOF
MULTILINE

if Dir.exist? "#{$fai_config_dir}/apt/sources.list"
      execute "fcopy -s #{$fai_config_dir}/apt -t #{nfsroot}/etc/apt sources.list"
    else
      # this copies sources.list into the nfsroot
      execute "cp -aL #{$fai_config_dir}/apt #{nfsroot}/etc"
    end

add_etc_hosts_entries

=begin
    # use a proxy if variable is defined
    if [ -n "$APTPROXY" ]; then
        echo "Acquire::http::Proxy \"$APTPROXY\";" > nfsroot/etc/apt/apt.conf.d/02proxy
    fi
=end

verb "Upgrading #{nfsroot}"
    upgrade_nfsroot
    add_packages_nfsroot

# remove file diversion, then create initrd
    FileUtils.rm "#{nfsroot}/etc/kernel/postinst.d/dracut"
    execute "#{rootcmd} dpkg-divert --rename --remove /etc/kernel/postinst.d/dracut"
    execute "#{rootcmd} dpkg-reconfigure dracut"

set_root_pw

# set timezone in nfsroot
    timezonefile = "/etc/timezone"
    if File.exist? timezonefile
      execute "cp -fH #{timezonefile}  etc/timezone"
    else
      File.open timezonefile, "w" do |file|
        timezone = execvalue %{timedatectl status | grep "Time zone" | awk '{ print $3 }'}
        file.write timezone
      end
    end
    execute "cp -a /etc/localtime etc/localtime"
    FileUtils.ln_sf "/proc/mounts", "etc/mtab"

FileUtils.ln_s "/usr/sbin/fai", "etc/init.d/rcS"

execute %{echo "AUTO -all" > etc/mdadm/mdadm.conf} # do not activate RAID arrays
    # this second rm of the same file is needed!
    FileUtils.rm_f "etc/udev/rules.d/70-persistent-net.rules"
    # remove Ubuntu-specific rules to auto-start volume groups, which confuses
    # setup-storage
    FileUtils.rm_f "lib/udev/rules.d/85-lvm2.rules"
    # definition for loopback device
    execute %{echo "iface lo inet loopback" > etc/network/interfaces}

execute <<MULTILINE
    cat >> root/.profile <<-EOF
        PATH=/usr/local/sbin:/usr/local/bin:/usr/lib/fai:/bin:/sbin:/usr/bin:/usr/sbin:
        export PATH
        . /usr/lib/fai/subroutines
        set -a
        . /tmp/fai/variables.log 2>/dev/null
EOF
MULTILINE

end

def setup_ssh

# nothing to do if no ssh is available in nfsroot
    return unless File.exist? "#{nfsroot}/usr/bin/ssh"

# enable root login
    execute "sed -i -e '/PermitRootLogin/d' #{nfsroot}/etc/ssh/sshd_config"
    execute "ainsl #{nfsroot}/etc/ssh/sshd_config 'PermitRootLogin yes'"

=begin
    if [ $sshpreserve -ge 1 ]; then
        tar -C $NFSROOT -xf $tmptar
        rm $tmptar
        return 0
    fi
=end

FileUtils.mkdir_p "#{nfsroot}/root/.ssh", mode: 700
=begin
    if [ -n "$LOGUSER" ] ; then
        loguserhome=$(eval "cd ~$LOGUSER 2>/dev/null && pwd;true")
        [ -f $loguserhome/.ssh/known_hosts ] && cp $loguserhome/.ssh/known_hosts $NFSROOT/root/.ssh/known_hosts
        [ -d $loguserhome/.ssh ] && {
            [ -f $loguserhome/.ssh/id_ed25519 ] &&
               cp -p $loguserhome/.ssh/id_ed25519* $NFSROOT/root/.ssh/
            [ -f $loguserhome/.ssh/id_rsa ] &&
               cp -p $loguserhome/.ssh/id_rsa* $NFSROOT/root/.ssh/
        }
    fi
=end

=begin
    if [ -f "$SSH_IDENTITY" ]; then
        install -m0644 $SSH_IDENTITY $NFSROOT/root/.ssh/authorized_keys
        echo You can log into install clients without password using $SSH_IDENTITY
    fi
=end

# setting StrictHostKeyChecking to no inside the nfsroot would enable
    # easy login to the install server even if the host key is not known

end

def check_nfsroot

# simple test, to see if important thing are available inside the nfsroot
    unless File.executable? "#{nfsroot}/usr/bin/dracut"
        raise "dracut was not installed into the nfsroot."
    end

unless File.exist? "#{nfsroot}/sbin/init"
        raise "sysvinit-core was not installed into the nfsroot."
    end

execute "readlink #{nfsroot}/sbin/init | grep -q systemd", fail: false
    if $?.success?
        raise "Do not use systemd inside the nfsroot."
    end

puts "FAI packages and related packages inside the nfsroot:"
    execute %{#{rootcmd} dpkg-query -W -f='${Package;-18} ${Version}\n' fai-client fai-nfsroot fai-setup-storage dracut-network dracut dracut-live dracut-squash 2>/dev/null}

# check if all important packages are installed
    n = execvalue("#{rootcmd} dpkg-query -l fai-client fai-nfsroot fai-setup-storage dracut-network dracut 2>/dev/null|grep -E ^ii |wc -l").to_i
    if n != 5
        puts "ERROR: Some essential packages are missing."
        bad_exit
    end

files = execvalue "ls #{nfsroot}/boot/initrd* 2>/dev/null"
    raise "No initrd installed." if files.empty?
    execute %{grep -E -q "^ERROR: |^E: Sub-process |^dpkg: error processing |^dpkg: dependency problems" /var/log/fai/fai-make-nfsroot.log}, fail: false
    bad_exit if $?.success?

nil
  end

def write_variables
    varfile = "/var/log/fai/variables"
    fai_version = execvalue "#{rootcmd} dpkg-query -W -f='${Version}' fai-client"

# remove old values from file
    execute "sed -i -e '/FAI_VERSION/d' -e '/NFSROOT/d' #{varfile}" if File.exist? varfile and !File.zero? varfile
    execute %{echo "FAI_VERSION=#{fai_version}" >> #{varfile}}
    execute %{echo "NFSROOT=#{nfsroot}" >> #{varfile}}
  end

private

def extract_base base_archive
    unless File.exist? base_archive
      raise "Base archive #{base_archive} could not be used."
    end

Dir.chdir nfsroot do
      verb "Extracting base archive #{base_archive}: "

case File.basename(base_archive).split(".", 2)[1]
      when "tar.gz", ".tgz"
        execute %{tar --xattrs --acls --xattrs-include=* -C "#{nfsroot}" -zxpf "#{base_archive}"}
        FileUtils.cp base_archive, "#{nfsroot}/var/tmp/base.tar.gz", preserve: true
      when "tar.xz", "txz"
        execute %{tar --xattrs --acls --xattrs-include=* -C "#{nfsroot}" -Jxpf "#{base_archive}"}
        FileUtils.cp base_archive, "#{nfsroot}/var/tmp/base.tar.xz", preserve: true
      end
    end
  end

def bad_exit
    puts "An error occured during fai-make-nfsroot."
    puts "Please fix the error or try fai-make-nfsroot -v"
    exit 1
  end
end0707010000001F000081A4000001C3000001CA000000015ED77A8D00000014000000000000000000000000000000000000003D00000000fai-toolkit-1591179917.2894a88/lib/create-nfsroot/nfsroot.rb
module Nfsroot
end
07070100000020000041ED000001C3000001CA000000025ED77A8D00000000000000000000000000000000000000000000003400000000fai-toolkit-1591179917.2894a88/lib/create-update-cd07070100000021000081A4000001C3000001CA000000015ED77A8D00000C60000000000000000000000000000000000000004800000000fai-toolkit-1591179917.2894a88/lib/create-update-cd/create-update-cd.rb
require "fileutils"
require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "methods"
require_relative "updatecd"

# Main class.
class Updatecd::CreateUpdateCd
  include Common
  include Output
  include Updatecd::Methods

def perform
    startup_checks
    real_perform
  end

def options
    options = Optimist::options do
      opt :mirror_dir,        "Location of fai-mirror",                                   type: :string
      opt :grub_config,       "Path to grub.cfg for this image",                          type: :string
      opt :filename,          "Filename to generate image to. For example image.iso",     type: :string
      opt :initrd,            "Use a custom initrd instead of one from NFSROOT",          type: :string
      opt :debug,             "Show debug output"
      opt :verbose,           "Show verbose output"
    end

raise "--mirror-dir is required." if options[:mirror_dir].nil?
    raise "--mirror-dir needs to be a valid directory." unless Dir.exist? options[:mirror_dir]
    $mirrordir = options[:mirror_dir]

raise "--grub-config is required." if options[:grub_config].nil?
    raise "--grub-config needs to be a valid file." unless File.exist? options[:grub_config]
    $grub_config = options[:grub_config]

raise "--filename is required." if options[:filename].nil?
    raise "--filename needs not to exist before. Remove the file if you wish to overwrite current image." if File.exist? options[:filename]
    $image_name = options[:filename]

raise "Given --initrd file is not valid." if not options[:initrd].nil? and not File.exist? options[:initrd]
    $initrd = options[:initrd]

set_opts options
  end

private

def real_perform
    create_ext3fs_image
    create_squashfs_image

$kernelversion = execvalue("ls -tr #{nfsroot}/boot/vmlinu?-* | tail -1 | sed -e 's#.*/vmlinuz-##'")
    faiversion = execvalue("dpkg --root=#{nfsroot} -l fai-client|awk '/fai-client/ {print $3}'")
    date_str = execvalue("date '+%Y %b %d - %R'")
    $isoversion = sprintf "FAI %-6s build #{date_str}", "#{faiversion}"

$aname = "Fully Automatic Installation, #{$isoversion}"

create_grub2_image

#provide_memtest_boot_option

mkiso

end

def startup_checks
    $logfile = File.open("/var/log/fai/create-update-cd.log", "w")
    at_exit do
      $logfile.close
    end

check_programs

# TODO: something better for this.
    $vname = "FAI_CD"

#: ${FAI:=/var/lib/fai/config} # default value

#if [ -n "$csdir" ]; then
    #    FAI_CONFIGDIR=$csdir # override by -c
    #fi

$ONFSROOT = nfsroot # save original nfsroot

#[ -d "$NFSROOT/etc/fai" ] || die 10 "Please create NFSROOT by calling fai-make-nfsroot or fai-setup."

#[ $hidevartmp -eq 1 ] && hidedirs+="/var/tmp"

$tmp = execvalue("mktemp -t -d fai-cd.XXXXXX")
    debug "Got tmp: #{$tmp}"
    raise "Temp creation failed." unless $?.success?
    $scratch = execvalue("mktemp -t -d scratch.XXXXXX")
    raise "Scratch creation failed." unless $?.success?

#trap "cleanup_liveos_mounts" EXIT ERR
    at_exit do
      cleanup_liveos_mounts
    end
  end
end
07070100000022000081A4000001C3000001CA000000015ED77A8D00002017000000000000000000000000000000000000003F00000000fai-toolkit-1591179917.2894a88/lib/create-update-cd/methods.rb
require_relative "updatecd"
require_relative "../common"

module Updatecd::Methods
  include Common

def check_programs
    msg = ""
    msg+="xorriso not found. Please install package." unless command_available? "xorriso"
    msg+="\nmksquashfs not found. Please install the package squashfs-tools." unless command_available? "mksquashfs"
    msg+="\nmkfs.vfat not found. Please install the package dosfstools." unless command_available? "mkfs.vfat"
    msg+="\nmcopy not found. Please install the package mtools." unless command_available? "mcopy"

unless msg.empty?
      puts msg
      exit 1
    end
  end

def error_space status, message
    STDERR.puts "Not enough space left on image for the #{message}"
    STDERR.puts execvalue("df -h #{$tmp}/#{$liveos}/mounted-ext3fs")
    exit status.exitstatus
  end

def customize_nfsroot
    FileUtils.mkdir_p "#{$tmp}/empty"

$hidedirs.each do |dir|
      if Dir.exist? "#{$ONFSROOT}/#{dir}"
        debug "hiding #{dir}"
        execute("mount --bind #{$tmp}/empty #{$ONFSROOT}/#{dir}")
      end
    end
  end

def unhide_dirs
    $hidedirs.each do |dir|
      if Dir.exist? "#{$ONFSROOT}/#{dir}"
        debug "disclosing #{dir}"
        execute("umount #{$ONFSROOT}/#{dir} 2>/dev/null", fail: false, log: false)
      end
    end
  end

# calculates the amount of space needed for the ext3fsimage
  def calculate_required_size
    size = execvalue("du -sc #{$ONFSROOT} #{$FAI_CONFIGDIR} #{$mirrordir}/{dists,pool} | tail -n1 | awk '{print $1}'").to_i

# not dividing by 1024 to get the exact, this allows us to get some additional space
    size / 850
  end

def addmirror
    Dir.mkdir "#{$tmp}/#{$liveos}/mounted-ext3fs/media/mirror"

verb "Copying the mirror to CD image"
    execute("cp -Pr #{$mirrordir}/{dists,pool} #{$tmp}/#{$liveos}/mounted-ext3fs/media/mirror 2>/dev/null")
    error_space $?, "mirror" unless $?.success?

# create the sources.list for the CD
    execute <<MULTILINE
cat > #{$tmp}/#{$liveos}/mounted-ext3fs/etc/apt/sources.list <<EOF
# mirror location for fai CD, file generated by fai-cd
EOF
MULTILINE

dists = execvalue("find #{$mirrordir} -name \"Packages*\" | grep binary | sed 's/binary-.*//' | sed \"s##{$mirrordir}/*dists/##\" | xargs -r -n 1 dirname | sort | uniq")
    raise "No suitable Packages file found in mirror." if dists.empty?

dists = dists.split " "
    dists.each do |dist|
      comp = execvalue("find #{$mirrordir}/dists/#{dist} -maxdepth 2 -type d -name \"binary-*\" | sed -e \"s##{$mirrordir}/*dists/#{dist}/##\" -e 's#/binary-.*##' | uniq | tr '\n' \" \"")
      execute("echo \"deb [trusted=yes] file:/media/mirror #{dist} #{comp}\" >> #{$tmp}/squashfs-root/LiveOS/mounted-ext3fs/etc/apt/sources.list")
    end
  end

def create_ext3fs_image

# create the folder that will be used for the squashfs.img later
    # along with it, a temporary folder is created to mount the ext3fs.img
    FileUtils.mkdir_p "#{$tmp}/#{$liveos}/mounted-ext3fs"

# hide some directories before copying the nfsroot
    customize_nfsroot
    count = calculate_required_size

# creates an empty file
    verb "Creating empty ext3fs.img"
    execute("dd if=/dev/zero of=#{$tmp}/#{$liveos}/ext3fs.img bs=1MB count=#{count}")
    raise "dd failed. Maybe no enough free space in #{$tmp}" unless $?.success?

execute("mkfs.ext4 -m1 -q -O^has_journal -F -L FAI-NFSROOT #{$tmp}/#{$liveos}/ext3fs.img")

# the resulting ext3fs.img is mounted
    execute("mount -o loop,noatime #{$tmp}/#{$liveos}/ext3fs.img #{$tmp}/#{$liveos}/mounted-ext3fs")

verb "Copying the nfsroot to CD image"
    # copy the contents of the nfsroot into it
    execute("cp -Ppr #{$ONFSROOT}/. #{$tmp}/#{$liveos}/mounted-ext3fs/")
    error_space $?, "nfsroot" unless $?.success?
    FileUtils.rm_f "#{$tmp}/#{$liveos}/mounted-ext3fs/etc/resolv.conf"

# execute("find $tmp/$liveos/mounted-ext3fs") # for debugging
    # copy config space into nfsroot-copy unless -d is given
    verb "Copying the config space to CD image"
    execute("cp -Ppr #{$FAI_CONFIGDIR}/* #{$tmp}/#{$liveos}/mounted-ext3fs/var/lib/fai/config/")
    error_space $?, "config directory" unless $?.success?
    # add the partial mirror to the CD
    addmirror

unhide_dirs
    execute("umount #{$tmp}/#{$liveos}/mounted-ext3fs")
    Dir.rmdir "#{$tmp}/#{$liveos}/mounted-ext3fs"
  end

def create_squashfs_image

# this is where the squashfs.img needs to be placed
    FileUtils.mkdir_p "#{$tmp}/LiveOS"

# create the squashfs.img
    output_path = "#{$tmp}/LiveOS/squashfs.img"

sqopt = "-comp xz"

execute("mksquashfs #{$tmp}/squashfs-root #{output_path} #{sqopt} -progress", direct_output: true)

File.delete "#{$tmp}/#{$liveos}/ext3fs.img"
    Dir.rmdir "#{$tmp}/#{$liveos}"
    Dir.rmdir "#{$tmp}/squashfs-root"
  end

def create_grub2_image

FileUtils.mkdir_p "#{$tmp}/boot"

unless File.exist? "#{nfsroot}/usr/lib/grub/x86_64-efi"
      raise "No grub-efi-amd64-bin installation found in NFSROOT. Aborting."
    end

FileUtils.cp $grub_config, "#{nfsroot}/tmp/grub.cfg"
    # insert date into grub menu
    execute("perl -pi -e \"s/_VERSIONSTRING_/   #{$isoversion}     /\" #{nfsroot}/tmp/grub.cfg")

execute(%{chroot #{nfsroot} grub-mkstandalone \
        --format=x86_64-efi \
        --output=/tmp/bootx64.efi \
        --locales="" --fonts="" \
        "boot/grub/grub.cfg=/tmp/grub.cfg"})
    FileUtils.move "#{nfsroot}/tmp/bootx64.efi", $scratch

verb "Creating efiboot.img"
    execute("dd if=/dev/zero of=#{$scratch}/efiboot.img bs=1M count=3")
    execute("mkfs.vfat #{$scratch}/efiboot.img", log: false)
    execute("mmd -i #{$scratch}/efiboot.img efi efi/boot")
    execute("mcopy -i #{$scratch}/efiboot.img #{$scratch}/bootx64.efi ::efi/boot/")

unless File.exist? "#{nfsroot}/usr/lib/grub/i386-pc"
      raise "No grub-pc installation found in NFSROOT. Aborting."
    end

execute(%{chroot #{nfsroot} grub-mkstandalone \
        --format=i386-pc \
        --output=/tmp/core.img \
        --locales="" --fonts="" \
        --install-modules="linux normal iso9660 biosdisk memdisk search ls echo test chain msdospart part_msdos part_gpt minicmd ext2 keystatus all_video font sleep gfxterm" \
        --modules="linux normal iso9660 biosdisk search" \
        "boot/grub/grub.cfg=/tmp/grub.cfg"})

execute("cat #{nfsroot}/usr/lib/grub/i386-pc/cdboot.img #{nfsroot}/tmp/core.img > #{$scratch}/bios.img")
    FileUtils.rm "#{nfsroot}/tmp/core.img"

execute("cp -p #{nfsroot}/boot/vmlinuz-#{$kernelversion} #{$tmp}/boot/vmlinuz")

$initrd ||= "#{nfsroot}/boot/initrd.img-#{$kernelversion}"
    execute("cp -p #{$initrd} #{$tmp}/boot/initrd.img")

execute("cp -p #{nfsroot}/boot/config-#{$kernelversion} #{$tmp}/boot/")
    execute("touch #{$tmp}/FAI-CD")
  end

# - - - - - - - - - - - - - - - - - - - - - - - - - -
  def mkiso
    # NOTE: I currently have too old xorriso to have this argument.
    # -iso_mbr_part_type 00 \
    verb "Writing FAI CD-ROM image to #{$image_name}. This may need some time."
    execute(%{xorriso -report_about HINT -as mkisofs -iso-level 3 \
        -full-iso9660-filenames \
        -volid "#{$vname}" -appid "#{$aname}" \
        -eltorito-boot boot/grub/bios.img  \
        -no-emul-boot -boot-load-size 4 -boot-info-table \
        --eltorito-catalog boot/grub/boot.cat \
        --grub2-boot-info \
        --grub2-mbr #{nfsroot}/usr/lib/grub/i386-pc/boot_hybrid.img \
        -eltorito-alt-boot -e EFI/efiboot.img -no-emul-boot \
        -append_partition 2 0xef #{$scratch}/efiboot.img \
        -o #{$image_name} -graft-points \
        --sort-weight 0 / --sort-weight 1 /boot \
        "#{$tmp}" \
        /boot/grub/bios.img=#{$scratch}/bios.img \
        /EFI/efiboot.img=#{$scratch}/efiboot.img}, log: false)
    raise "xorriso failed." unless $?.success?

puts "ISO image size and filename: #{execute "du -h #{$image_name}"}"

unhide_dirs
  end

def cleanup_liveos_mounts
    execute("mountpoint -q #{$tmp}/#{$liveos}/mounted-ext3fs && umount #{$tmp}/#{$liveos}/mounted-ext3fs", fail: false, log: false)
    unhide_dirs
    FileUtils.rm_rf [ $tmp, $scratch ], secure: true
  end

end
07070100000023000081A4000001C3000001CA000000015ED77A8D00000015000000000000000000000000000000000000004000000000fai-toolkit-1591179917.2894a88/lib/create-update-cd/updatecd.rb
module Updatecd
end
07070100000024000041ED000001C3000001CA000000025ED77A8D00000000000000000000000000000000000000000000002900000000fai-toolkit-1591179917.2894a88/lib/media07070100000025000081A4000001C3000001CA000000015ED77A8D00000012000000000000000000000000000000000000003200000000fai-toolkit-1591179917.2894a88/lib/media/media.rb
module Media
end
07070100000026000081A4000001C3000001CA000000015ED77A8D000005D4000000000000000000000000000000000000003200000000fai-toolkit-1591179917.2894a88/lib/media/mount.rb
require "json"

require_relative "media"
require_relative "../common"

class Media::Mount
  include Common

MOUNT_POINT = "/mnt/cdrom"
  SQUASH_MOUNT_POINT = "/mnt/squash"
  FS_MOUNT_POINT = "/mnt/filesystem"

# Mounts a FAI-style CD so that it can be used for updating.
  def mount_cd
    cd_devices = find_fai_cd

Dir.mkdir MOUNT_POINT unless Dir.exist? MOUNT_POINT
    Dir.mkdir SQUASH_MOUNT_POINT unless Dir.exist? SQUASH_MOUNT_POINT
    Dir.mkdir FS_MOUNT_POINT unless Dir.exist? FS_MOUNT_POINT

update_cd_found = false
    cd_devices.each do |cd_device|
      dev_path = "/dev/#{cd_device["name"]}"
      execute "mount #{dev_path} #{MOUNT_POINT}"
      if File.exist? "#{MOUNT_POINT}/LiveOS/squashfs.img"
        update_cd_found = true
        break
      end
    end

raise "FATAL: None of attached rom devices were an update cd. Please check the media you have inserted." unless update_cd_found

execute %{mount "#{MOUNT_POINT}/LiveOS/squashfs.img" "#{SQUASH_MOUNT_POINT}"}
    execute %{mount "#{SQUASH_MOUNT_POINT}/LiveOS/ext3fs.img" "#{FS_MOUNT_POINT}"}
    execute %{mount --bind "#{FS_MOUNT_POINT}/media/mirror" /media/mirror}
  end

private

def find_fai_cd
    json = `lsblk -a -J`
    data = JSON.parse json

cd_devices = []

data["blockdevices"].each do |device|
      if device["type"] == "rom"
        cd_devices << device
      end
    end

raise "FATAL: No suitable ROM devices found." if cd_devices.empty?

cd_devices
  end
end
07070100000027000081A4000001C3000001CA000000015ED77A8D00000137000000000000000000000000000000000000002D00000000fai-toolkit-1591179917.2894a88/lib/output.rb
# Outputting utilities
module Output
  def debug str
    return unless $_debug
    puts "[DEBUG] #{str}"
    $logfile.write "[DEBUG] #{str}" if $do_log
  end

def verb str
    return unless $_verbose
    puts "[INFO] #{str}"
    $logfile.write str if $do_log
  end

def self.logfile
    $logfile
  end
end
07070100000028000041ED000001C3000001CA000000025ED77A8D00000000000000000000000000000000000000000000003100000000fai-toolkit-1591179917.2894a88/lib/system-update07070100000029000081A4000001C3000001CA000000015ED77A8D000003F8000000000000000000000000000000000000003C00000000fai-toolkit-1591179917.2894a88/lib/system-update/perform.rb
require "fileutils"
require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "../media/mount"
require_relative "system-update"

# Main class.
class SystemUpdate::Perform
  include Common

def perform
    mount = Media::Mount.new

# Only mount once; this script can be run multiple times to help with debugging.
    unless Dir.exist? "/mnt/filesystem/var/lib/fai/config"
      mount.mount_cd
    end

sourceslist = "/etc/apt/sources.list"
    FileUtils.mv sourceslist, "#{sourceslist}.BAK"
    FileUtils.cp "/mnt/filesystem/#{sourceslist}", sourceslist
    at_exit do
      FileUtils.rm sourceslist
      FileUtils.mv "#{sourceslist}.BAK", sourceslist
    end

execute %{fai -v -N -C /mnt/filesystem/etc/fai -s file:///mnt/filesystem/var/lib/fai/config softupdate}
  end

def options
    options = Optimist::options do
      opt :debug,             "Show debug output"
      opt :verbose,           "Show verbose output"
    end

set_opts options
  end

end
0707010000002A000081A4000001C3000001CA000000015ED77A8D00000019000000000000000000000000000000000000004200000000fai-toolkit-1591179917.2894a88/lib/system-update/system-update.rb
module SystemUpdate
end
07070100000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000B00000000TRAILER!!!167 blocks

Places

File _service:obs_scm:fai-toolkit-1591179917.2894a88.obscpio of Package fai-toolkit (Revision 36077ce53d3a609566d5bb9cb3bfcd8a)

Places