File _service:obs_scm:fai-toolkit-1600669692.02e37ec... of Package fai-toolkit

Overview Repositories Revisions Requests Users Attributes Meta

File _service:obs_scm:fai-toolkit-1600669692.02e37ec.obscpio of Package fai-toolkit (Revision ff38dfb86bdda42d85c6b4ee77674711)

Currently displaying revision ff38dfb86bdda42d85c6b4ee77674711 , Show latest

07070100000000000081A4000001C3000001CA000000015F6847FC00000016000000000000000000000000000000000000002B00000000fai-toolkit-1600669692.02e37ec/.kateconfigkate: indent-width 2;
07070100000001000081A4000001C3000001CA000000015F6847FC000004C3000000000000000000000000000000000000002700000000fai-toolkit-1600669692.02e37ec/LICENSEMost of scripts here are derivative of original scripts from FAI:

* create-nfsroot.rb is derivate of fai-make-nfsroot
* create-base.rb is derivate of fai-make-nfsroot
* create-update-cd.rb is derivate of fai-cd.

The original scripts are licensed under GPL-2.0-or-later and hence these
scripts are also available under GPL-2.0-or-later. See more at
https://spdx.org/licenses/GPL-2.0-or-later.html.

Additionally, for all parts that legally can be licensed under ISC,
are licensed under ISC license:

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH REGARD
TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT,
OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE,
DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE
OF THIS SOFTWARE.07070100000002000081A4000001C3000001CA000000015F6847FC000001F4000000000000000000000000000000000000002900000000fai-toolkit-1600669692.02e37ec/README.md# FAI update toolkit

FAI’s default utilities were not useful enough for me, so I went and converted them to Ruby in order to modify them as needed.

For many parts, feature sets of the scripts are smaller than original scripts’, though there is some subtle changes and one feature addition: ability to use own custom initrd in a FAI CD. There is also separate base archive creation script.

# License

See LICENSE. In short, GPL-2.0-or-later and ISC.

# Contributions

Contributions are welcome.07070100000003000041ED000001C3000001CA000000035F6847FC00000000000000000000000000000000000000000000002300000000fai-toolkit-1600669692.02e37ec/bin07070100000004000081A4000001C3000001CA000000015F6847FC0000010C000000000000000000000000000000000000003800000000fai-toolkit-1600669692.02e37ec/bin/create-base-image.rb#!/usr/bin/env ruby

# Creates a base image for FAI to use.

raise "create-base-image must be run as root." unless Process.uid == 0

require "fai-toolkit/create-base-image/base-image"

base_image = CreateBaseImage::BaseImage.new
base_image.options

base_image.perform
07070100000005000081A4000001C3000001CA000000015F6847FC000000E0000000000000000000000000000000000000003200000000fai-toolkit-1600669692.02e37ec/bin/create-base.rb#!/usr/bin/env ruby

# Creates FAI’s nfsroot.

raise "create-nfsroot must be run as root." unless Process.uid == 0

require "fai-toolkit/base-image/create-base"

base = BaseImage::CreateBase.new
base.options

base.perform
07070100000006000081A4000001C3000001CA000000015F6847FC00000119000000000000000000000000000000000000003800000000fai-toolkit-1600669692.02e37ec/bin/create-fai-mirror.rb#!/usr/bin/env ruby

# A script to create a mirror FAI uses for installation purposes.

#raise "create-update must be run as root." unless Process.uid == 0

require "fai-toolkit/create-fai-mirror/fai-mirror"

mirror = CreateFaiMirror::FaiMirror.new

mirror.options
mirror.perform

07070100000007000081A4000001C3000001CA000000015F6847FC00000B96000000000000000000000000000000000000003400000000fai-toolkit-1600669692.02e37ec/bin/create-fai-vm.rb#!/usr/bin/env ruby

raise "create-fai-vm must be run as root." unless Process.uid == 0

# Quick wrapper around fai-diskimage to have some sane defaults and logic.

require "optimist"

require "fileutils"

require "fai-toolkit/common"

class CreateFaiVm
  include Common

def initialize
    # TODO: proper FAI server needs to be set even for testing, as that’s how systems would communicate in real life.
    # Actually, maybe above doesn’t matter much. Maybe? Maybe it’s just for fai-monitor.
    @server = "localhost"

# TODO: for some reason this isn’t read from /etc/fai/nfsroot.conf, and it’s not present anywhere else.
    # I think this warrants more investigation if time allows.
    #set FAI_DEBOOTSTRAP "cskoeln /srv/fai/mirrorroot/srv/fai/mirror/debian_buster"
    #set FAI_DEBOOTSTRAP "buster http://deb.debian.org/debian"
    @fai_debootstrap = "buster http://ryza.smar.fi:3142/debian/debian"

# Instead of this, there could be a manipulated DNS server, which returns correct IP, but since this is my local setup,
    # it doesn’t make much sense.
    @nfsroot_etc_hosts = "192.168.1.230   ryza.smar.fi"
  end

def options
    options = Optimist::options do
      opt :image_name,        "Path to resulting image. For example, image.qcow2",
          type: :string,      short: "n"
      opt :class,             "Includes given class in image creation",
          type: :string,      short: "c",                   multi: true
      opt :size,              "Size of resulting image. For example, 20G.",
          type: :string,      short: "s", default: "10G"
      opt :hostname,          "Hostname that is used to generate the image.",
          type: :string,      short: "h"
      opt :force,             "Overwrite existing image.",
                              short: "f"
      opt :verbose,           "Show verbose output",
                              short: "v"
      opt :debug,             "Show debug output",
                              short: "d"
    end

set_opts options

raise "--image-name must be given. " if options[:image_name].nil?
    @image_name = options[:image_name]

if File.exist? @image_name
      unless options[:force]
        raise "File exists at target #{@image_name} already"
      else
        FileUtils.rm @image_name
      end
    end

@classes = options[:class]
    raise "At least one --class must be given." if @classes.nil? or @classes.empty?

@image_size = options[:size]

raise "--hostname must be given." if options[:hostname].nil?
    @hostname = options[:hostname]
  end

def perform
    execute %{env -i PATH="/usr/bin:/usr/sbin" SERVER="#{@server}" LOGUSER= FAI_LOGPROTO=none FAI_DEBOOTSTRAP="#{@fai_debootstrap}" NFSROOT_ETC_HOSTS="#{@nfsroot_etc_hosts}" \
            fai-diskimage -v --hostname "#{@hostname}" -S"#{@image_size}" -c"#{@classes.join(",")}" "#{@image_name}"}, direct_output: true
  end

end

vm = CreateFaiVm.new
vm.options
vm.perform07070100000008000081A4000001C3000001CA000000015F6847FC000000EA000000000000000000000000000000000000003500000000fai-toolkit-1600669692.02e37ec/bin/create-nfsroot.rb#!/usr/bin/env ruby

# Creates FAI’s nfsroot.

raise "create-nfsroot must be run as root." unless Process.uid == 0

require "fai-toolkit/nfsroot/create-nfsroot"

nfsroot = Nfsroot::CreateNfsroot.new
nfsroot.options

nfsroot.perform
07070100000009000081A4000001C3000001CA000000015F6847FC000001CE000000000000000000000000000000000000003700000000fai-toolkit-1600669692.02e37ec/bin/create-update-cd.rb#!/usr/bin/env ruby

# A script to create a cd that can be used to boot to update mode.
#
# fai-make-nfsroot doesn’t put xfs to initrd, which is useful for real installations.
# Instead of hackery over a bash script, let’s just create the image by ourselves.

raise "create-update must be run as root." unless Process.uid == 0

require "fai-toolkit/create-update-cd/create-update-cd"

updatecd = Updatecd::CreateUpdateCd.new
updatecd.options
updatecd.perform0707010000000A000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000002D00000000fai-toolkit-1600669692.02e37ec/bin/packaging0707010000000B000081A4000001C3000001CA000000015F6847FC000000AE000000000000000000000000000000000000003E00000000fai-toolkit-1600669692.02e37ec/bin/packaging/build-package.rb#!/usr/bin/env ruby

# Builds a package in a special VM.

require "fai-toolkit/packaging/build-package"

builder = Packaging::BuildPackage.new
builder.options
builder.perform0707010000000C000081A4000001C3000001CA000000015F6847FC00000001000000000000000000000000000000000000004700000000fai-toolkit-1600669692.02e37ec/bin/packaging/generate-build-machine.rb
0707010000000D000081A4000001C3000001CA000000015F6847FC000000AC000000000000000000000000000000000000003C00000000fai-toolkit-1600669692.02e37ec/bin/perform-online-update.rb#!/usr/bin/env ruby

# Performs automated online update.

require "fai-toolkit/online-update/perform"

perform = OnlineUpdate::Perform.new
perform.options
perform.perform

0707010000000E000081A4000001C3000001CA000000015F6847FC000000CD000000000000000000000000000000000000003C00000000fai-toolkit-1600669692.02e37ec/bin/perform-system-update.rb#!/usr/bin/env ruby

# Performs automated system update.

require "fai-toolkit/media/mount"
require "fai-toolkit/system-update/perform"

perform = SystemUpdate::Perform.new
perform.options
perform.perform
0707010000000F000081A4000001C3000001CA000000015F6847FC000000C8000000000000000000000000000000000000003D00000000fai-toolkit-1600669692.02e37ec/bin/perform-update-on-host.rb#!/usr/bin/env ruby

# Updater side of perform-online-update script.

require "fai-toolkit/online-update/perform-on-host.rb"

perform = OnlineUpdate::PerformOnHost.new
perform.options
perform.perform
07070100000010000081A4000001C3000001CA000000015F6847FC000000C3000000000000000000000000000000000000003900000000fai-toolkit-1600669692.02e37ec/bin/sync-configuration.rb#!/usr/bin/env ruby

# Syncs updated configuration to a target host.

require "fai-toolkit/configuration/sync-configuration"

conf = Configuration::SyncConfiguration.new
conf.options
conf.perform07070100000011000081A4000001C3000001CA000000015F6847FC00000060000000000000000000000000000000000000003500000000fai-toolkit-1600669692.02e37ec/bin/update-from-cd.rb#!/usr/bin/env ruby

require "fai-toolkit/media/mount"

mount = Media::Mount.new
mount.mount_cd
07070100000012000041ED000001C3000001CA0000000C5F6847FC00000000000000000000000000000000000000000000002300000000fai-toolkit-1600669692.02e37ec/lib07070100000013000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000002E00000000fai-toolkit-1600669692.02e37ec/lib/base-image07070100000014000081A4000001C3000001CA000000015F6847FC00000016000000000000000000000000000000000000003C00000000fai-toolkit-1600669692.02e37ec/lib/base-image/base-image.rb
module BaseImage
end
07070100000015000081A4000001C3000001CA000000015F6847FC000007D2000000000000000000000000000000000000003D00000000fai-toolkit-1600669692.02e37ec/lib/base-image/create-base.rb
require "fileutils"
require "tmpdir"

require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "../fai-config"
require_relative "base-image"
require_relative "methods"

class BaseImage::CreateBase
  include Common
  include Output
  include BaseImage::Methods

def perform
    startup_checks

tmpdir = Dir.mktmpdir "create-base_"
    at_exit do
      FileUtils.remove_entry_secure tmpdir
    end
    self.nfsroot= tmpdir

# I find it strange that the cleanup is necessary, but eh... I have a mistake somewhere.
    #umount_dirs

resolve_sources
    create_base

# Wait for compression to be done.
    Process.wait

path = "#{nfsroot}/var/tmp/base.tar.xz"

# Move the file to the location one wants it to be.
    FileUtils.mv path, @base_path
  end

def options
    options = Optimist::options do
      opt :filename,          "Path to the file to be written.",                              type: :string
      opt :fai_config_dir,    "FAI’s main config files.",                                     type: :string,      default: "/etc/fai",    short: "c"
      opt :force,             "Overwrite existing base image.",                                                   default: false
      opt :debug,             "Show debug output"
      opt :verbose,           "Show verbose output"
    end

raise "Given --fai-config-dir is not valid." if not Dir.exist? options[:fai_config_dir]
    @fai_config_dir = options[:fai_config_dir]

raise "--filename is required." if options[:filename].nil?
    @base_path = options[:filename]
    raise "File #{@base_path} already exists. You can use --force to overwrite existing file." if File.exist? @base_path and options[:force] != true

$logfile = File.open("/var/log/fai/create_base.log", "w")
    at_exit do
      $logfile.close
    end

set_opts options
  end

private

def startup_checks
    @config = FaiConfig.new @fai_config_dir
    @config.load_nfsroot_conf
  end
end
07070100000016000081A4000001C3000001CA000000015F6847FC00001BF5000000000000000000000000000000000000003900000000fai-toolkit-1600669692.02e37ec/lib/base-image/methods.rb
require_relative "base-image"
require_relative "../common"

module BaseImage::Methods
  include Common

# Creates a temporary base archive that is supplied with generated nfsroot.
  def create_base
    raise "Can't find debootstrap command. Aborting." unless command_available? "debootstrap"
    call_debootstrap
    execute "#{rootcmd} apt-get clean"
    execute "rm -f #{@config.nfsroot}/etc/resolv.conf #{@config.nfsroot}/etc/hostname #{@config.nfsroot}/etc/udev/rules.d/70-persistent-net.rules"
    verb "Creating base.tar.xz"
    execute "tar --xattrs --acls --one-file-system -C #{@config.nfsroot} -cf #{@config.nfsroot}/var/tmp/base.tar --exclude etc/machine-id --exclude var/tmp/base.tar --exclude 'var/lib/apt/lists/*_*' ."
    execute "nice xz -q #{@config.nfsroot}/var/tmp/base.tar", wait: false
  end

# put contents of ramdisk back to original location on harddisk
  def umount_ramdisk path

disk = execvalue "readlink -f #{path}"
    ram = "#{disk}-in-ramdisk"

# test, if a ramdisk is mounted to this directory
    if `mount | grep -q " #{disk} type tmpfs"` && !$?.success?
        puts "#{disk} is not a mounted ramdisk."
        exit 1
    end

# Overly cautious check to be sure I don’t accidentally delete root.
    raise "Totally invalid disk var." if disk.strip.size < 3

# There may be a leftover ramdisk from a buggy run.
    if Dir.exist? ram
      # Sometimes there are zombie programs that still are running over the nfsroot.
      # Probably there is better place to kill them than here.
      execute "kill -TERM $(lsof -t /srv/fai/nfsroot)", fail: false, log: false

execute "umount #{ram}", log: false
      FileUtils.rmdir ram
    end

Dir.mkdir ram
    # move current ramdisk to a new location
    execute "mount --bind #{disk} #{ram}", log: false
    execute "umount #{disk}", log: false

FileUtils.rm_rf Dir.glob("#{disk}/*")
    FileUtils.rm_rf Dir.glob("#{disk}/.??*")

# copy ramdisk contents to disk
    execute "cp -a  #{ram}/. #{disk}/.", log: false

execute "umount #{ram}", log: false
    debug "Ramdisk on #{disk} umounted"
    FileUtils.rmdir ram
  end

def umount_dirs
    debug "Performing cleanup..."

Dir.chdir "/"
    execute "umount #{@config.nfsroot}/proc", log: false if Dir.exist? "#{@config.nfsroot}/proc/self"
    execute "umount #{@config.nfsroot}/sys", log: false if Dir.exist? "#{@config.nfsroot}/sys/class"
    raise "/proc still mounted inside the nfsroot." if Dir.exist? "#{@config.nfsroot}/proc/self"
    execute "umount #{@config.nfsroot}/dev/pts", log: false, fail: false

if `mount | grep -q "on #{@config.nfsroot}/var/lib/dpkg type tmpfs"` and $?.success?
      #execute "/usr/lib/fai/mkramdisk -u #{@config.nfsroot}/var/lib/dpkg #{@config.nfsroot}/var/cache", log: false, fail: false
      #execute "/usr/lib/fai/mkramdisk -u #{@config.nfsroot}/var/lib/dpkg #{@config.nfsroot}/var/cache", log: false
      umount_ramdisk "#{@config.nfsroot}/var/lib/dpkg"
      umount_ramdisk "#{@config.nfsroot}/var/cache"
    end
    # A leftover from when this script has had a bug in wrong place.
    if `mount | grep -q "/srv/fai/.will-now-be-deleted/var/lib/dpkg type tmpfs"` and $?.success?
      execute "/usr/lib/fai/mkramdisk -u /srv/fai/.will-now-be-deleted/var/lib/dpkg /srv/fai/.will-now-be-deleted/var/cache", log: false
    end

if !$FAI_DEBMIRROR.nil? and !$FAI_DEBMIRROR.empty?
      if Dir.exist? "#{@config.nfsroot}/#{@config.mirror_mountpoint}"
        #execute "umount #{@config.nfsroot}/#{@config.mirror_mountpoint}", log: false, fail: false
        execute "umount #{@config.nfsroot}/#{@config.mirror_mountpoint}", log: false
      end
      #execute "rmdir #{@config.nfsroot}/#{@config.mirror_mountpoint}", fail: false, log: false
      execute "rmdir #{@config.nfsroot}/#{@config.mirror_mountpoint}", log: false
    end

# show directories still mounted on nfsroot
    mounted = execvalue %{mount | grep " on #{@config.nfsroot} " || true}
    puts "[WARNING]: Still mounted: #{mounted}" unless mounted.empty?
  end

# Reads target and repository url from /etc/fai/apt/sources.list.
  # Compared to fai-make-nfsroot, this avoids having to pass extra data just for base creation.
  def resolve_sources
    path = "#{@fai_config_dir}/apt/sources.list"
    raise "#{path} does not exist" unless File.exist? path

lines = File.readlines path

# By convention, first real repo is main repo. Do some extra checks anyway to be sure.
    suite = nil
    repourl = nil
    lines.each do |line|
      next unless line[0..2] == "deb"
      splitted = line.split " "
      suite = splitted[2]
      repourl = splitted[1]

break unless suite.include? "/"
      suite = nil
      repourl = nil
    end

raise "Failed to parse #{path}" if suite.nil? or repourl.nil?

@repository_suite = suite
    @repository_url = repourl
  end

def call_debootstrap
    # check if NFSROOT directory is mounted with bad options
    fs = execvalue "df -P #{@config.nfsroot} | tail -1 | awk '{print $6}'"
    if execvalue("mount | grep \"on #{fs} \" |  awk '{print $6}'").match?(/nosuid|nodev/)
      raise "NFSROOT directory #{@config.nfsroot} is mounted using nosuid or nodev. Aborting"
    end

debootstrap_version = execvalue("debootstrap --version").split(" ")[1]
    verb "Creating base system using debootstrap version #{debootstrap_version}"

# Check if we need cross architecture debootstrap
    targetarch = execvalue %{echo "$(expr "#{@config.fai_debootstrap_opts}" : '.*--arch[=[:space:]]$[^[:space:]]*$' || true)" | tail -n 1}
    hostarch1 = execvalue "dpkg --print-architecture"
    hostarch2 = execvalue "dpkg --print-foreign-architectures"
    hostarch2 = hostarch1 if hostarch2.empty?

debootstrap = "debootstrap"

=begin
    _debootstrap=qemu-debootstrap
    if [ -z "$targetarch" ]; then
        _debootstrap=debootstrap
    else
        if [ $targetarch = $hostarch1 -o $targetarch = $hostarch2 ]; then
            _debootstrap=debootstrap
        fi
    fi
    if [ $_debootstrap = "qemu-debootstrap" ]; then
        if ! command -v $_debootstrap >&/dev/null; then
            die 1 "qemu-debootstrap not found. Please install the package qemu-user-static."
        fi
    fi
=end
    # suite target mirror
    # Calling debootstrap --exclude=wget --include=ca-certificates,gnupg buster /srv/fai/nfsroot http://ryza.smar.fi:3142/debian/debian

verbose = $_verbose ? "--verbose" : ""
    command = "#{debootstrap} #{verbose} #{@config.fai_debootstrap_opts} #{@repository_suite} #{@config.nfsroot} #{@repository_url}"
    verb "Calling #{command}"
    execute command, fail: false
    unless $?.success?
      puts "ERROR: #{debootstrap} did not complete successfully."
      puts "This is mostly caused by a broken mirror."
      puts "Please check your mirror or use an official mirror."
      puts "Exit code was: #{$?.exitstatus}"
      exit 10
    end

unless File.exist? "#{@config.nfsroot}/usr/bin/apt-get"
      puts "No apt-get executable available inside the NFSROOT."
      puts "Maybe #{debootstrap} did not finish successfully. Aborting."
      exit 11
    end
  end

end
07070100000017000081A4000001C3000001CA000000015F6847FC00000B11000000000000000000000000000000000000002D00000000fai-toolkit-1600669692.02e37ec/lib/common.rb
require_relative "output"

# Common utilities

module Common

# Everything should only be usable for the class inheriting this module.
  private

include Output

class CommandError < StandardError
  end

class ExecutionFailed < CommandError
  end

@@opts = nil
  @@nfsroot = nil
  @@rootcmd = nil

def opts
    @@opts
  end

def set_opts options
    @@opts = options

if options[:verbose]
        $verbose = "-v"
        $_verbose = true
    else
        $verbose = ""
    end

$_debug = true if options[:debug]
  end

def rootcmd
    raise "rootcmd is not set." if @@rootcmd.nil?
    @@rootcmd
  end

def rootcmd= cmd
    @@rootcmd = cmd
  end

def nfsroot
    raise "nfsroot is not set." if @@nfsroot.nil?
    @@nfsroot
  end

def nfsroot= path
    set_nfsroot path
  end

def set_nfsroot path
    @@nfsroot = path
    @@rootcmd = %{chroot #{@@nfsroot} env LC_ALL="C"}
  end

# Check if requested program can be found.
  #
  # Should work cross-platform and in restricted environents pretty well.
  def command_available? command
    exts = ENV["PATHEXT"] ? ENV["PATHEXT"].split(';') : ['']
    ENV["PATH"].split(File::PATH_SEPARATOR).each do |path|
      exts.each do |ext|
        exe = File.join(path, "#{command}#{ext}")
        return true if File.executable?(exe) && !File.directory?(exe)
      end
    end
    false
  end

# Returns pid of executed command if wait is false.
  # Returns command output if wait is true.
  #
  # fail == If true, an error is raised in case the command returns failure code.
  def execute command, fail: true, wait: true, log: true, direct_output: false, env: {}
    debug "Executing “#{command.force_encoding('UTF-8')}” at #{caller_locations(1,1)[0]}" if log == true

r, w = IO.pipe
    if direct_output
      pipe_w = w
      w = STDOUT
    end
    pid = spawn(env, command, err: [:child, :out], out: w)
    w = pipe_w if direct_output
    w.close
    unless wait
      r.close
      return pid
    end
    buffer = String.new capacity: 1024
    output = ""
    while not r.eof?
      r.read_nonblock 1024, buffer
      if log
        print buffer if $_verbose
        Output.logfile.write buffer unless Output.logfile.nil?
      end
      output << buffer
    end
    r.close

# This is instant return, but allows us to have $? set.
    Process.wait pid

raise ExecutionFailed, "Command execution failed.", caller if fail == true and !$?.success?

output.strip
  end

# Same as ::execute, but uses log: false to avoid unnecessary output when we’re just getting a value for internal needs.
  #
  # Defaults to fail: false, since when one just wants to get the value, there is not that much need to unconditionally die if getting bad exit code.
  def execvalue *args, fail: false
    execute *args, fail: fail, log: false
  end
end
07070100000018000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000003100000000fai-toolkit-1600669692.02e37ec/lib/configuration07070100000019000081A4000001C3000001CA000000015F6847FC00000044000000000000000000000000000000000000004200000000fai-toolkit-1600669692.02e37ec/lib/configuration/configuration.rb
# Base module for configuration utility.
module Configuration

end
0707010000001A000081A4000001C3000001CA000000015F6847FC00000DCF000000000000000000000000000000000000004700000000fai-toolkit-1600669692.02e37ec/lib/configuration/sync-configuration.rb
require "optimist"
require "psych"
require "socket"

require_relative "../common"
require_relative "../output"
require_relative "configuration"

# Main class.
#
# Source configuration dir is expected to follow structured layout.
# TODO: this needs to be documented properly.

# TODO: identifier can be changed to site.
class Configuration::SyncConfiguration
  include Common
  include Output

def perform
    startup_checks
    real_perform
  end

def options
    options = Optimist::options do
      opt :confdir,               "Path to directory where configuration will be read from",
          type: :string,    short: "c"
      opt :target,                "Name of the target host",
          type: :string,    short: "t", required: true
      opt :site,                  "Identifier that describes where the host is",
          type: :string,    short: "s", required: true
      opt :debug,                 "Show debug output",
          short: "d"
      opt :verbose,               "Show verbose output",
          short: "v"
    end

set_opts options

@target_host = options[:target]
    @target_site = options[:site]
    @configuration_dir = options[:confdir]

if @configuration_dir.nil? or !Dir.exist? @configuration_dir
      raise "Given configuration directory “#{@configuration_dir}” does not exist."
    end
  end

private

# By default, configuration is transferred under /etc to a folder which name comes from package name in hosts.yaml.
  def real_perform
    @configuration.each do |identifier, packages|
      packages.each do |package|
        local_config_path = "#{@configuration_dir}/#{identifier}/#{package}/#{@target_site}/*"
        remote_config_path = "/etc/#{package}/"
        execute %{rsync -avz -e "ssh -p #{@remote_ssh_port}" #{local_config_path} root@#{@remote_ip}:#{remote_config_path}}
        debug "Sync completed."
      end
    end
  end

def startup_checks
    config = Psych.safe_load(File.readlines("#{@configuration_dir}/hosts.yaml").join)
    raise "hosts.yaml in given configuration directory is malformed." if config["sites"].nil?
    sites = config["sites"]
    raise "Given target site #{@target_site} is missing from hosts.yaml" if sites[@target_site].nil?
    site = sites[@target_site]
    raise "Given target #{@target_host} is missing from hosts.yaml" if site[@target_host].nil?
    host = site[@target_host]

@remote_ip = host["remote_ip"]
    @remote_ssh_port = host["remote_ssh_port"]

if @remote_ip.nil? or @remote_ssh_port.nil?
      raise "remote_ip and remote_ssh_port must be specified in hosts.yaml."
    end

begin
      Socket.tcp(@remote_ip, @remote_ssh_port, connect_timeout: 3) {}
    rescue Errno::ETIMEDOUT => e
      puts.STDERR "Connecting to host #{@remote_ip}:#{@remote_ssh_port} failed in 3 seconds. Please check if there is a connection problem."
      raise e
    end

raise "Configuration is not specified for given host in hosts.yaml" if host["configuration"].nil?
    @configuration = host["configuration"]

# Some extra checks to see necessary config really exists.
    @configuration.each do |identifier, packages|
      unless Dir.exist? "#{@configuration_dir}/#{identifier}"
        raise "Configuration directory for identifier #{identifier} is missing."
      end

packages.each do |package|
        unless Dir.exist? "#{@configuration_dir}/#{identifier}/#{package}"
          raise "Configuration files for package #{identifier}/#{package} are missing."
        end
      end
    end
  end
end0707010000001B000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000003500000000fai-toolkit-1600669692.02e37ec/lib/create-base-image0707010000001C000081A4000001C3000001CA000000015F6847FC00000E3A000000000000000000000000000000000000004300000000fai-toolkit-1600669692.02e37ec/lib/create-base-image/base-image.rbrequire "fileutils"
require "tmpdir"
require "psych"

require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "methods"

# Main class.
class CreateBaseImage::BaseImage
  include Common
  include Output
  include CreateBaseImage::Methods

def perform
    startup_checks
    real_perform
  end

def options
    options = Optimist::options do
      opt :list_distributions,    "Show list of supported distributions"
      opt :foreign_arch,          "Build for given foreign architecture",                           type: :string
      opt :distribution,          "Distribution to build the image from",                           type: :string
      opt :preserve_tmpdir,       "If given, preserves temporary files after script has finished.", type: :boolean, default: false
      opt :fai_config_space,      "Path to FAI’s config space.",                                    type: :string,  default: "/srv/fai/config"
      opt :debug,                 "Show debug output"
      opt :verbose,               "Show verbose output"
    end

if options[:list_distributions]
      print_distributions
      exit 1
    end

if options[:distribution].nil? or options[:distribution].size < 1
      raise "--distribution must be given."
    end

unless Dir.exist? options[:fai_config_space]
      raise "FAI’s config space is used for base image configuration, but config space is missing. Please provide correct config space or create a new one."
    end

@create_base_image_config_file = "#{options[:fai_config_space]}/basefiles/create_base_image.yaml"
    unless File.exist? @create_base_image_config_file
      raise "FAI’s config space is used for base image configuration, but configuration file #{@create_base_image_config_file} is missing. Please add correct config file to config path. For now, you need to investigate this script how to do that."
    end
    lines = File.readlines(@create_base_image_config_file).join("\n")
    @create_base_image_config = Psych.safe_load lines

set_opts options
  end

private

def real_perform
    # These could be used to use other compression algos.
    @ext = "tar.xz"
    @zip = "xz -8"
    @tar_attributes = "--xattrs --selinux --acls"

@tmpdir = Dir.mktmpdir "create-base-image_"
    FileUtils.chmod 755, @tmpdir

# also the name of the output file
    target = opts[:distribution]

case target
    when "OPENSUSE_15_1_32"
      opensuse target, "i386", "15.1"
    when "OPENSUSE_15_1_64"
      opensuse target, "amd64", "15.1"
    when "CENTOS5_32"
      centos target, "i386", 5
    when "CENTOS5_64"
      centos target, "amd64", 5
    when "CENTOS6_32"
      centos target, "i386", 6
    when "CENTOS6_64"
      centos target, "amd64", 6
    when "CENTOS7_32"
      centos target, "i386", 7
    when "CENTOS7_64"
      centos target, "amd64", 7
    when "CENTOS8_64"
      centos target, "amd64", 8
    when "SLC5_32"
      slc target, "i386", 5
    when "SLC5_64"
      slc target, "amd64", 5
    when "SLC6_32"
      slc target, "i386", 6
    when "SLC6_64"
      slc target, "amd64", 6
    when "SLC7_64"
      slc target, "amd64", 7
    when /TRUSTY*|XENIAL*|BIONIC*/
      debgeneric target, @create_base_image_config["MIRROR_UBUNTU"]
    when /SQUEEZE*|WHEEZY*|JESSIE*|STRETCH*|BUSTER*|SID*/
      # TODO: arch
      debgeneric target, @create_base_image_config["MIRROR_DEBIAN"], $ARCH
    else
      puts "Unknown distribution. Aborting."
      exit 1
    end

# cleanup
    if not opts[:preserve_tmpdir] == true
      FileUtils.rm_rf @tmpdir
    end
  end

def startup_checks
    # Nya.
  end

end

0707010000001D000081A4000001C3000001CA000000015F6847FC0000001D000000000000000000000000000000000000004A00000000fai-toolkit-1600669692.02e37ec/lib/create-base-image/create-base-image.rb
module CreateBaseImage
end

0707010000001E000081A4000001C3000001CA000000015F6847FC000017D2000000000000000000000000000000000000004000000000fai-toolkit-1600669692.02e37ec/lib/create-base-image/methods.rb
require_relative "create-base-image"

module CreateBaseImage::Methods

def setarch arch
    arch == "i386" ? "linux32" : ""
  end

def check
    execute "mknod #{@tmpdir}/test-dev-null c 1 3"
    unless $?.success?
      puts "Cannot create device files on #{@tmpdir}, aborting."
      puts "Perhaps this directory is mounted with option nodev."
      FileUtils.rm_rf @tmpdir
      exit 1
    end

testfile = "#{@tmpdir}/test-dev-null"
    File.open testfile, "w" do |file|
      file.write "test"
    end

unless File.exist? testfile
      puts "Cannot create device files on #{@tmpdir}, aborting."
      puts "Perhaps this directory is mounted with option nodev."
      FileUtils.rm_rf @tmpdir
      exit 1
    end

FileUtils.rm_f testfile
  end

def mkpost_opensuse
    raise "MIRROR_OPENSUSE missing." if @create_base_image_config["MIRROR_OPENSUSE"].empty?

TODO
=begin
    postfile = "#{@tmpdir}/post"
    File.open postfile, "w" do |file|
      file.write <<~EOF
      #!/bin/sh
      mkdir -p #{@tmpdir}/etc/yum.repos.d/orig
      cp -p #{@tmpdir}/etc/yum.repos.d/*.repo #{@tmpdir}/etc/yum.repos.d/orig
      perl -pi -e 's,mirrorlist=,#mirrorlist=,; s,#baseurl=http://mirror.centos.org,baseurl=#{@create_base_image_config["MIRROR_CENTOS"]},;' #{@tmpdir}/etc/yum.repos.d/CentOS-Base.repo
      EOF
    end

FileUtils.chmod 555, postfile
=end
  end

def mkpost_centos
    raise "MIRROR_CENTOS missing." if @create_base_image_config["MIRROR_CENTOS"].empty?

postfile = "#{@tmpdir}/post"
    File.open postfile, "w" do |file|
      file.write <<~EOF
      #!/bin/sh
      mkdir -p #{@tmpdir}/etc/yum.repos.d/orig
      cp -p #{@tmpdir}/etc/yum.repos.d/*.repo #{@tmpdir}/etc/yum.repos.d/orig
      perl -pi -e 's,mirrorlist=,#mirrorlist=,; s,#baseurl=http://mirror.centos.org,baseurl=#{@create_base_image_config["MIRROR_CENTOS"]},;' #{@tmpdir}/etc/yum.repos.d/CentOS-Base.repo
      EOF
    end

FileUtils.chmod 555, postfile
  end

def mkpost_slc version
    raise "MIRROR_SLC missing" if @create_base_image_config["MIRROR_SLC"].empty?

postfile = "#{@tmpdir}/post"
    File.open postfile, "w" do |file|
      file.write <<~EOF
      #! /bin/sh
      mkdir -p #{@tmpdir}/etc/yum.repos.d/orig
      cp -p #{@tmpdir}/etc/yum.repos.d/*.repo #{@tmpdir}/etc/yum.repos.d/orig
      perl -pi -e 's,baseurl=http://linuxsoft.cern.ch,baseurl=#{@create_base_image_config["MIRROR_SLC"]},;' #{@tmpdir}/etc/yum.repos.d/slc#{version}-os.repo
      perl -pi -e 's,baseurl=http://linuxsoft.cern.ch,baseurl=#{@create_base_image_config["MIRROR_SLC"]},;' #{@tmpdir}/etc/yum.repos.d/slc#{version}-updates.repo
      EOF
    end

FileUtils.chmod 555, postfile
  end

def cleanup_deb
    execute "chroot #{@tmpdir} apt-get clean"
    FileUtils.rm_f [ "#{@tmpdir}/etc/hostname", "#{@tmpdir}/etc/resolv.conf",
          "#{@tmpdir}/var/lib/apt/lists/*_*", "#{@tmpdir}/usr/bin/qemu-*-static",
          "#{@tmpdir}/etc/udev/rules.d/70-persistent-net.rules" ]
    File.truncate "#{@tmpdir}/etc/machine-id", 0
  end

def cleanup_rinse

# check if chroot works
    puts "Installed packages in chroot:"
    puts execvalue "chroot #{@tmpdir} rpm -qa|sort"
    print "CHROOT rpm -qa: "
    puts execvalue "chroot #{@tmpdir} rpm -qa|wc -l"

FileUtils.rm_f [ "#{@tmpdir}/etc/resolv.conf", "#{@tmpdir}/post" ]
    if Dir.exist? "#{@tmpdir}/etc/yum.repos.d/orig"
        execute "mv #{@tmpdir}/etc/yum.repos.d/orig/* #{@tmpdir}/etc/yum.repos.d/"
        FileUtils.rm_rf "#{@tmpdir}/etc/yum.repos.d/orig"
    end
  end

def tarit filename
    execute "tar #{@tar_attributes} --numeric-owner --one-file-system -C #{@tmpdir} -cf - . | #{@zip} > #{filename}.#{@ext}"
  end

def opensuse distribution, arch, version
    check
    l32 = setarch arch
    #mkpost_opensuse # TODO: It looks like something not useful... Could do same feature with --mirror directly?
    execute "#{l32} rinse --directory #{@tmpdir} --distribution opensuse-#{version} --arch #{arch} --before-post-install #{@tmpdir}/post"
    cleanup_rinse
    tarit distribution
  end

def centos distribution, arch, version
    domain = execvalue "domainname"

check
    l32 = setarch arch
    mkpost_centos
    execute "#{l32} rinse --directory #{@tmpdir} --distribution centos-#{version} --arch #{arch} --before-post-install #{@tmpdir}/post"
    execute "domainname domain" # workaround for #613377
    cleanup_rinse
    tarit distribution
  end

def slc distribution, arch, version

check
    l32 = setarch arch
    mkpost_slc version
    execute "#{l32} rinse --directory #{@tmpdir} --distribution slc-#{version} --arch #{arch} --before-post-install #{@tmpdir}/post"
    cleanup_rinse
    tarit distribution
  end

def debgeneric distribution, mirror, arch

raise "Not done. You probably should use fai’s example script for Debian instead."

# TODO: no idea what to do with this dist thing.

#dist=${DIST%%[0-9][0-9]}
    #local exc="EXCLUDE_$dist"
    #[ -n "${!exc}" ] && exc="--exclude=${!exc}" || unset exc
    #dist=${dist,,}

check

#if [ -n "$INCLUDE_DEBIAN" ]; then
    #  local inc="--include=$INCLUDE_DEBIAN"
    #fi

unless arch.empty?
      execute "qemu-debootstrap --arch #{arch} #{exc} #{inc} #{distribution} #{@tmpdir} #{mirror}"
      # If this gets implemented, this target needs to be handled.
      target = "#{target}_#{arch}"
    else
      arch = "i386"
      arch = "amd64" if distribution.include? "64"
      execute "debootstrap --arch #{arch} #{exc} #{inc} #{distribution} #{@tmpdir} #{mirror}"
    end
    cleanup_deb
    tarit distribution
  end

def print_distributions

puts <<EOF
Available:

CENTOS5_32   CENTOS5_64
    CENTOS6_32   CENTOS6_64
    CENTOS7_32   CENTOS7_64
                 CENTOS8_64
    SLC5_32      SLC5_64
    SLC6_32      SLC6_64
                 SLC7_64
    TRUSTY32     TRUSTY64
    XENIAL32     XENIAL64
                 BIONIC64
    SQUEEZE32    SQUEEZE64
    WHEEZY32     WHEEZY64
    JESSIE32     JESSIE64
    STRETCH32    STRETCH64
    BUSTER32     BUSTER64
    SID32        SID64
EOF
  end

end
0707010000001F000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000003500000000fai-toolkit-1600669692.02e37ec/lib/create-fai-mirror07070100000020000081A4000001C3000001CA000000015F6847FC0000001C000000000000000000000000000000000000004A00000000fai-toolkit-1600669692.02e37ec/lib/create-fai-mirror/create-fai-mirror.rb
module CreateFaiMirror
end
07070100000021000081A4000001C3000001CA000000015F6847FC00001D87000000000000000000000000000000000000003F00000000fai-toolkit-1600669692.02e37ec/lib/create-fai-mirror/debian.rbrequire "fileutils"

require_relative "create-fai-mirror"
require_relative "../common"

class CreateFaiMirror::Debian
  include Common

def initialize opts
    @opts = opts
  end

# Installs packages to the mirror.
  def install_packages package_list
    package_list = package_list.join(" ") if package_list.respond_to? :join
    execute "apt-get -d #{@aptoptions} -y --fix-missing install #{package_list}", direct_output: true
  end

def refresh_repositories
    execute "apt-get #{@aptoptions} update", log: false
  end

def _post_initialize mirror_directory

# TODO: root is only needed when FAI_DEBMIRROR is defined. Then we must mount a directory.

# holds the package cache data
    @aptcache = "#{mirror_directory}/aptcache"

@archivedir = "#{@aptcache}/var/cache/apt/archives"
    statefile = "#{@aptcache}/statefile"

# also used in install_packages.conf
    @aptoptions = <<~EOF
      -o Aptitude::Log=/dev/null
      -o Aptitude::CmdLine::Ignore-Trust-Violations=yes
      -o APT::Get::AllowUnauthenticated=true
      -o Acquire::AllowInsecureRepositories=true
      -o DPkg::force-conflicts::=yes
      -o Dir::State=#{@aptcache}/var/lib/apt
      -o Dir::Log=#{@aptcache}/var/log/apt
      -o Dir::State::extended_states=#{@aptcache}/var/lib/apt/lists/extended_states
      -o Dir::State::status=#{statefile}
      -o Dir::Cache=#{@aptcache}/var/cache/apt
      -o Dir::State=#{@aptcache}/var/cache/apt
      -o Dir::Cache::Archives=#{@archivedir}
      -o Dir::Etc=#{@aptcache}/etc/apt/
      -o Dir::State::Lists=#{@aptcache}/var/lib/apt/lists/
    EOF
    @aptoptions.gsub! "\n", " "

# Add architecture.
    if !$arch.nil? and !$arch.empty?
      @aptoptions += " -o APT::Architecture=#{$arch}"
    end

# Used by install_packages.
    ENV["aptoptions"] = @aptoptions
    ENV["FAI_ARCHIVE_DIR"] = @archivedir

# we only need some empty dirs
    FileUtils.mkdir_p [ "#{@archivedir}/partial", "#{@aptcache}/etc/apt/preferences.d", "#{@aptcache}/var/lib/apt/lists/partial", "#{@aptcache}/var/log/apt" ]
    File.open statefile, "w" do |f|
      # Empties statefile.
    end
  end

def startup_checks
    raise "reprepro not found. Please install the package." unless command_available? "reprepro"
  end

# Hooks to make it visually easier to understand the flow.

def _pre_real_perform fai_config_dir, mirror_location
    # if we are using nfs mounts for Debian mirror, this may fail here, since inside a chroot environment different dir are used
    # if sources.list includes file AND FAI_DEBMIRROR is defined we Mhave to mount
    # otherwise mounting is not needed. call task_mirror
    if !$FAI_DEBMIRROR.nil? and !$FAI_DEBMIRROR.empty?
        FileUtils.mkdir_p @mirror_location
        execute "mount -r #{$FAI_DEBMIRROR} #{@mirror_location}"
    end

# TODO: use -p to preserve sources.list
    execute "sed -e 's/file:/copy:/' #{fai_config_dir}/apt/sources.list > #{@aptcache}/etc/apt/sources.list"
    test = execvalue "ls -A #{fai_config_dir}/apt/sources.list.d/ 2> /dev/null"
    unless test.empty?
      execute "sed -e 's/file:/copy:/' #{fai_config_dir}/apt/sources.list.d/* >> #{@aptcache}/etc/apt/sources.list"
    end

FileUtils.cp(Dir.glob("#{fai_config_dir}/apt/preferences.d/*"), "#{@aptcache}/etc/apt/preferences.d/")

# aptpref comes from cli arg on original script. Not implemented.
    #if File.exist? $aptpref
    #  FileUtils.cp $aptpref, "#{@aptcache}/etc/apt/preferences"
    #end

# use the trusted keys from the host
    execute "cp -a /etc/apt/trusted.gpg.d #{@aptcache}/etc/apt"
  end

def _post_real_perform mirror_directory
    # in still undefined, use host architecture
    if $arch.nil? or $arch.empty?
      $arch = execvalue "dpkg  --print-architecture"
      $arch = "#{$arch} #{execvalue "dpkg --print-foreign-architectures"}"
    end

# create mirror directory structure
    verb "Calling reprepro"
    FileUtils.mkdir "#{mirror_directory}/conf" unless Dir.exist? "#{mirror_directory}/conf"
    File.open "#{mirror_directory}/conf/distributions", "w" do |f|
      f.write <<~EOF
      # generate config file for reprepro
      Codename: cskoeln
      Architectures: #{$arch}
      Components: main non-free contrib
      DebIndices: Packages Release . .xz
      Description: Package reprository created by fai-mirror
      Label: fai-project.org
      Origin: fai-mirror
      EOF
    end

# check if backports are used
    bponame = execvalue "grep -E '^deb ' #{@aptcache}/etc/apt/sources.list | grep -P -o '\S+-backports'"

# add entry for backports
    unless bponame.empty?
      File.open "#{mirror_directory}/conf/distributions", "w" do |f|
        f.write <<~EOF

Codename: #{bponame}
        Architectures: #{$arch}
        Components: main non-free contrib
        DebIndices: Packages Release . .xz
        Description: Package reprository created by fai-mirror
        Label: fai-project.org
        Origin: fai-mirror
        EOF
      end

# maybe using reprepro pulls it's possible to move instead of copy the packages
      execvalue = "ls #{@archivedir}/*~bpo[0-9]*+[0-9-]*_*.deb"
      if $?.success and !execvalue.empty?
        execute "reprepro -b #{mirror_directory} includedeb #{$bponame} #{@archivedir}/*~bpo[0-9]*+[0-9-]*_*.deb"
        FileUtils.rm_f Dir.glob("#{@archivedir}/*~bpo[0-9]*+[0-9-]*_*.deb")
      end
    end

# Allow package without the .deb suffix. At least they must start with a char or number and contain an underscore or hyphen.
    execute "reprepro --ignore=extension -b #{mirror_directory} includedeb cskoeln #{@archivedir}/[0-9a-zA-z]*[_-]*"
    FileUtils.rm_r Dir.glob("#{@archivedir}/[0-9a-zA-z]*[_-]*")
  end

# Quasi-hooks

def download_packages_for_classes classes, fai_config_space
    verb "Downloading packages for classes: #{classes}"

vflag = ""
    vflag << " -v" if @opts[:verbose]

execute %{FAI=#{fai_config_space} install_packages -N -d #{vflag}}, direct_output: true, fail: false
    unless $?.success?
        raise "ERROR when downloading packages. Your mirror probably is broken."
    end
  end

def use_temporary_sources_list
    # Save original sources.list
    FileUtils.mv "#{@aptcache}/etc/apt/sources.list", @aptcache

# Construct FAI_DEBOOTSTRAP string from sources.list

#FAI_DEBOOTSTRAP="buster http://ryza.smar.fi:3142/debian/debian"
    # Let’s assume first entry is main source.
    config_dir_sources_list = "#{@opts[:fai_config_dir]}/apt/sources.list"
    lines = File.readlines config_dir_sources_list
    repo_url = nil
    repo_target = nil
    lines.each do |line|
      line.strip!
      if line[0..3] == "deb "
        splitted = line.split " "
        next if splitted.size < 3

repo_url = splitted[1]
        repo_target = splitted[2]
        break
      end
    end

raise "Failed to parse main repo from #{config_dir_sources_list}." if repo_target.nil?
    fai_debootstrap = "#{repo_target} #{repo_url}"

# now use different sources.list for debootstrap packages
    execute %{echo "#{fai_debootstrap}" | awk '{printf "deb %s %s main\\n",$2,$1}' | sed -e 's/file:/copy:/' > #{@aptcache}/etc/apt/sources.list}
  end

def restore_sources_list
    FileUtils.mv "#{@aptcache}/sources.list", "#{@aptcache}/etc/apt/sources.list"
    refresh_repositories
  end

def try_base_package_delete p, arch
    if Dir.glob("#{@archivedir}/#{p}_*#{arch}.deb").size > 0
      debug "deleting package #{p} #{arch}"
      FileUtils.rm Dir.glob("#{@archivedir}/#{p}_*#{arch}.deb")
    end
  end
end07070100000022000081A4000001C3000001CA000000015F6847FC00002990000000000000000000000000000000000000004300000000fai-toolkit-1600669692.02e37ec/lib/create-fai-mirror/fai-mirror.rb
require "optimist"

require_relative "../fai-config"
require_relative "create-fai-mirror"
require_relative "methods"
require_relative "debian"
require_relative "suse"

# TODO: probably either --run-inside-nfsroot should use the chroot nfsroot for operation, or something wiser needs to be done for this.

# NOTE: Generating the mirror is a bit annoying, since what is going to included is dependent on classes.
# And since classes can be distribution agnostic, I think there should be some system to automatically resolve necessary dependencies.
# For now, classes that should be included should be specified as argument to this command. Though that’s not yet implemented..
class CreateFaiMirror::FaiMirror
  include Common
  include Output
  include CreateFaiMirror::Methods

def perform
    startup_checks
    real_perform
  end

def options
    argv_copy = ARGV.dup

options = Optimist::options do
      opt :fai_config_dir,    "FAI’s system configuration directory",                   type: :string,    default: "/etc/fai",            short: "c"
      opt :fai_config_space,  "FAI’s config space",                                     type: :string,    default: "/srv/fai/config",     short: "s"
      opt :mirror_location,   "Directory where mirror will be generated to",            type: :string,    default: "/srv/fai/mirror",     short: "m"
      opt :force,             "Overwrite existing mirror",                              type: :boolean,   default: false,                 short: "f"
      opt :target,            "Target distribution. Supported values: debian, suse",    type: :string,                                    short: "t"
      opt :exclude_class,     "Excludes given class if it otherwise would be present",  type: :string, multi: true,                       short: "x"
      #opt :set_variables,     "Reads and sets variables from class/*.var if used in package_config/<classes>",                            short: "V"
      opt :nfsroot,           "Override FAI nfsroot location from FAI config dir’s nfsroot.conf",
                                                                                        type: :string,                                    short: "n"
      opt :run_inside_nfsroot, "Run inside specified FAI nfsroot. This is a path to FAI config, for example /etc/fai.",
                                                                                        type: :string,                                    short: "r"
      opt :update,            "Update instead of creating new mirror",                  type: :boolean,   default: false,                 short: "u"
      opt :maxpackages,       "How many packages will be processed with single apt-get install command",
                                                                                        type: :integer,   default: 1,                     short: "M"
      opt :debug,             "Show debug output",                                                                                        short: "d"
      opt :verbose,           "Show verbose output",                                                                                      short: "v"
    end

set_opts options

unless Dir.exist? "#{options[:fai_config_space]}/class"
      raise "FAI config space seems to be missing or invalid. Create a valid config space or specify path to one with --fai-config-space."
    end
    @fai_config_space = options[:fai_config_space]

# First check if we need to hop into a chroot.
    unless options[:run_inside_nfsroot].nil?
      run_inside_nfsroot argv_copy, options
      exit
    end

# Then parse options.

unless Dir.exist? options[:fai_config_dir]
      raise "#{options[:fai_config_dir]} does not exist. Please create it or specify alternative path with --fai-config-dir."
    end
    @fai_config_dir = options[:fai_config_dir]

@config = FaiConfig.new @fai_config_dir
    @config.load_nfsroot_conf

if options[:nfsroot].nil?
      set_nfsroot @config.nfsroot

# Not sure what uses this, but it’s exported in original script.
      ENV["NFSROOT"] = @config.nfsroot
    else
      if Dir.exist? options[:nfsroot]
        set_nfsroot options[:nfsroot]

# Not sure what uses this, but it’s exported in original script.
        ENV["NFSROOT"] = options[:nfsroot]
      else
        raise "#{options[:nfsroot]} does not exist. Please create it or specify alternative path with --nfsroot."
      end
    end

@target_distribution = options[:target]
    case @target_distribution
    when "debian"
      raise "apt-get is missing. Are you sure you are on Debian?" unless command_available? "apt-get"
      @installer = CreateFaiMirror::Debian.new options
    when "suse"
      raise "zypper is missing. Are you sure you are on SUSE?" unless command_available? "zypper"
      @installer = CreateFaiMirror::Suse.new options
    else
      raise "Invalid target distribution: #{@target_distribution.inspect}"
    end

if options[:force] and options[:update]
      raise "Only --force or --update can be given."
    end

if options[:mirror_location].size < 3
      raise "Please specify proper mirror location. Root is not allowed for safety."
    end
    if Dir.exist? options[:mirror_location]
      if options[:force]
        FileUtils.rm_rf "#{options[:mirror_location]}/#{@target_distribution}"
      elsif options[:update]
        # Updating, so there is nothing else to do.
      else
        raise "Mirror #{options[:mirror_location]} already exists. If you want to redo it, use --force. If you want to update it, use --update."
      end
    else
      if options[:update]
        raise "For --update, mirror dir #{options[:mirror_location]} needs to exist."
      else
        FileUtils.mkdir options[:mirror_location]
      end
    end
    @mirror_location = options[:mirror_location]

@excluded_classes = options[:exclude_class]

@maxpackages = options[:maxpackages]

end

private

def real_perform

@installer._pre_real_perform @fai_config_dir, @mirror_location

verb "Getting package information"
    @installer.refresh_repositories

# This is basic operation, i.e. no -B or -b flags in fai-mirror.
    # TODO: if those flags are added, this needs to be guarded.
    #[ $add -eq 1 ] && add_base_packages
    add_base_packages

@installer.download_packages_for_classes @classes, @fai_config_space

umount_dirs

# TODO: implement?
    #[ $add -eq 0 ] && delete_base_packages

@installer._post_real_perform @mirror_directory

verb "create-fai-mirror finished."
    if opts[:debug]
      debugdf = execvalue "du -sh --exclude aptcache --exclude db #{@mirror_directory}"
      debug "Mirror location: #{@mirror_directory.inspect}"
      debug "Mirror size: #{debugdf}"
    end
    cleandirs
  end

def run_inside_nfsroot argv_copy, options
    nfsroot_fai_config_dir = options[:run_inside_nfsroot]

raise "When using --run-inside-nfsroot, root is required (to construct nfsroot)." unless Process.uid == 0

verb ""
    verb "[IMPORTANT] Using chroot nfsroot for mirroring; with a separate version of this script inside chroot."
    verb ""
    sleep 1.8

unless Dir.exist? "#{nfsroot_fai_config_dir}"
      raise "Given FAI config dir #{nfsroot_fai_config_dir} does not exist. Please create your config dir for the mirror nfsroot."
    end

nfsroot_config = FaiConfig.new nfsroot_fai_config_dir
    nfsroot_config.load_nfsroot_conf

vdflags = ""
    vdflags << "-d" if options[:debug]
    vdflags << " -v" if options[:verbose]

unless Dir.exist? nfsroot_config.nfsroot
      if Dir.exist? nfsroot_config.nfsroot
        raise "FAI nfsroot #{nfsroot_config.nfsroot} seems to already exist, but it’s not valid nfsroot. Please move it away if it is unwanted or use another path."
      end

execute %{create-nfsroot --fai-config-dir #{nfsroot_fai_config_dir} #{vdflags}}, direct_output: true
    else
      execute %{create-nfsroot --fai-config-dir #{nfsroot_fai_config_dir} #{vdflags} --refresh}, direct_output: true
    end

# Sync FAI config to the chroot.
    cleaned = nfsroot_fai_config_dir.delete_suffix "/"
    execute %{rsync -a #{cleaned}/ #{nfsroot_config.nfsroot}/#{cleaned}/}

# Sync FAI config space to the chroot.
    cleaned = @fai_config_space.delete_suffix "/"
    nfsroot_config_space_path = "#{nfsroot_config.nfsroot}/#{nfsroot_config.config_space}/"
    dir = File.dirname nfsroot_config_space_path
    Dir.mkdir dir unless Dir.exist? dir
    execute %{rsync -a #{@fai_config_space}/ #{nfsroot_config_space_path}}

# Instruct to use correct FAI config
    args = []
    argv_copy.each do |arg|
      if arg == "-r" or arg == "--run-inside-nfsroot"
        arg = "--fai-config-dir"
      end
      args << arg
    end
    args = args.join " "

#execute %{mount --bind }

execute %{chroot #{nfsroot_config.nfsroot} create-fai-mirror #{args} --nfsroot /}
  end

def startup_checks
    at_exit do
      umount_dirs
    end

# This should trick install_packages not to use chroot when we are
    # already in chroot, by --run-inside-nfsroot.
    ENV["FAI_ROOT"] = "/" if opts[:nfsroot] == "/"

#verb "Using configuration files from #{@fai_config_dir}"
    #[ $bcount -gt 1 ] && die 7 "You can't use -b and -B simultaneously."
    #. #{@fai_config_dir}/fai.conf
    #. #{@fai_config_dir}/nfsroot.conf

@installer.startup_checks

#[ -n "$exclasses" -a -n "$cclasses" ] && die 3 "Options -x and -c not allowed at the same time."

# use first argument if given, use variable mirrordir if not argument was given
    #[ -n "$1" ] && mirrordir=$1
    #[ -z "@mirror_directory" ] && die 2 "Please give the absolute path to the mirror."
    #{ echo @mirror_directory | grep -E -q '^/'; } || die 4 "Mirrordir must start with a slash /."

@mirror_directory = "#{@mirror_location}/#{@target_distribution}"
    FileUtils.mkdir @mirror_directory unless opts[:update]

"package_config/ missing in #{@fai_config_pace}" unless Dir.exist? "#{@fai_config_space}/package_config"

# Defaults to 1.
    ENV["MAXPACKAGES"] = @maxpackages.to_s

set_classes

# TODO: implementing this is not easy, since it requires some conversion from bash to ruby.
    # Probably those files could be read and parsed manually, but not doing that now.
=begin
    if opts[:set_variables]
      variable_files = mk_class_var_list
      set -a
      for f in $flist; do
        echo Sourcing variable definitions in $f
        source $flist
      end
      set +a
    end
=end

@installer._post_initialize @mirror_directory
  end
end07070100000023000081A4000001C3000001CA000000015F6847FC00001391000000000000000000000000000000000000004000000000fai-toolkit-1600669692.02e37ec/lib/create-fai-mirror/methods.rb
require "fileutils"

require_relative "create-fai-mirror"

module CreateFaiMirror::Methods

# removes/excludes all classes in $* from @classes
  def exclude_classes excluded_classes
    @classes.delete_if do |c|
      excluded_classes.include? c
    end
    ENV["classes"] = @classes.join("\n")
  end

def umount_dirs
    return if $FAI_DEBMIRROR.nil? or $FAI_DEBMIRROR.empty?
    execute "umount #{@mirror_location}", fail: false, log: false
  end

def cleandirs
    return # currently nothing to do
  end

# now delete all packages that are already included in base.tar.xz
  def delete_base_packages
    base_packages_file = "#{nfsroot}/var/tmp/base-pkgs.lis"
    unless File.exist? base_packages_file
        debug "#{base_packages_file} not available."
        verb "Can't remove wasteful packages that are already in base.tar.xz."
        return
    end

verb "Removing packages that are already included in base.tar.xz"

File.readlines(base_packages_file).each do |p|
        all = p.gsub(":", " ").split(" ")
        p = all[0]
        arch = all[1]

@installer.try_base_package_delete p, arch
    end
  end

# add packages from base.tar.xz and additional packages in nfsroot
  def add_base_packages

# arch dependent packages defined in fai-make-nfsroot
    verb "Adding packages of #{@fai_config_dir}/NFSROOT."
    if File.exist? "#{nfsroot}/var/tmp/packages.nfsroot"
        plist = File.readlines("#{nfsroot}/var/tmp/packages.nfsroot")[0].split(" ")
        @installer.install_packages plist
    else
        puts.STDERR "WARNING: #{nfsroot}/var/tmp/packages.nfsroot does not exist."
        puts.STDERR "Can't add those packages. Maybe the nfsroot is not yet created."
        exit 1
    end

@installer.use_temporary_sources_list

base_packages_file = "#{nfsroot}/var/tmp/base-pkgs.lis"
    verb "Adding packages from #{base_packages_file}"
    if File.exist? base_packages_file
        plist = File.readlines base_packages_file
        plist.map! do |package|
          package.strip
        end
        @installer.refresh_repositories
        @installer.install_packages plist
    else
        puts.STDERR "WARNING: #{base_packages_file} does not exist."
        puts.STDERR "Can't add those packages. Maybe the nfsroot is not yet created."
        exit 1
    end

@installer.restore_sources_list
  end

def set_classes

# if -c is given, ignore -x
    unless $cclasses.nil? or $cclasses.empty?
      @classes = $cclasses
      return
    end

# all available file names are classes
    classes = execvalue %{cd #{@fai_config_space}/package_config; ls -1 | grep -E -i "^[a-zA-Z0-9_-]+$"}
    classes = classes.split "\n"
    addclasses = execvalue %{grep -h PACKAGES #{@fai_config_space}/package_config/* | sed -e 's/#.*//' | awk '{printf $3"\\n"$4"\\n"$5"\\n"$6"\\n"}'}, fail: true
    addclasses = addclasses.split "\n"
    #@classes = execvalue %{echo -e "#{classes}\n#{addclasses}\n" | sort | uniq}
    @classes = (classes + addclasses).sort.uniq
    @classes.delete_if do |c|
      c.empty?
    end
    ENV["classes"] = @classes.join "\n"

exclude_classes @excluded_classes if !@excluded_classes.nil? and !@excluded_classes.empty?
  end

# create a list of files named class/*.var that contain a definition of a  variable that is
  # used in package_config/<classes>

# we assume, that variables used in package_config are defined in class/*.var.
  # For all classes search if there's a variables used in package_config/CLASS
  # For all variables found look which files class/CLASS.var define these variables.
  def mk_class_var_list

filelist = ""

#filelist: all files in package_config which are also a defined class
    @classes.each do |c|
      if File.exist? "#{@fai_config_space}/package_config/#{c}"
        filelist+=" $c"
      end
    end

return if filelist.empty?

# make the list of variables that are used
    # regex: $abc or ${abc}
    variables = execvalue(%{($(cd #{@fai_config_space}/package_config;grep -h -P -o '\$[A-Za-z0-9_]+|\$\{[A-Za-z0-9_]+\}' #{$filelist}|sort|uniq))}).split("\n")

# debug
    #variables.each do |variable|
    #  debug "Variable found: #{variable}"
    #end

# create a regex like varname= for all variables

variables.map! do |variable|
      variable.gsub! /[{}$]/, ""
      variable += "="
    end

classlist = []

# classlist: make list of files in class, which are also a class
    @classes.each do |c|
      if File.exist? "#{@fai_config_space}/class/#{c}.var"
        classlist << "#{@fai_config_space}/class/#{$c}.var"
      end
    end
    # look for variable definition in this list

sourcefiles = []

classlist.each do |f|
      variables.each do |variable|
        execute %[grep -q -P "#{variable}" #{f}], log: false, fail: false
        if $?.success
          sourcefiles << f
          break
        end
      end
    end

# these files contain those variable definitions which are needed
    sourcefiles
  end
end07070100000024000081A4000001C3000001CA000000015F6847FC00000509000000000000000000000000000000000000003D00000000fai-toolkit-1600669692.02e37ec/lib/create-fai-mirror/suse.rbrequire "fileutils"

require_relative "create-fai-mirror"
require_relative "../common"

class CreateFaiMirror::Suse
  include Common

def initialize opts
    @opts = opts
  end

# Installs packages to the mirror.
  def install_packages package_list
    execute %[zypper #{@zypper_options} download #{package_list.join(" ")}], fail: false
    if $?.exitstatus == 106
      puts "WARNING: a repository sync failed. It may mean that this command needs to be run as root."
    end
  end

def refresh_repositories
    #TODO
  end

def _post_initialize mirror_directory
    #TODO

raise "For some reason mirror_directory is invalid..." if mirror_directory.empty?

@zypper_options = <<~EOF
    --non-interactive
    --cache-dir=#{mirror_directory}
    EOF
    @zypper_options.gsub! "\n", " "
  end

def startup_checks
    #TODO
  end

# Hooks to make it visually easier to understand the flow.

def _pre_real_perform fai_config_dir, mirror_location
    #TODO
  end

def _post_real_perform mirror_directory
    #TODO
  end

# Quasi-hooks

def download_packages_for_classes classes, fai_config_space
    #TODO
  end

def use_temporary_sources_list
    #TODO
  end

def restore_sources_list
    #TODO
  end

def try_base_package_delete p, arch
    #TODO
  end

end
07070100000025000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000003400000000fai-toolkit-1600669692.02e37ec/lib/create-update-cd07070100000026000081A4000001C3000001CA000000015F6847FC00001D90000000000000000000000000000000000000004800000000fai-toolkit-1600669692.02e37ec/lib/create-update-cd/create-update-cd.rb
require "fileutils"
require "tmpdir"

require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "../fai-config"
require_relative "methods"
require_relative "updatecd"

# TODO: how base files should be supplied?
# Now, they are inside config space’s basefiles dir, but this isn’t really a good way; probably a base file should be explicitly pointed out, so those that are wanted can be bundled, not all of them.

# Main class.
class Updatecd::CreateUpdateCd
  include Common
  include Output
  include Updatecd::Methods

LIVEOS_PATH = "squashfs-root/LiveOS/"

HIDEDIRS = [
    "/usr/share/locale",
    "/usr/share/doc",
    "/var/lib/apt",
    "/var/lib/aptitude",
    "/var/cache/debconf",
    "/var/cache/apt",
    "/var/cache/man",
    "/usr/share/man",
    "/var/lib/dpkg/info",
    "/media/mirror/aptcache"
  ]

def perform
    startup_checks
    real_perform

# Triple-remove to be sure they actually gets removed...
    FileUtils.remove_entry @tempdir if Dir.exist? @tempdir
    FileUtils.remove_entry @scratch if Dir.exist? @scratch
  end

def options
    options = Optimist::options do
      opt :fai_config_dir,    "FAI’s system configuration directory",
          type: :string,    short: "c",   default: "/etc/fai"
      opt :mirror_dir,        "Location of fai-mirror",
          type: :string,    short: "m"
      opt :grub_config,       "Path to grub.cfg for this image",
          type: :string,    short: "g"
      opt :filename,          "Filename to generate image to. For example image.iso",
          type: :string,    short: "n"
      opt :initrd,            "Use a custom initrd instead of one from NFSROOT",
          type: :string,    short: "i"
      opt :target_distribution, "Target distribution. Currently support values are “debian” and “suse”.",
          type: :string,    short: "t"
      opt :media_label,         "Name/label of the CD",
          type: :string,    short: "l"
      opt :target_site,         "Optionally specify a site that can be used to define default values for various settings",
          type: :string,    short: "s"
      opt :target_host,         "Optionally specify target host. Required when --target-site is used.",
          type: :string,    short: "H"
      opt :add_class,           "Add a class to be set in the resulting image.",
          type: :string,    short: "C", multi: true
      opt :install,             "Specified if resulting image should be of install type.",
          type: :boolean,   short: :none
      opt :update,              "Tells that generated CD should only be used for updates.",
          type: :boolean,   short: :none
      opt :force,               "Allow overriding existing image",
          type: :boolean,   short: "f"
      opt :debug,             "Show debug output"
      opt :verbose,           "Show verbose output"
    end

force = options[:force]

@install_type = options[:install]
    @update_type = options[:update]

@config = FaiConfig.new @fai_config_dir
    @config.load_nfsroot_conf
    @config.load_fai_conf

@target_site = options[:target_site]
    @target_host = options[:target_host]

if @target_site and @target_host.nil?
      raise "--target-host must also be given if --target-site is used."
    end

if @target_site and @target_host
      verb "Looking for host configuration for site #{@target_site}, host #{@target_host}"
      @config.load_target_config @target_site, @target_host
    end

raise "--mirror-dir is required." if options[:mirror_dir].nil?
    raise "--mirror-dir needs to be a valid directory." unless Dir.exist? options[:mirror_dir]
    @mirrordir = options[:mirror_dir]

if options[:grub_config].nil?
      if @target_site
        if @config.target.grub_config
          @grub_config = @config.target.grub_config
        else
          raise "Please specify grub config file in host configurations or with --grub-config."
        end
      else
        raise "--grub-config is required."
      end
    else
      raise "--grub-config needs to be a valid file." unless File.exist? options[:grub_config]
      @grub_config = options[:grub_config]
    end

filename = options[:filename]
    if filename.nil?
      if @target_site
        raise "Invalid filename" if @config.target.image_name.empty?
        filename = "#{@config.target.image_name}.iso"
        if @update_type
          filename = "update_#{filename}"
        end
      else
        raise "--filename is required."
      end
    end

if File.exist? filename
      unless force
        raise "--filename needs not to exist before. Remove the file if you wish to overwrite current image."
      else
        FileUtils.rm filename
      end
    end
    @image_name = filename

raise "Given --initrd file is not valid." if not options[:initrd].nil? and not File.exist? options[:initrd]
    @initrd = options[:initrd]

@target_distribution = options[:target_distribution]

if @target_distribution.nil? and @target_site
      @target_distribution = @config.target.distribution
    end

case @target_distribution
    when "debian"
      # Platform specific checks can be done here.
    when "suse"
      # Platform specific checks can be done here.
    else
      raise "Invalid target distribution: #{@target_distribution.inspect}"
    end

@cd_label = options[:media_label]
    @cd_label = "FAI_CD" if @cd_label.nil?

if @install_type and @update_type
      raise "Only one of --install and --update can be used. Leave both out to create unspecified image."
    end

@added_classes = options[:add_class]
    if @added_classes.nil? and @target_site
      if @config.target.classes
        @added_classes = @config.target.classes
      end
    end

set_opts options
  end

private

def hidedirs
    HIDEDIRS
  end

def liveos_path
    LIVEOS_PATH
  end

def real_perform
    create_ext3fs_image
    create_squashfs_image

@kernelversion = execvalue("ls -tr #{@config.nfsroot}/boot/vmlinu?-* | tail -1 | sed -e 's#.*/vmlinuz-##'")
    faiversion = execvalue("dpkg --root=#{@config.nfsroot} -l fai-client|awk '/fai-client/ {print $3}'")
    date_str = execvalue("date '+%Y %b %d - %R'")
    @isoversion = sprintf "FAI %-6s build #{date_str}", "#{faiversion}"

@aname = "Fully Automatic Installation, #{@isoversion}"

create_grub2_image

#provide_memtest_boot_option

add_extra_files

mkiso @cd_label
  end

def startup_checks
    $logfile = File.open("/var/log/fai/create-update-cd.log", "w")
    at_exit do
      $logfile.close
    end

check_programs

#[ $hidevartmp -eq 1 ] && hidedirs+="/var/tmp"

@tempdir = Dir.mktmpdir "create-update-cd-"
    debug "Got tmp: #{@tempdir}"
    @scratch = Dir.mktmpdir "scratch-"

at_exit do
      cleanup_liveos_mounts

# Double-remove to be sure they actually gets removed.
      FileUtils.remove_entry @tempdir if Dir.exist? @tempdir
      FileUtils.remove_entry @scratch if Dir.exist? @scratch
    end

# Don’t show an exception when pressing ctrl+c to interrupt execution.
    trap "SIGINT" do
      #startcolor = "\x1b[38;2;255;255;255m"
      startcolor = color(255, 255, 255, bold: true)
      puts "#{startcolor}Aborting FAI install media creation...#{stopcolor}"
      exit 100
    end
  end
end
07070100000027000081A4000001C3000001CA000000015F6847FC00002618000000000000000000000000000000000000003F00000000fai-toolkit-1600669692.02e37ec/lib/create-update-cd/methods.rb
require_relative "updatecd"
require_relative "../common"

module Updatecd::Methods
  include Common

def check_programs
    msg = ""
    msg+="xorriso not found. Please install package." unless command_available? "xorriso"
    msg+="\nmksquashfs not found. Please install the package squashfs-tools." unless command_available? "mksquashfs"
    msg+="\nmkfs.vfat not found. Please install the package dosfstools." unless command_available? "mkfs.vfat"
    msg+="\nmcopy not found. Please install the package mtools." unless command_available? "mcopy"

unless msg.empty?
      puts msg
      exit 1
    end
  end

def error_space status, message
    STDERR.puts "Not enough space left on image for the #{message}"
    STDERR.puts execvalue("df -h #{@tempdir}/#{liveos_path}/mounted-ext3fs")
    exit status.exitstatus
  end

def customize_nfsroot
    FileUtils.mkdir_p "#{@tempdir}/empty"

hidedirs.each do |dir|
      if Dir.exist? "#{@config.nfsroot}/#{dir}"
        debug "hiding #{dir}"
        execute("mount --bind #{@tempdir}/empty #{@config.nfsroot}/#{dir}")
      end
    end
  end

def unhide_dirs
    hidedirs.each do |dir|
      if Dir.exist? "#{@config.nfsroot}/#{dir}"
        debug "disclosing #{dir}"
        execute("umount #{@config.nfsroot}/#{dir} 2>/dev/null", fail: false, log: false)
      end
    end
  end

# calculates the amount of space needed for the ext3fsimage
  def calculate_required_size
    if @target_distribution == "debian"
      size = execvalue("du -scL #{@config.nfsroot} #{@config.config_space} #{@mirrordir}/{dists,pool} 2> /dev/null | tail -n1 | awk '{print $1}'").to_i
    elsif @target_distribution == "suse"
      size = execvalue("du -scL #{@config.nfsroot} #{@config.config_space} #{@mirrordir} 2> /dev/null | tail -n1 | awk '{print $1}'").to_i
    end

# not dividing by 1024 to get the exact, this allows us to get some additional space
    size / 850
  end

def addmirror
    Dir.mkdir "#{@tempdir}/#{liveos_path}/mounted-ext3fs/media/mirror"

verb "Copying the mirror to CD image"
    if @target_distribution == "debian"
      execute("cp -Pr #{@mirrordir}/{dists,pool} #{@tempdir}/#{liveos_path}/mounted-ext3fs/media/mirror")
    elsif @target_distribution == "suse"
      execute("cp -Pr #{@mirrordir} #{@tempdir}/#{liveos_path}/mounted-ext3fs/media/mirror")
    end

error_space $?, "mirror" unless $?.success?

if @target_distribution == "debian"
      # create the sources.list for the CD
      execute <<~MULTILINE
      cat > #{@tempdir}/#{liveos_path}/mounted-ext3fs/etc/apt/sources.list <<EOF
      # mirror location for fai CD, file generated by fai-cd
      EOF
      MULTILINE

dists = execvalue("find #{@mirrordir} -name \"Packages*\" | grep binary | sed 's/binary-.*//' | sed \"s##{@mirrordir}/*dists/##\" | xargs -r -n 1 dirname | sort | uniq")
      raise "No suitable Packages file found in mirror." if dists.empty?

dists = dists.split " "
      dists.each do |dist|
        comp = execvalue("find #{@mirrordir}/dists/#{dist} -maxdepth 2 -type d -name \"binary-*\" | sed -e \"s##{@mirrordir}/*dists/#{dist}/##\" -e 's#/binary-.*##' | uniq | tr '\n' \" \"")
        execute("echo \"deb [trusted=yes] file:/media/mirror #{dist} #{comp}\" >> #{@tempdir}/squashfs-root/LiveOS/mounted-ext3fs/etc/apt/sources.list")
      end
    elsif @target_distribution == "suse"
      # TODO: generate a /etc/zypp/repos.d/fai_cd
    end
  end

def create_ext3fs_image

# create the folder that will be used for the squashfs.img later
    # along with it, a temporary folder is created to mount the ext3fs.img
    FileUtils.mkdir_p "#{@tempdir}/#{liveos_path}/mounted-ext3fs"

# hide some directories before copying the nfsroot
    customize_nfsroot
    count = calculate_required_size

# creates an empty file
    verb "Creating empty ext3fs.img"
    execute("dd if=/dev/zero of=#{@tempdir}/#{liveos_path}/ext3fs.img bs=1MB count=#{count}")
    raise "dd failed. Maybe no enough free space in #{@tempdir}" unless $?.success?

execute("mkfs.ext4 -m1 -q -O^has_journal -F -L FAI-NFSROOT #{@tempdir}/#{liveos_path}/ext3fs.img")

# the resulting ext3fs.img is mounted
    execute("mount -o loop,noatime #{@tempdir}/#{liveos_path}/ext3fs.img #{@tempdir}/#{liveos_path}/mounted-ext3fs")

verb "Copying the nfsroot to CD image"
    # copy the contents of the nfsroot into it
    execute("cp -Ppr #{@config.nfsroot}/. #{@tempdir}/#{liveos_path}/mounted-ext3fs/")
    error_space $?, "nfsroot" unless $?.success?
    FileUtils.rm_f "#{@tempdir}/#{liveos_path}/mounted-ext3fs/etc/resolv.conf"

# execute("find #{@tempdir}/liveos_path/mounted-ext3fs") # for debugging
    # copy config space into nfsroot-copy unless -d is given
    verb "Copying the config space to CD image"
    execute("cp -Ppr #{@config.config_space}/* #{@tempdir}/#{liveos_path}/mounted-ext3fs/var/lib/fai/config/")
    error_space $?, "config directory" unless $?.success?
    # add the partial mirror to the CD
    addmirror

unhide_dirs
    execute("umount #{@tempdir}/#{liveos_path}/mounted-ext3fs")
    Dir.rmdir "#{@tempdir}/#{liveos_path}/mounted-ext3fs"
  end

def create_squashfs_image

# this is where the squashfs.img needs to be placed
    FileUtils.mkdir_p "#{@tempdir}/LiveOS"

# create the squashfs.img
    output_path = "#{@tempdir}/LiveOS/squashfs.img"

sqopt = "-comp xz"

execute("mksquashfs #{@tempdir}/squashfs-root #{output_path} #{sqopt} -progress", direct_output: true)

File.delete "#{@tempdir}/#{liveos_path}/ext3fs.img"
    Dir.rmdir "#{@tempdir}/#{liveos_path}"
    Dir.rmdir "#{@tempdir}/squashfs-root"
  end

def create_grub2_image
    FileUtils.mkdir_p "#{@tempdir}/boot"

unless File.exist? "#{@config.nfsroot}/usr/lib/grub/x86_64-efi"
      raise "No grub-efi-amd64-bin installation found in NFSROOT. Aborting."
    end

FileUtils.cp @grub_config, "#{@config.nfsroot}/tmp/grub.cfg"
    # insert date into grub menu
    execute("perl -pi -e \"s/_VERSIONSTRING_/   #{@isoversion}     /\" #{@config.nfsroot}/tmp/grub.cfg")

execute(%{chroot #{@config.nfsroot} grub-mkstandalone \
        --format=x86_64-efi \
        --output=/tmp/bootx64.efi \
        --locales="" --fonts="" \
        "boot/grub/grub.cfg=/tmp/grub.cfg"})
    FileUtils.move "#{@config.nfsroot}/tmp/bootx64.efi", @scratch

verb "Creating efiboot.img"
    execute("dd if=/dev/zero of=#{@scratch}/efiboot.img bs=1M count=3")
    execute("mkfs.vfat #{@scratch}/efiboot.img", log: false)
    execute("mmd -i #{@scratch}/efiboot.img efi efi/boot")
    execute("mcopy -i #{@scratch}/efiboot.img #{@scratch}/bootx64.efi ::efi/boot/")

unless File.exist? "#{@config.nfsroot}/usr/lib/grub/i386-pc"
      raise "No grub-pc installation found in NFSROOT. Aborting."
    end

execute(%{chroot #{@config.nfsroot} grub-mkstandalone \
        --format=i386-pc \
        --output=/tmp/core.img \
        --locales="" --fonts="" \
        --install-modules="linux normal iso9660 biosdisk memdisk search ls echo test chain msdospart part_msdos part_gpt minicmd ext2 keystatus all_video font sleep gfxterm" \
        --modules="linux normal iso9660 biosdisk search" \
        "boot/grub/grub.cfg=/tmp/grub.cfg"})

execute("cat #{@config.nfsroot}/usr/lib/grub/i386-pc/cdboot.img #{@config.nfsroot}/tmp/core.img > #{@scratch}/bios.img")
    FileUtils.rm "#{@config.nfsroot}/tmp/core.img"

execute("cp -p #{@config.nfsroot}/boot/vmlinuz-#{@kernelversion} #{@tempdir}/boot/vmlinuz")

# If no specific initrd has been given, default to nfsroot’s initrd.
    @initrd ||= "#{@config.nfsroot}/boot/initrd.img-#{@kernelversion}"
    execute("cp -p #{@initrd} #{@tempdir}/boot/initrd.img")

execute("cp -p #{@config.nfsroot}/boot/config-#{@kernelversion} #{@tempdir}/boot/")

filelabel = "FAI-CD"
    filelabel = "FAI-INSTALL-CD" if @install_type
    filelabel = "FAI-UPDATE-CD" if @update_type
    execute("touch #{@tempdir}/#{filelabel}")
  end

# Adds extra files that are not normally present.
  def add_extra_files
    # If some new classes should be added to the host, there is limited ways to perform it.
    # This method works with some initrd trickery, which adds these extra classes as kernel
    # cmdline parameters.
    if @added_classes
      File.open "#{@tempdir}/new_classes", "w" do |f|
        @added_classes.each do |c|
          f.puts c
        end
      end
    end
  end

def mkiso cd_label
    # NOTE: I currently have too old xorriso to have this argument.
    # -iso_mbr_part_type 00 \
    verb "Writing FAI CD-ROM image to #{@image_name}. This may need some time."
    execute(%{xorriso -report_about HINT -as mkisofs -iso-level 3 \
        -full-iso9660-filenames \
        -volid "#{cd_label}" -appid "#{@aname}" \
        -eltorito-boot boot/grub/bios.img  \
        -no-emul-boot -boot-load-size 4 -boot-info-table \
        --eltorito-catalog boot/grub/boot.cat \
        --grub2-boot-info \
        --grub2-mbr #{@config.nfsroot}/usr/lib/grub/i386-pc/boot_hybrid.img \
        -eltorito-alt-boot -e EFI/efiboot.img -no-emul-boot \
        -append_partition 2 0xef #{@scratch}/efiboot.img \
        -o #{@image_name} -graft-points \
        --sort-weight 0 / --sort-weight 1 /boot \
        "#{@tempdir}" \
        /boot/grub/bios.img=#{@scratch}/bios.img \
        /EFI/efiboot.img=#{@scratch}/efiboot.img}, log: false)

puts "ISO image size and filename: #{execute "du -h #{@image_name}"}"

unhide_dirs
  end

def cleanup_liveos_mounts
    execute("mountpoint -q #{@tempdir}/#{liveos_path}/mounted-ext3fs && umount #{@tempdir}/#{liveos_path}/mounted-ext3fs", fail: false, log: false)
    unhide_dirs
    FileUtils.rm_rf [ @tempdir, @scratch ], secure: true
  end

end
07070100000028000081A4000001C3000001CA000000015F6847FC00000015000000000000000000000000000000000000004000000000fai-toolkit-1600669692.02e37ec/lib/create-update-cd/updatecd.rb
module Updatecd
end
07070100000029000081A4000001C3000001CA000000015F6847FC000013D6000000000000000000000000000000000000003100000000fai-toolkit-1600669692.02e37ec/lib/fai-config.rb
require "psych"

# Convenience class to load FAI config files.
class FaiConfig
  attr_reader :nfsroot
  attr_reader :config_space
  attr_reader :nfsroot_etc_hosts

# This is just for mimicking how official scripts works.
  attr_reader :fai_rootpw, :fai_debootstrap_opts

# From fai.conf
  attr_reader :fai_config_src, :loguser, :fai_logproto, :fai_debmirror, :fai_monitor_port
  attr_reader :mirror_mountpoint, :nfsroot_config_space

# Host-specific configuration directory, containing for example IP addresses of the hosts
  # and other miscellaneous information that does not fit under FAI config space.
  attr_reader :host_configurations

# Site specific target host’s configuration.
  #
  # Contains a FaiConfig::Target struct.
  attr_reader :target

Target = Struct.new(:image_name,
                      :grub_config,
                      :classes,
                      :distribution,
                      :distribution_version)

def initialize fai_config_dir
    @fai_config_dir = fai_config_dir

@mirror_mountpoint = "/media/mirror"
    @nfsroot_config_space = "/var/lib/fai/config"
  end

def load_nfsroot_conf
    parse_config_file "#{@fai_config_dir}/nfsroot.conf" do |variable, value|
      #puts variable.inspect
      #puts value.inspect

case variable
      when "NFSROOT"
        @nfsroot = value
      when "FAI_CONFIGDIR"
        # Upstream FAI project is using this strange CONFIGDIR name for config space in many variables...
        @config_space = value
      when "FAI_ROOTPW"
        # This is just for mimicking how official scripts works.
        @fai_rootpw = value
      when "FAI_DEBOOTSTRAP_OPTS"
        # This is just for mimicking how official scripts works.
        @fai_debootstrap_opts = value
      when "NFSROOT_ETC_HOSTS"
        @nfsroot_etc_hosts = value
      end
    end
  end

# NOTE: This seems to be rarely used, but maybe I’ve made mistake with that and this should always be loaded in environment for FAI.
  def load_fai_conf

parse_config_file "#{@fai_config_dir}/fai.conf" do |variable, value|

# There is many variables that could be used (see man fai.conf):
      # - FAI_CONFIG_SRC
      # - LOGUSER
      # - FAI_LOGPROTO
      # - FAI_DEBMIRROR
      # - FAI_MONITOR_PORT

# There is also some undocumented variables:
      # - FAI
      # - MNTPOINT

case variable
      when "FAI_CONFIG_SRC"
        @fai_config_src = value
      when "LOGUSER"
        @loguser = value
      when "FAI_LOGPROTO"
        @fai_logproto = value
      when "FAI_DEBMIRROR"
        @fai_debmirror = value
      when "FAI_MONITOR_PORT"
        @fai_monitor_port = value
      when "MNTPOINT"
        @mirror_mountpoint = value
      when "FAI"
        @nfsroot_config_space = value
      when "HOST_CONFIGURATIONS"
        @host_configurations = value
      else
        puts "WARNING: Unrecognized variable in fai.conf: #{variable}"
      end
    end
  end

def load_target_config target_site, target_host
    if @host_configurations.nil?
      raise "HOST_CONFIGURATIONS variable needs to be set in fai.conf in order to load site specific configuration."
    end

site_path = "#{@host_configurations}/sites/#{target_site}"
    unless Dir.exist? site_path
      raise "Site directory #{site_path} does not exist. Please fix your host configurations directory."
    end

targets_file = "#{site_path}/targets.yaml"

unless File.exist? targets_file
      raise "Targets file #{targets_file} is missing. Please fix your host configurations directory."
    end

targets = Psych.safe_load(File.open(targets_file))

target = nil
    targets.each do |host, configuration|
      if host == target_host
        target = configuration
        break
      end
    end

grub_config = target["grub_config"]
    unless grub_config.nil?
      grub_config = "#{@host_configurations}/grub/#{target["grub_config"]}"
      unless File.exist? grub_config
        raise "GRUB config file specified in #{targets_file} for host #{target_host} does not exist."
      end
    end

@target = Target.new
    @target.image_name =              target["image_name"]
    @target.grub_config =             grub_config
    @target.classes =                 target["classes"]
    @target.distribution =            target["distribution"]
    @target.distribution_version =    target["distribution_version"]
  end

private

def parse_config_file configfile
    variableregex = /^(\w+)=(.*)$/
    valueregex = /^([^"^'.].*)(#.*[^"^'])$/

File.readlines(configfile).each do |line|
      line = line.strip
      matches = line.match variableregex

next unless matches

variable = matches[1]

valuedata = matches[2].strip

# Remove potential comments
      if valuedata.match? valueregex
        valuedata.sub! valueregex, '\1'
      end

value = valuedata.strip

# Strip quotes
      value.delete_suffix!('"')
      value.delete_prefix!('"')
      value.delete_suffix!("'")
      value.delete_prefix!("'")

yield variable, value
    end
  end
end0707010000002A000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000002900000000fai-toolkit-1600669692.02e37ec/lib/media0707010000002B000081A4000001C3000001CA000000015F6847FC00000012000000000000000000000000000000000000003200000000fai-toolkit-1600669692.02e37ec/lib/media/media.rb
module Media
end
0707010000002C000081A4000001C3000001CA000000015F6847FC00000649000000000000000000000000000000000000003200000000fai-toolkit-1600669692.02e37ec/lib/media/mount.rb
require "json"

require_relative "media"
require_relative "../common"

class Media::Mount
  include Common

MOUNT_POINT = "/mnt/cdrom"
  SQUASH_MOUNT_POINT = "/mnt/squash"
  FS_MOUNT_POINT = "/mnt/filesystem"

# Mounts a FAI-style CD so that it can be used for updating.
  def mount_cd
    cd_devices = find_fai_cd

Dir.mkdir MOUNT_POINT unless Dir.exist? MOUNT_POINT
    Dir.mkdir SQUASH_MOUNT_POINT unless Dir.exist? SQUASH_MOUNT_POINT
    Dir.mkdir FS_MOUNT_POINT unless Dir.exist? FS_MOUNT_POINT

update_cd_found = false
    cd_devices.each do |cd_device|
      dev_path = "/dev/#{cd_device["name"]}"
      execute "mount #{dev_path} #{MOUNT_POINT}"
      if File.exist? "#{MOUNT_POINT}/LiveOS/squashfs.img"
        update_cd_found = true
        break
      end
    end

raise "FATAL: None of attached rom devices were an update cd. Please check the media you have inserted." unless update_cd_found

execute %{mount "#{MOUNT_POINT}/LiveOS/squashfs.img" "#{SQUASH_MOUNT_POINT}"}
    execute %{mount "#{SQUASH_MOUNT_POINT}/LiveOS/ext3fs.img" "#{FS_MOUNT_POINT}"}
    execute %{mount --bind "#{FS_MOUNT_POINT}/media/mirror" /media/mirror}
  end

private

def find_fai_cd
    json = `lsblk -a -J -o name,size,rm,ro,mountpoint,fstype,type,label`
    data = JSON.parse json

cd_devices = []

data["blockdevices"].each do |device|
      if device["type"] == "rom"
        cd_devices << device
      elsif device["label"] == "FAI_CD"
        cd_devices << device
      end
    end

raise "FATAL: No suitable ROM devices found." if cd_devices.empty?

cd_devices
  end
end
0707010000002D000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000002B00000000fai-toolkit-1600669692.02e37ec/lib/nfsroot0707010000002E000081A4000001C3000001CA000000015F6847FC00001181000000000000000000000000000000000000003D00000000fai-toolkit-1600669692.02e37ec/lib/nfsroot/create-nfsroot.rb
require "fileutils"
require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "../fai-config"
require_relative "methods"

# Main class.
class Nfsroot::CreateNfsroot
  include Common
  include Output
  include Nfsroot::Methods

def perform
    startup_checks

at_exit do
      $do_log = false
      $logfile.close
      umount_dirs
    end

if opts[:refresh]
      return perform_refresh
    end

perform_fresh_install
  end

def options
    options = Optimist::options do
      opt :fai_config_dir,    "FAI’s main config files.",
          type: :string,      default: "/etc/fai",    short: "c"
      opt :all_packages,      "Installs full set of packages to nfsroot.",
                              default: true,          short: "a"
      opt :refresh,           "Installs new packages from NFSROOT and updates all packages.",
                                                      short: "r"
      opt :force,             "Removes existing nfsroot before installing a new one.",
                              default: false,         short: "f"
      opt :base_archive,      "Use existing base archive instead of creating a new one.",
                                                      short: "b"
      opt :nonfree,           "Also install packages with NONFREE class in NFSROOT file.",
                              default: true,          short: "N"
      opt :run_scripts,       "Runs scripts defined in FAI config space’s (determined from nfsroot.conf) nfsroot/scripts directory. Defaults to true.",
                              default: true,          short: :none
      opt :debug,             "Show debug output",
                                                      short: "d"
      opt :verbose,           "Show verbose output",
                                                      short: "v"
    end

raise "Given --fai-config-dir is not valid." if not Dir.exist? options[:fai_config_dir]
    @fai_config_dir = options[:fai_config_dir]

@nonfree = options[:nonfree] ? "NONFREE" : ""

@all_packages = options[:all_packages]

@run_scripts = options[:run_scripts]

$logfile = File.open("/var/log/fai/fai-make-nfsroot.log", "w")
    at_exit do
      $logfile.close
    end

set_opts options

# TODO: wtf...
    $do_log = false
  end

private

def startup_checks
    verb "Using configuration files from #{@fai_config_dir}"

@config = FaiConfig.new @fai_config_dir
    @config.load_fai_conf
    ENV["FAI"] = @config.nfsroot_config_space

@config.load_nfsroot_conf
    set_nfsroot @config.nfsroot

# TODO: implement this
    #$NFSROOT_HOOKS = NFSROOT_HOOKS

raise "NFSROOT needs to be defined in nfsroot.conf" if @config.nfsroot.empty?

raise "#{@fai_config_dir}/apt does not exist." unless Dir.exist? "#{@fai_config_dir}/apt"

@deldir = @config.nfsroot.dup

# Required so that apt-get won’t try to run debconf.
    ENV["DEBIAN_FRONTEND"] = "noninteractive"

# Ensure scripts won’t be run if they don’t exist.
    if @run_scripts and !Dir.exist?("#{@config.config_space}/nfsroot/scripts")
      debug "nfsroot scripts does not exist, so not running them."
      @run_scripts = false
    end
  end

# Updates existing nfsroot.
  def perform_refresh
    verb "Upgrading nfsroot and installing new packages into the nfsroot."

execute "#{rootcmd} apt-get update"
    execute "#{rootcmd} apt-get -y dist-upgrade"
    add_packages_nfsroot

# Not implemented feature.
    #setup_tftp

run_scripts

exit
  end

# Installs a completely new nfsroot.
  def perform_fresh_install

initialize_logging
    begin
      initialize_nfsroot_dir
    rescue ExecutionFailed
      # There is some bug that requires a cycle. Let’s just emulate that to see if it helps.
      umount_dirs
      initialize_nfsroot_dir
    end

# These are from Methods.
    resolve_sources
    create_nfsroot
    #setup_tftp
    setup_ssh
    run_scripts
    check_nfsroot
    write_variables

=begin
    if [ $generic = 1 ]; then
        # remove fai.conf which comes from package fai-client
        rm -f $NFSROOT/etc/hosts $NFSROOT/etc/resolv.conf* $NFSROOT/etc/fai/fai.conf $NFSROOT/var/lib/apt/lists/*_*
        ainsl -as $NFSROOT/etc/hosts "127.0.0.1 localhost"
    fi
=end

umount_dirs

Process.waitall

verb "fai-make-nfsroot finished."
    opts[:verbose] and puts "Log file written to /var/log/fai/fai-make-nfsroot.log"
  end
end0707010000002F000081A4000001C3000001CA000000015F6847FC00004346000000000000000000000000000000000000003600000000fai-toolkit-1600669692.02e37ec/lib/nfsroot/methods.rb
require_relative "nfsroot"
require_relative "../common"
require_relative "../base-image/methods"

module Nfsroot::Methods
  include Common
  include BaseImage::Methods

def initialize_nfsroot_dir
    if Dir.exist? @config.nfsroot and !opts[:force]
      STDERR.puts "-----------------------------------------------------------------"
      STDERR.puts "Error: #{@config.nfsroot} exists already."
      STDERR.puts "Neither force nor update option present, exiting."
      STDERR.puts "-----------------------------------------------------------------"
      STDERR.puts "NOTE: Use --force to force overwriting an existing nfsroot."
      STDERR.puts "      Use --refresh to update/install packages defined in NFSROOT config."
      exit 1
    end

verb "Creating FAI nfsroot in #{@config.nfsroot}"

if Dir.exist? "#{@config.nfsroot}/proc"
      # Since the umount seems to be unfinished in some cases, let’s do it here too...
      umount_dirs

execute "umount #{@config.nfsroot}/dev/pts >&/dev/null", fail: false
      execute "umount #{@config.nfsroot}/proc", fail: false if File.symlink? "#{@config.nfsroot}/proc/self"
      raise "/proc is still mounted inside the nfsroot." if File.symlink? "#{@config.nfsroot}/proc/self"
      execute "umount #{@config.nfsroot}/#{@config.mirror_mountpoint}  2>/dev/null", fail: false # it's safer to try to umount
      # remove old nfsroot using a background process
      execute "rm -rf #{@deldir}/../.will-now-be-deleted", fail: false
      execute "mv #{@deldir} #{@deldir}/../.will-now-be-deleted", fail: false
      unless $?.success?
        execute "rm -rf #{@deldir}"
      else
        execute "nice rm -rf #{@deldir}/../.will-now-be-deleted", wait: false
      end
    end
  end

# repeat message, so it will appear in the log file (though this seems to be broken thing.)
  def initialize_logging
    FileUtils.rm_f "/var/log/fai/fai-make-nfsroot.log"
    first_line = "Using configuration files from #{@fai_config_dir}"
    verb first_line
    execute "echo #{first_line} > /var/log/fai/fai-make-nfsroot.log"

# Commands to be logged follows
    $do_log = true
  end

def setclasses

# Export all the information we have about the system
    iarch = execvalue "#{rootcmd} dpkg --print-architecture|tr /a-z/ /A-Z/"
    distro = execvalue ". #{@config.nfsroot}/etc/os-release; echo ${ID} ${ID}_${VERSION_ID} | tr '[:lower:].' '[:upper:]_'"

# do we want to install the full list of packages?
    c = "FULL" if @all_packages

classes = "NFSROOT #{c} #{iarch} #{distro} #{@nonfree}"
    ENV["classes"] = classes
    verb "Classes are set to #{classes}"
  end

def add_packages_nfsroot
    ENV["FAI_ROOT"] = @config.nfsroot

setclasses
    execute "#{rootcmd} apt-cache dumpavail > #{@config.nfsroot}/var/lib/dpkg/available", log: false
    execute %{env LC_ALL="C" install_packages -l -p#{@fai_config_dir} > #{@config.nfsroot}/var/tmp/packages.nfsroot}, log: false

puts "Adding additional packages to #{@config.nfsroot}:"
    puts execvalue "cat #{@config.nfsroot}/var/tmp/packages.nfsroot"
    execute %{env LC_ALL="C" install_packages -v -p#{@fai_config_dir}}

FileUtils.rm "#{@config.nfsroot}/var/lib/dpkg/available"
  end

def add_etc_hosts_entries
    # some default entries
    execute %{ainsl -as #{@config.nfsroot}/etc/hosts "127.0.0.1 localhost"}
    execute %{ainsl     #{@config.nfsroot}/etc/hosts "# Following entries are specific to your environment"}
    execute %{ainsl #{$verbose}  #{@config.nfsroot}/etc/hosts "#{@config.nfsroot_etc_hosts}"}

# add entries for all network devices
    ips = execvalue("ip addr show up| grep -P -o '(?<=inet )[^/]+' | grep -v 127.0.0.1").split("\n")

ips.each do |eth|
      line = execvalue "getent hosts #{eth}", fail: false
      unless line.empty?
        execute %{ainsl #{$verbose} #{@config.nfsroot}/etc/hosts "#{line}"}
      else
        puts "Warning: no hostname for #{eth} found, not adding to /etc/hosts."
      end
    end

if File.exist? "/etc/resolv.conf"
      execute "cp -Lp #{$verbose} /etc/resolv.conf #{@config.nfsroot}/etc/resolv.conf-installserver"

# this is needed during fai-make-nfsroot
      execute "cp -Lp #{$verbose} /etc/resolv.conf #{@config.nfsroot}/etc/resolv.conf"
    end
  end

# Calls dpkg-divert to mark a file not to be overridden when a package updates.
  def fdivert *args
    for item in args
      execute "#{rootcmd} dpkg-divert --quiet --add --rename #{item}"
    end
  end

def upgrade_nfsroot

FileUtils.mkdir_p "#{@config.nfsroot}/dev/pts"
    execute "mount -t proc   /proc  #{@config.nfsroot}/proc"
    execute "mount -t sysfs  /sys   #{@config.nfsroot}/sys"
    execute "mount -t devpts devpts #{@config.nfsroot}/dev/pts"
    execute "/usr/lib/fai/mkramdisk #{@config.nfsroot}/var/lib/dpkg #{@config.nfsroot}/var/cache"

# stop mdadm from calling mkconf
    FileUtils.mkdir "#{@config.nfsroot}/etc/mdadm"
    FileUtils.touch "#{@config.nfsroot}/etc/mdadm/mdadm.conf"

# add any 3rd party repository keys that you may need installed into the NFSROOT
    debug "Copying apt keys"
    if Dir.exist? "#{@fai_config_dir}/apt/keys"
      for f in `find #{@fai_config_dir}/apt/keys -type f -name '*.asc' -o -name '*.gpg'`.split("\n")
        debug "Copying #{f} to #{@config.nfsroot}/etc/apt/trusted.gpg.d"
        FileUtils.cp f, "#{@config.nfsroot}/etc/apt/trusted.gpg.d"
      end
    end

# if we have the keyring on the host, install the key of the fai-project.org package repository
    execute <<MULTILINE
if [ ! -f #{@config.nfsroot}/etc/apt/trusted.gpg.d/fai-project.gpg -a -n "$(apt-key list 2BF8D9FE074BCDE4 2>/dev/null)" ]; then
    HOME=/root apt-key export 2BF8D9FE074BCDE4 > #{@config.nfsroot}/etc/apt/trusted.gpg.d/fai-project.asc
fi
MULTILINE

pkgs = @all_packages ? "nfs-common" : ""

execute "#{rootcmd} apt-get update"

# check if we need more dracut packages (dracut >=049)
    execute "#{rootcmd} apt show dracut-live dracut-squash >/dev/null 2>/dev/null"
    pkgs += " dracut-live dracut-squash" if $?.success?

# disable generating the initrd
    fdivert "/etc/kernel/postinst.d/dracut"
    execute "#{rootcmd} ln -s /bin/true /etc/kernel/postinst.d/dracut"

# fai-nfsroot must be successfully installed before anything else
    # TODO: why fai-nfsroot must be first installed?
    execute "#{rootcmd} apt-get -yf --no-install-recommends install fai-nfsroot dracut-network dracut-config-generic #{pkgs}"
    execute "#{rootcmd} apt-get -y dist-upgrade"
    # patch dracut. Do not fail if network is not available
    fn = execvalue "find #{@config.nfsroot}/usr/lib/dracut/modules.d/ -name net-genrules.sh"
    execute "sed -i -e 's/exit 1/exit 0/' #{fn}"
    fdivert "/usr/sbin/update-grub"
  end

def set_root_pw
    # NOTE: $generic is not implemented for now.
    return if $generic # do nothing here if we generate a generic nfsroot

# set root pw inside the nfsroot
    execute %{#{rootcmd} usermod -p "#{@config.fai_rootpw}" root}
  end

def create_nfsroot
    FileUtils.mkdir_p "#{@config.nfsroot}/#{@config.nfsroot_config_space}"
    Dir.chdir @config.nfsroot

if opts[:base_archive]
      extract_base
    else
      create_base
    end

FileUtils.touch ".THIS_IS_THE_FAI_NFSROOT"
    # save the list of all packages in the base.tar.gz
    execute "#{rootcmd} dpkg --get-selections | awk '/install$/ {print $1}' > var/tmp/base-pkgs.lis"

if !@config.fai_debmirror.nil? and !@config.fai_debmirror.empty?
      verb "Mounting #{@config.fai_debmirror} to #{@config.nfsroot}/#{@config.mirror_mountpoint}."
      FileUtils.mkdir_p "#{@config.nfsroot}/#{@config.mirror_mountpoint}"
      execute "mount -o ro,noatime #{@config.fai_debmirror} #{config.nfsroot}/#{@config.mirror_mountpoint}"
      raise "Can't mount #{@config.fai_debmirror} to #{@config.nfsroot}/#{@config.mirror_mountpoint}." unless $?.success?
    end

# liloconfig, dump needs this file
    FileUtils.touch "etc/fstab"

# We need these option before installing the first package. So we
    # can't put this into fai-nfsroot /etc/apt/apt.conf.d/90fai
    File.open "#{@config.nfsroot}/etc/apt/apt.conf.d/10fai", "w" do |f|
      f.puts <<~EOF
        APT::Install-Recommends no;
        APT::Get::AllowUnauthenticated true;
        Acquire::AllowInsecureRepositories "true";
        Aptitude::CmdLine::Ignore-Trust-Violations yes;
        Acquire::Languages none;
        Acquire::Check-Valid-Until false;
      EOF
    end

# currently we do not want to use systemd, but CD booting, also we
    # do not need all those filesystems inside the initrd
    FileUtils.mkdir "#{@config.nfsroot}/etc/dracut.conf.d"
    File.open "#{@config.nfsroot}/etc/dracut.conf.d/12-no-systemd.conf", "w" do |f|
      f.puts 'omit_dracutmodules+=" dracut-systemd systemd systemd-initrd systemd-networkd systemd-initrd "'
    end

fslist = ""
    File.open "#{@config.nfsroot}/etc/dracut.conf.d/13-add-cd-support.conf", "w" do |f|
      if @all_packages
        f.puts 'add_dracutmodules+=" livenet "'
        fslist = "nfs lockd ext4 xfs"
      else
        f.puts 'add_dracutmodules+=" dmsquash-live "'
        fslist = "ext4 xfs"
      end

# Always bundle iso9660, so reading from CD can work even if booted kernel
      # is different from the kernel inside installed system.
      # This is rarely used, but it only increases size of the initrd by ~80 kB, so
      # there is little reason not to include it.
      f.puts 'add_drivers+=" isofs "'
    end

File.open "#{@config.nfsroot}/etc/dracut.conf.d/20-generic-image.conf", "w" do |f|
      f.puts 'hostonly="no"'
    end

File.open "#{@config.nfsroot}/etc/dracut.conf.d/01-omit.conf", "w" do |f|
      f.puts 'omit_dracutmodules+=" btrfs crypt dash lvm resume usrmount modsign mdraid shutdown dracut-systemd systemd systemd-initrd systemd-networkd systemd-initrd "'
      f.puts %{filesystems="#{fslist}"}
    end

if Dir.exist? "#{@fai_config_dir}/apt/sources.list"
      execute "fcopy -s #{@fai_config_dir}/apt -t #{@config.nfsroot}/etc/apt sources.list"
    else
      # this copies sources.list into the nfsroot
      execute "cp -aL #{@fai_config_dir}/apt #{@config.nfsroot}/etc"
    end

add_etc_hosts_entries

=begin
    # use a proxy if variable is defined
    if [ -n "$APTPROXY" ]; then
        echo "Acquire::http::Proxy \"$APTPROXY\";" > nfsroot/etc/apt/apt.conf.d/02proxy
    fi
=end

verb "Upgrading #{@config.nfsroot}"
    upgrade_nfsroot
    add_packages_nfsroot

# remove file diversion, then create initrd
    FileUtils.rm "#{@config.nfsroot}/etc/kernel/postinst.d/dracut"
    execute "#{rootcmd} dpkg-divert --rename --remove /etc/kernel/postinst.d/dracut"
    execute "#{rootcmd} dpkg-reconfigure dracut"

set_root_pw

# set timezone in nfsroot
    timezonefile = "/etc/timezone"
    if File.exist? timezonefile
      execute "cp -fH #{timezonefile}  etc/timezone"
    else
      File.open timezonefile, "w" do |file|
        timezone = execvalue %{timedatectl status | grep "Time zone" | awk '{ print $3 }'}
        file.write timezone
      end
    end
    execute "cp -a /etc/localtime etc/localtime"
    FileUtils.ln_sf "/proc/mounts", "etc/mtab"

# Set FAI as the init (pid 1) when booting to a live/install CD.
    FileUtils.ln_s "/usr/sbin/fai", "etc/init.d/rcS"

# do not activate RAID arrays
    File.open "etc/mdadm/mdadm.conf", "w" do |f|
      f.puts "AUTO -all"
    end

# This second rm of the same file is needed!
    FileUtils.rm_f "etc/udev/rules.d/70-persistent-net.rules"

# remove Ubuntu-specific rules to auto-start volume groups, which confuses setup-storage
    FileUtils.rm_f "lib/udev/rules.d/85-lvm2.rules"

# definition for loopback device
    File.open "etc/network/interfaces", "w" do |f|
      f.puts "iface lo inet loopback"
    end

File.open "root/.profile", "a" do |f|
      f.puts <<~EOF
        PATH=/usr/local/sbin:/usr/local/bin:/usr/lib/fai:/bin:/sbin:/usr/bin:/usr/sbin:
        export PATH
        . /usr/lib/fai/subroutines
        set -a
        . /tmp/fai/variables.log 2>/dev/null
      EOF
    end
  end

def setup_ssh
    # nothing to do if no ssh is available in nfsroot
    return unless File.exist? "#{@config.nfsroot}/usr/bin/ssh"

# enable root login
    execute "sed -i -e '/PermitRootLogin/d' #{@config.nfsroot}/etc/ssh/sshd_config"
    execute "ainsl #{@config.nfsroot}/etc/ssh/sshd_config 'PermitRootLogin yes'"

=begin
    if [ $sshpreserve -ge 1 ]; then
        tar -C $NFSROOT -xf $tmptar
        rm $tmptar
        return 0
    fi
=end

FileUtils.mkdir_p "#{@config.nfsroot}/root/.ssh", mode: 700
=begin
    if [ -n "$LOGUSER" ] ; then
        loguserhome=$(eval "cd ~$LOGUSER 2>/dev/null && pwd;true")
        [ -f $loguserhome/.ssh/known_hosts ] && cp $loguserhome/.ssh/known_hosts $NFSROOT/root/.ssh/known_hosts
        [ -d $loguserhome/.ssh ] && {
            [ -f $loguserhome/.ssh/id_ed25519 ] &&
               cp -p $loguserhome/.ssh/id_ed25519* $NFSROOT/root/.ssh/
            [ -f $loguserhome/.ssh/id_rsa ] &&
               cp -p $loguserhome/.ssh/id_rsa* $NFSROOT/root/.ssh/
        }
    fi
=end

=begin
    if [ -f "$SSH_IDENTITY" ]; then
        install -m0644 $SSH_IDENTITY $NFSROOT/root/.ssh/authorized_keys
        echo You can log into install clients without password using $SSH_IDENTITY
    fi
=end

# setting StrictHostKeyChecking to no inside the nfsroot would enable
    # easy login to the install server even if the host key is not known

end

# Simple tests to see if important things are available inside the nfsroot.
  def check_nfsroot
    unless File.executable? "#{@config.nfsroot}/usr/bin/dracut"
        raise "dracut was not installed into the nfsroot."
    end

unless File.exist? "#{@config.nfsroot}/sbin/init"
        raise "sysvinit-core was not installed into the nfsroot."
    end

execute "readlink #{@config.nfsroot}/sbin/init | grep -q systemd", fail: false
    if $?.success?
        raise "Do not use systemd inside the nfsroot."
    end

puts "FAI packages and related packages inside the nfsroot:"
    execute %{#{rootcmd} dpkg-query -W -f='${Package;-18} ${Version}\n' fai-client fai-nfsroot fai-setup-storage dracut-network dracut dracut-live dracut-squash 2>/dev/null}

# check if all important packages are installed
    n = execvalue("#{rootcmd} dpkg-query -l fai-client fai-nfsroot fai-setup-storage dracut-network dracut 2>/dev/null|grep -E ^ii |wc -l").to_i
    if n != 5
        puts "ERROR: Some essential packages are missing."
        bad_exit
    end

files = execvalue "ls #{@config.nfsroot}/boot/initrd* 2>/dev/null"
    raise "No initrd installed." if files.empty?
    execute %{grep -E -q "^ERROR: |^E: Sub-process |^dpkg: error processing |^dpkg: dependency problems" /var/log/fai/fai-make-nfsroot.log}, fail: false
    bad_exit if $?.success?

nil
  end

def write_variables
    varfile = "/var/log/fai/variables"
    fai_version = execvalue "#{rootcmd} dpkg-query -W -f='${Version}' fai-client"

# remove old values from file
    execute "sed -i -e '/FAI_VERSION/d' -e '/NFSROOT/d' #{varfile}" if File.exist? varfile and !File.zero? varfile
    execute %{echo "FAI_VERSION=#{fai_version}" >> #{varfile}}
    execute %{echo "NFSROOT=#{@config.nfsroot}" >> #{varfile}}
  end

# Runs scripts inside FAI config space’s nfsroot/scripts dir.
  #
  # WARNING: This does not utilize FAI’s fai-do-scripts nor fully mimics how it works.
  def run_scripts
    unless @run_scripts
      debug "Skipping nfsroot scripts"
      return
    end

env = {
      "ROOTCMD" => rootcmd,
      "NFSROOT" => @config.nfsroot
    }

for i in Dir.glob("#{@config.config_space}/nfsroot/scripts/*")
      name = File.basename i
      suffix = i[-3,3]
      # NOTE: No special features has been done for shell scripts. Maybe
      # they could all be run in one sesion, for example. Or the env vars could be collected?
      if suffix == ".sh"
        verb "Executing shell script #{name}"
        execute i, env: env
        next
      end

if suffix == ".rb"
        verb "Loading Ruby script #{name}"
        load i
        next
      end

verb "Executing custom script #{name}"
      execute i, env: env
    end
  end

private

def extract_base base_archive
    unless File.exist? base_archive
      raise "Base archive #{base_archive} could not be used."
    end

Dir.chdir nfsroot do
      verb "Extracting base archive #{base_archive}: "

case File.basename(base_archive).split(".", 2)[1]
      when "tar.gz", ".tgz"
        execute %{tar --xattrs --acls --xattrs-include=* -C "#{@config.nfsroot}" -zxpf "#{base_archive}"}
        FileUtils.cp base_archive, "#{@config.nfsroot}/var/tmp/base.tar.gz", preserve: true
      when "tar.xz", "txz"
        execute %{tar --xattrs --acls --xattrs-include=* -C "#{@config.nfsroot}" -Jxpf "#{base_archive}"}
        FileUtils.cp base_archive, "#{@config.nfsroot}/var/tmp/base.tar.xz", preserve: true
      end
    end
  end

def bad_exit
    puts "An error occured during fai-make-nfsroot."
    puts "Please fix the error or try fai-make-nfsroot -v"
    exit 1
  end
end07070100000030000081A4000001C3000001CA000000015F6847FC00000014000000000000000000000000000000000000003600000000fai-toolkit-1600669692.02e37ec/lib/nfsroot/nfsroot.rb
module Nfsroot
end
07070100000031000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000003100000000fai-toolkit-1600669692.02e37ec/lib/online-update07070100000032000081A4000001C3000001CA000000015F6847FC0000001A000000000000000000000000000000000000004200000000fai-toolkit-1600669692.02e37ec/lib/online-update/online-update.rb
module OnlineUpdate
end

07070100000033000081A4000001C3000001CA000000015F6847FC00001F4B000000000000000000000000000000000000004400000000fai-toolkit-1600669692.02e37ec/lib/online-update/perform-on-host.rb
require "fileutils"
require "socket"
require "tempfile"
require "optimist"

require_relative "../common"
require_relative "online-update"

# Main class.
class OnlineUpdate::PerformOnHost
  include Common

def perform
    startup_checks

cleanup

@fai_mirror_path = "/var/tmp/fai_mirror"
    @packages_dir_path = "/var/tmp/packages"

at_exit do
      # Comment out in order to debug leftover state of the script.
      #cleanup
    end

# Transfer temporary sources.list to target.

content = generate_sources_list
    tmpfile = Tempfile.new "perform-on-host"
    tmpfile.write content
    tmpfile.flush
    execute %{scp -P #{@ssh_port} #{tmpfile.path} root@#{@target_host}:/var/tmp/update.sources.list}
    # TODO: remove the tmpfile as it’š not needed any more.

# Transfer packages to target.

if @packages_dir
      #archive = "/tmp/packages.tar"
      #path = File.dirname @packages_dir
      #target_dir = File.basename @packages_dir
      #excludes = @include_debug ? '--exclude="*dbgsym*"' : ""
      #execute %{tar #{excludes} -cvf #{archive} -C #{path} #{target_dir}}
      #execute %{scp -P #{@ssh_port} #{archive} root@#{@target_host}:/var/tmp}

#execute %{#{ssh_command} 7z x /var/tmp/packages.7z -o/var/tmp}
      #execute %{#{ssh_command} tar xf /var/tmp/packages.tar -C /var/tmp}

# Ensure there is target dir on target host.
      execute %{#{ssh_command} mkdir -p #{@packages_dir_path}}

# TODO: delete locally removed files from target.
      execute %{rsync -az -e "ssh -p #{@ssh_port}" #{@packages_dir}/* root@#{@target_host}:#{@packages_dir_path}}
    end

if @fai_mirror
      # Ensure there is mirror dir on target host.
      execute %{#{ssh_command} mkdir -p #{@fai_mirror_path}}

if @target_distribution == "debian"
        #execute %{scp -r -P #{@ssh_port} #{@fai_mirror}/{dists,pool} root@#{@target_host}:#{@fai_mirror_path}}
        execute %{rsync -az -e "ssh -p #{@ssh_port}" #{@fai_mirror}/{dists,pool} root@#{@target_host}:#{@fai_mirror_path}}
      elsif @target_distribution == "suse"
        execute %{rsync -az -e "ssh -p #{@ssh_port}" #{@fai_mirror}/* root@#{@target_host}:#{@fai_mirror_path}}
      end
    end

# Transfer config space to target.
    config_space_archive = "config_space.tar.xz"
    config_space_path = File.dirname @config_space
    config_space_target_dir = File.basename @config_space
    execute %{tar --exclude="basefiles/*" --exclude=".git/*" -cf /tmp/#{config_space_archive} -C #{config_space_path} #{config_space_target_dir}}
    execute %{scp -P #{@ssh_port} /tmp/#{config_space_archive} root@#{@target_host}:/var/tmp}

execute %{#{ssh_command} tar xf /var/tmp/#{config_space_archive} -C /var/tmp}

# Renames the file to correct name, to avoid extra confusion.
    execute %{#{ssh_command} mv /var/tmp/#{config_space_target_dir} /var/tmp/fai_config}

# Perform actual update.

class_string = @add_classes.map do |c|
      "--add-class #{c}"
    end.join(" ")

execute %{#{ssh_command} perform-online-update #{class_string} --sources-list /var/tmp/update.sources.list}
  end

def options
    options = Optimist::options do
      opt :packages_dir,          "Path to directory containing packages that will be used for update",       type: :string,    short: "D"
      opt :target_host,           "Host where update will be performed at",                                   type: :string,    short: "H"
      opt :ssh_port,              "Port where target host’s SSHD is listening at",                            type: :integer,   short: "P"
      opt :config_space,          "Path to FAI config space",                                                 type: :string,    short: "c", default: "/srv/fai/config"
      opt :fai_mirror,            "Path to FAI mirror, created by fai-mirror or create-fai-mirror",           type: :string,    short: "m"
      opt :target,                "Target distribution. Currently support values are “debian” and “suse”",    type: :string,    short: "t"
      opt :add_class,             "Add a class to target host. Can be given multiple times",                  type: :string, multi: true, short: "a"
      opt :include_debug,         "Include debug (dbgsym) packages when transferring packages dir to target", type: :boolean,   short: "e"
      opt :debug,                 "Show debug output",                                                                          short: "d"
      opt :verbose,               "Show verbose output",                                                                        short: "v"
    end

if !options[:packages_dir].nil? and !options[:fai_mirror].nil?
      raise "Only one of --fai-mirror and --packages-dir can be used."
    end

unless Dir.exist? options[:config_space]
      raise "Given FAI config space (#{options[:sources_list]}) does not exist."
    end
    @config_space = options[:config_space]
    unless Dir.exist? "#{@config_space}/class"
      raise "Given FAI config space (#{options[:sources_list]}) is malformed."
    end

if options[:packages_dir].nil?
      if options[:fai_mirror].nil? or !Dir.exist? options[:fai_mirror]
        raise "Given FAI mirror dir (#{options[:fai_mirror].inspect}) does not exist."
      end
      @fai_mirror = options[:fai_mirror]
    end

if options[:fai_mirror].nil?
      if options[:packages_dir].nil? or !Dir.exist? options[:packages_dir]
        raise "Given path to packages dir “#{options[:sources_list]}” does not exist."
      end
      @packages_dir = options[:packages_dir]
    end

@target_host = options[:target_host]
    @ssh_port = options[:ssh_port]
    @add_classes = options[:add_class]
    @include_debug = options[:include_debug]

if @target_host.nil? or @ssh_port.nil?
      raise "--target-host and --ssh-port must be given"
    end

begin
      Socket.tcp(@target_host, @ssh_port, connect_timeout: 3) {}
    rescue Errno::ETIMEDOUT => e
      puts.STDERR "Connecting to host #{@target_host}:#{@ssh_port} failed. Please check if there is a connection problem."
      raise e
    end

@target_distribution = options[:target]
    case @target_distribution
    when "debian"
      # Platform specific checks can be done here.
    when "suse"
      puts "WARNING: This path has not been tested. Remove following exit locally if you want to test this for SUSE."
      exit 1
      # Platform specific checks can be done here.
    else
      raise "Invalid target distribution: #{@target_distribution.inspect}"
    end

set_opts options
  end

private

def generate_sources_list
    if @fai_mirror
      # TODO: The mirror should be signed, even though the effect is limited. It still has its used.
      # After it’s been signed, trusted=yes can be removed.
      content = <<~EOF
      deb [trusted=yes] file:#{@fai_mirror_path}/ cskoeln main non-free contrib
      EOF
    elsif @packages_dir
      content = <<~EOF
      deb file:#{@packages_dir_path} ./
      EOF
    end

content
  end

def ssh_command
    %{ssh -p #{@ssh_port} root@#{@target_host}}
  end

def startup_checks
    unless command_available? "ssh"
      raise "You need to have ssh (Secure SHell) available."
    end

unless command_available? "scp"
      raise "You need to have scp (a utility to copy files over ssh) available."
    end

unless command_available? "tar"
      raise "You need to have tar (archiver) to create a package archive."
    end
  end

def cleanup
    execute %{#{ssh_command} rm -rf /var/tmp/fai_config /var/tmp/update.sources.list}, fail: false
    execute %{#{ssh_command} rm -rf /var/tmp/config_space.tar.xz /var/tmp/packages.tar}, fail: false

# This could be used to clean up existing package data, but to save bandwidth, that shouldn’t be done at least on my development environments.
    #execute %{#{ssh_command} rm -rf #{@packages_dir_path}, fail: false
    #execute %{#{ssh_command} rm -rf #{@fai_mirror_path}, fail: false
  end
end
07070100000034000081A4000001C3000001CA000000015F6847FC00000408000000000000000000000000000000000000003C00000000fai-toolkit-1600669692.02e37ec/lib/online-update/perform.rb
require "fileutils"
require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "online-update"

# Main class.
class OnlineUpdate::Perform
  include Common

def perform

sourceslist = "/etc/apt/sources.list"
    FileUtils.mv sourceslist, "#{sourceslist}.BAK"
    FileUtils.cp @sources_list, sourceslist
    at_exit do
      FileUtils.rm sourceslist
      FileUtils.mv "#{sourceslist}.BAK", sourceslist
    end

execute %{fai -v -N -C /mnt/filesystem/etc/fai -s file:///mnt/filesystem/var/lib/fai/config softupdate}
  end

def options
    options = Optimist::options do
      opt :sources_list,      "Path to APT’s sources.list containing necessary configuration"
      opt :debug,             "Show debug output"
      opt :verbose,           "Show verbose output"
    end

unless File.exist? options[:sources_list]
      raise "Given sources.list (#{options[:sources_list]}) does not exist."
    end
    @sources_list = options[:sources_list]

set_opts options
  end

end

07070100000035000081A4000001C3000001CA000000015F6847FC000004BF000000000000000000000000000000000000002D00000000fai-toolkit-1600669692.02e37ec/lib/output.rb
# Outputting utilities
module Output

# Everything should only be usable for the class inheriting this module.
  private

def debug str
    return unless $_debug
    puts "#{color(190, 190, 190)}[DEBUG]#{stopcolor} #{str}"
    $logfile.write "[DEBUG] #{str}" if $do_log
  end

def verb str
    return unless $_verbose
    puts "#{color(130, 230, 130, bold: true)}[INFO]#{stopcolor} #{str}"
    $logfile.write str if $do_log
  end

def warn str
    STDERR.puts "#{color(190, 190, 110, bold: true)}[WARN]#{stopcolor} #{str}"
    $logfile.write str if $do_log
  end

def error str, exitcode: 1
    STDERR.puts "#{color(230, 110, 110, bold: true)}[ERROR]#{stopcolor} #{str}"
    $logfile.write str if $do_log
    exit exitcode
  end

def color red, green, blue, bold: false
    return if ENV["COLORTERM"] != "truecolor" && ENV["COLORTERM"] != "24bit"

out = ""
    out += "\x1b[38;1;1m" if bold
    # https://stackoverflow.com/questions/4842424/list-of-ansi-color-escape-sequences
    out += "\x1b[38;2;#{red};#{green};#{blue}m"

out
  end

def stopcolor
    return if ENV["COLORTERM"] != "truecolor" && ENV["COLORTERM"] != "24bit"

"\x1b[0m"
  end

def self.logfile
    $logfile
  end
end
07070100000036000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000002D00000000fai-toolkit-1600669692.02e37ec/lib/packaging07070100000037000081A4000001C3000001CA000000015F6847FC0000248E000000000000000000000000000000000000003E00000000fai-toolkit-1600669692.02e37ec/lib/packaging/build-package.rbrequire "fileutils"

# Gems.
require "optimist"
require "mimemagic"

require_relative "../fai-config"
require_relative "../common"
require_relative "packaging"
require_relative "qemu"
require_relative "ssh"

# Compress an image:
#   zstdmt --long -15 --zstd=hlog=26 disks/build_machine_image.qcow2

# TODO: Maybe there could be simpler check for raw images that checks that given file is ”raw image” by file command and
# then validate it’s raw image with qemu-img info?
class Packaging::BuildPackage
  include Common

DISABLE_SSH_KEY_CHECK = "-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"

def perform
    startup_checks
    real_perform
  end

def options
    options = Optimist::options do
      opt :source_path,       "Path to source code that needs to be built.",
          short: "s", type: :string
      opt :packages_path,     "Path where built packages as written to",
          short: "p", type: :string
      opt :build_machine,     "Path to build machine image. Given machine must be checkable with “qemu-img check”",
          short: "m", type: :string
      opt :hypervisor,        "Virtualization platform to use. Default is platform specific. To get list of available platforms, use --hypervisor help",
          short: "H", type: :string
      opt :target_project,    "Target OBS project. Defaults to source path’s innermost directory name.",
          short: "t", type: :string
      opt :target_distro,     "Target distribution for building",
          short: "D", type: :string
      opt :target_arch,       "Target architecture for building",
          short: "a", type: :string,    default: "x86_64"
      opt :debug,             "Show debug output",
          short: "d"
      opt :verbose,           "Show verbose output",
          short: "v"
    end

set_opts options

# TODO: some platform specific detection probably should be here.
    @hypervisor = options[:hypervisor]
    @hypervisor = "qemu" if @hypervisor.nil?
    if @hypervisor == "help"
      puts
      puts "Available hypervisors:"
      puts "  - qemu"
      puts
      exit 1
    end

available_hypervisors = [
      "qemu"
    ]
    unless available_hypervisors.include? @hypervisor
      raise "Given hypervisor is not available. See --hypervisor help for available hypervisors."
    end

if @hypervisor == "qemu"
      @impl = Packaging::Qemu.new options
    end

if options[:build_machine].nil? or options[:build_machine].empty?
      raise "--build-machine must be given."
    end
    @build_machine = options[:build_machine]
    @impl.build_machine = @build_machine

raise "#{@build_machine} is not valid file." unless File.exist? @build_machine

if options[:packages_path].nil? or options[:packages_path].empty?
      raise "--packages-path must be given."
    end
    @packages_path = options[:packages_path]

FileUtils.mkdir @packages_path unless Dir.exist? @packages_path

if options[:source_path].nil? or options[:source_path].empty?
      raise "--source-path must be given."
    end
    @source_path = options[:source_path]

raise "--source-path must be a valid directory" unless Dir.exist? @source_path

@mac = generate_mac
    @impl.mac = @mac

@tap_device = "buildtap0"
    @impl.tap_device = @tap_device
    @bridge_device = "virbr1"
    @impl.bridge_device = @bridge_device

@target_distribution = options[:target_distro]
    raise "--target must be given. See relevant project in OBS for valid values." if @target_distribution.nil? or @target_distribution.empty?
    @target_architecture = options[:target_arch]
    raise "--target-arch must be given. See relevant project in OBS for valid values." if @target_architecture.nil? or @target_architecture.empty?

# Default to directory where files to be built resides.
    @target_project = options[:target_project]
    @target_project = File.basename @source_path if @target_project.nil?
  end

private

def real_perform
    @impl.start_vm @build_machine

# Wait for VM to boot up.
    sleep 3

@impl.wait_for_connection

# Allow user to log in as build user.
    ssh_key = resolve_ssh_key
    execute_on_remote "root", %{ainsl -a -v /home/build/.ssh/authorized_keys "#{ssh_key}"}

remote_path = "cache/KC/#{@target_project}"

# Transfer files to target.
    execute_on_remote "build", %{cd #{remote_path} ; rm -rf #{@target_project}}
    # rsync is missing from build_machine, and it doesn’t make much sense anyway.
    #execute %{rsync -e "ssh -l build" -avzh #{@source_path} build@#{@impl.vm_ip}:build}
    execute %{scp #{DISABLE_SSH_KEY_CHECK} -r #{@source_path} build@#{@impl.vm_ip}:#{remote_path}/#{@target_project}}

execute_on_remote "build", %{cd #{remote_path} ; tar Jcf "#{@target_project}.tar.xz" "#{@target_project}"}

execute_on_remote "build", %{cd #{remote_path} ; env BUILD_DIR=/usr/lib/obs-build osc build --trust-all-projects -o "#{@target_distribution}" "#{@target_architecture}"}

if @target_distribution.include? "Debian"
      packagedir = "DEBS"
    else
      raise "Not implemented for other distributions than Debian so far."
    end
    execute %{scp #{DISABLE_SSH_KEY_CHECK} root@#{@impl.vm_ip}:/var/tmp/build-root/#{@target_distribution}-#{@target_architecture}/usr/src/packages/#{packagedir}/* #{@packages_path}}

#puts "DEBUG"
    #sleep 100000

@impl.close_hypervisors
  end

def startup_checks

# Set up mimemagic recognition bits.

MimeMagic.add "application/x-zstd", {
      extensions: [ "zst" ],
      magic: [
        [0, "\050\265\057\375".force_encoding("ASCII-8BIT")] #ASCII-8BIT is what Ruby says to be binary data.
      ],
      comment: "Zstandard v0.8+"
    }

# This probably matches some non-qcow2 files too.
    MimeMagic.add "x-application/qemu-image-format", {
      extensions: [ "qcow", "qcow2" ],
      magic: [
        [0, "\121\106\111\373"]
      ],
      comment: "Qemu image format."
    }

# Decompress image as necessary.
    mime = MimeMagic.by_magic(File.open @build_machine)

if mime.nil?
      raise "Were not able to recognize give file. Probably you’re using some new type of file that does not yet have support added to this script."
    end

case mime.subtype
    when "x-zstd"
      verb "Extracting image..."
      FileUtils.rm_f "/tmp/build_machine" # Remove existing image if it exists.
      execute %{zstdmt -d #{@build_machine} -o /tmp/build_machine}
      @build_machine = "/tmp/build_machine"

# Remove the extracted image
      at_exit do
        FileUtils.rm "/tmp/build_machine"
      end
    when "qemu-image-format"
      verb "Using qcow2 image directly. You can use compressed image to save some extra space."
    else
      raise "Unrecognized file type for #{@build_machine}. Currently handled qcow2 and zstd compressed qcow2 images."
    end

# If image was decompressed, the path needs to be updated on worker class too.
    @impl.build_machine = @build_machine

# Run platform specific checks.
    @impl.startup_checks

# Set up networking.

network_setup
    use_bridging
    at_exit do
      remove_tap_device
    end
  end

def generate_mac
    # Deterministic macs can be generated with sample(random: Random.new(1)).
    #22-22-22-xx-xx-xx
    "22:22:22:" + 6.times.map { num = (0..15).to_a.sample; num.to_s(16) }.each_slice(2).to_a.map { |arr| arr.join('') }.join(":").upcase
  end

def remove_tap_device
    # Only link delete is required, since it automatically deletes the tap device.
    #sudo ip tuntap del dev "#{@tap_device}" mode tap
    execute %{sudo ip link del dev "#{@tap_device}" master "#{@bridge_device}"}
  end

def use_bridging
    # Use libvirt’s bridge device to provide connection.
    verb "Adding #{@bridge_device}"

existing = execvalue %(ip -br link show #{@bridge_device} | awk '{ print $1 }')
    verb "Create tap device"
    execute "sudo ip tuntap add dev #{@tap_device} mode tap vnet_hdr"
    verb "Set tap device $tap_device’s master: $bridge_device"
    execute %{sudo ip link set dev "#{@tap_device}" master "#{@bridge_device}"} # set $bridge_device as the target bridge for $tap_device
    verb "Bring tap up"
    execute %{sudo ip link set dev "#{@tap_device}" up}
  end

def network_setup
    # Ensure that apt-cacher-ng is usable inside VM.
    execute "sudo firewall-cmd --add-port=3142/tcp", fail: false, log: false
    @@port_originally_open = !$?.success?

# Necessary to get traffic working with wlan interface.
    @@original_proxy_arp = execvalue "sudo sysctl net.ipv4.conf.all.proxy_arp -n"
    execute "sudo sysctl net.ipv4.conf.all.proxy_arp=1", log: false

at_exit do
      execute("sudo firewall-cmd --add-port=3142/tcp", fail: false, log: false) unless @@port_originally_open
      execute "sudo sysctl net.ipv4.conf.all.proxy_arp=#{@@original_proxy_arp}", log: false
    end
  end

# Just a dummy method to read one public key from ~/.ssh
  def resolve_ssh_key
    keys = Dir.glob File.expand_path("~/.ssh/id_*.pub")
    raise "Failed to resolve ssh public key. Probably an option should be added to manually specify one." if keys.empty?
    keys.sort!
    File.readlines(keys.first).first
  end

def execute_on_remote user, command
    execute %{/usr/bin/ssh #{DISABLE_SSH_KEY_CHECK} -p 22 #{user}@#{@impl.vm_ip} '#{command}'}
  end
end07070100000038000081A4000001C3000001CA000000015F6847FC00000433000000000000000000000000000000000000004700000000fai-toolkit-1600669692.02e37ec/lib/packaging/generate-build-machine.rb
require "fileutils"
require "optimist"

require_relative "../fai-config"
require_relative "packaging"

# TODO: this is probably just useless, since this is pretty one-off script. Better just have necessary steps documented somewhere.
class Packaging::GenerateBuildMachine
  include Common

def perform
    startup_checks
    real_perform
  end

def options
    options = Optimist::options do
      opt :filename,          "Path to resulting file",
          short: "n", type: :string
      opt :force,             "Overwrite existing image",
          short: "f"
      opt :debug,             "Show debug output",
          short: "d"
      opt :verbose,           "Show verbose output",
          short: "v"
    end

set_opts options

if File.exist? options[:filename]
      unless options[:force]
        raise "File #{options[:filename]} already exists."
      else
        FileUtils.rm options[:filename]
      end
    end
    @result_filename = options[:filename]
  end

private

def real_perform
    TODO
  end

def startup_checks
    TODO
  end
end
07070100000039000081A4000001C3000001CA000000015F6847FC00000016000000000000000000000000000000000000003A00000000fai-toolkit-1600669692.02e37ec/lib/packaging/packaging.rb
module Packaging
end
0707010000003A000081A4000001C3000001CA000000015F6847FC000015BC000000000000000000000000000000000000003500000000fai-toolkit-1600669692.02e37ec/lib/packaging/qemu.rb
require_relative "packaging"
require_relative "../common"

# TODO: check for sudo available for qemu process.
class Packaging::Qemu
  include Common

@@qemu_pids = []

attr_accessor :mac, :tap_device, :bridge_device, :build_machine
  attr_reader :vm_ip

def initialize opts
    unless command_available? "qemu-system-x86_64"
      raise "qemu for amd64 needs to be installed"
    end

unless command_available? "qemu-img"
      raise "qemu-img needs to be installed"
    end

@opts = opts
  end

def startup_checks
    execute %{qemu-img check #{@build_machine}}
    raise "Validating #{@build_machine} failed." unless $?.success?

at_exit do
      terminate_qemus
    end
  end

def start_vm build_machine
    #graphics_params="-display none -nographic"
    graphics_params=""

network_device = "-device virtio-net-pci,netdev=hostnet0,id=net0,mac=#{@mac}"
    netconfig = "-netdev tap,id=hostnet0,ifname=#{@tap_device},vhost=on,script=no,downscript=no #{network_device}"
    # cd_image: UPDATE_IMAGE_NAME can be used for debugging.
    qemu_pid = start_qemu_vm "Build machine VM", disk_image_name: build_machine, netconfig: netconfig,
        additional_params: "-no-reboot", graphics_params: graphics_params
  end

def close_hypervisors
    terminate_qemus
  end

# Wait for SSH connection to be available.
  def wait_for_connection
    # Connection may not yet be up, so this may block for a bit.
    @vm_ip = find_vm_ip @mac

# Then wait for SSH to be available
    wait_for_ssh

@vm_ip
  end

private

# Waits for approximately maximum of 20 seconds.
  def wait_for_ssh counter: 0
    remote_ssh_port = 22
    begin
      Socket.tcp(@vm_ip, remote_ssh_port, connect_timeout: 3) {}
    rescue Errno::ETIMEDOUT => e
      if counter > 19
        puts.STDERR "Connecting to host #{@vm_ip}:#{remote_ssh_port} failed. Please check if there is a connection problem."
        raise e
      else
        wait_for_ssh counter: (counter + 3)
      end
    rescue Errno::ECONNREFUSED => e
      raise "Failed to connect to #{@vm_ip}:22: Connection refused." if counter > 19
      sleep 1
      wait_for_ssh counter: (counter + 1)
    end

nil
  end

# May block for a bit.
  def find_vm_ip mac, counter: 0
    vm_ip = execvalue %{ip neigh | grep -i #{mac} | cut -d" " -f1}
    begin
      IPAddr.new vm_ip
    rescue IPAddr::InvalidAddressError
      raise "Were not able to find out test VM’s IP address." if counter > 10
      sleep 1
      vm_ip = find_vm_ip mac, counter: (counter + 1)
    end
    #debug "Found ip #{vm_ip}"
    vm_ip
  end

def terminate_qemus
    @@qemu_pids.delete_if do |pid|
      silent_kill pid
      true
    end
  end

def start_qemu_vm vm_name, disk_image_name:, memory: 1000, netconfig:, cd_boot: false, cd_image: nil, additional_params: "", graphics_params: ""
    graphics_stack = "-display egl-headless -spice addr=127.0.0.1,port=18000,disable-ticketing,seamless-migration=on,playback-compression=off,image-compression=off"
    #monitor_params = "-monitor unix:/tmp/qemu_monitor"
    # mode=readline == “human” control mode, mode=control == QMP (json protocol)
    monitor_params = "-chardev socket,id=mon0,path=/tmp/qemu_monitor,server,nowait -mon chardev=mon0,mode=readline,pretty=on"
    disk_params = "-drive file=#{disk_image_name},driver=qcow2,if=virtio,index=1,aio=native,media=disk,cache=unsafe,cache.direct=on,cache.no-flush=on,discard=unmap,detect-zeroes=unmap"

cd_boot_params = cd_boot ? "-boot d" : ""
    cd_params = cd_image ? "-cdrom #{cd_image}" : ""
    params = "-smp #{(Etc.nprocessors / 2.0).round} -accel kvm -enable-kvm #{disk_params} #{cd_boot_params} #{cd_params} -m #{memory} #{monitor_params} #{additional_params}"
    command = %[sudo qemu-system-x86_64 -global isa-fdc.driveA= -name "#{vm_name}" #{params} #{netconfig} #{graphics_stack} #{graphics_params}]
    pid = execute command, wait: false

sleep 0.1
    if not running? pid
      raise "Starting qemu failed."
    end

@@qemu_pids << pid
    pid
  end

def send_to_qemu_monitor command
    # TODO: use ruby for this
    #execute "echo #{command} | socat - UNIX-CONNECT:/tmp/qemu_monitor"
    execute "echo #{command} | sudo nc -N -U /tmp/qemu_monitor", fail: false
  end

def shutdown_qemu qemu_pid
    send_to_qemu_monitor "system_powerdown"

begin
      Timeout.timeout(10) do
        Process.wait qemu_pid
      end
    rescue Timeout::Error
      raise "Failed to cleanly close qemu with pid #{qemu_pid}"
    end
  end

# First checks if the process even is running.
  def silent_kill pid
    return unless running? pid
    # Just killing the sudo process sometimes seems to fail, so let’s kill actual qemu process.
    sub_pids = execvalue "pgrep -P #{pid}"
    unless sub_pids.empty?
      execute "sudo kill #{sub_pids}", fail: false
      unless $?.success?
        debug "Killing subpids #{sub_pids} didn’t work... If there is a warning following, something is wrong. If not, the process probably already was killed. Or following code is broken."
        if running? sub_pids.to_i
          sleep 0.2
          puts "[WARNING] Had to kill #{pid} with force"
          execute "sudo kill -9 #{pid}"
        end
      end
    else
      execute "sudo kill #{pid}", fail: false
      unless $?.success?
        sleep 0.2
        puts "[WARNING] Had to kill #{pid} with force"
        execute "sudo kill -9 #{pid}"
      end
    end
  end

def running? pid
    begin
      Process.getpgid pid
      true
    rescue Errno::ESRCH
      false
    end
  end
end0707010000003B000081A4000001C3000001CA000000015F6847FC00000644000000000000000000000000000000000000003400000000fai-toolkit-1600669692.02e37ec/lib/packaging/ssh.rb
require_relative "../common"

class Packaging::SSH
  include Common

def self.ssh ip, command_to_run, user: "root"
    verb "SSH testing: #{command_to_run}"

# -o PasswordAuthentication=no
    # -o BatchMode=yes
    command = "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no #{user}@#{ip} #{command_to_run}"
    debug "SSH test command: #{command}"

read, write, pid = PTY.spawn command

begin
      result = read.expect /('s p|P)assword: /, 3
      if result.nil?
        puts "Got bad result from command #{command}:"
        puts result
        Process.waitall
        raise "Failed to run command"
      end
      debug "expect result: #{result.inspect}"
    rescue Errno::EIO
      # On Linux, this is thrown if a program has closed the stream.
      # It may or may not be error; if password was expected, it is an error.
      Process.wait pid
      read.close
      write.close
      return ""
    end

# These sleeps are probably unnecessary, but just to be more safe.
    sleep 0.1

# Totally insecure password I’ve used in testing.
    write.puts "mau"

sleep 0.1

output = ""
    Timeout.timeout(5) do
      buffer = String.new capacity: 1024
      begin
        while not read.eof?
          read.read_nonblock 1024, buffer
          #debug "Buffer: #{buffer}"
          output << buffer
        end
      rescue Errno::EIO
        # On Linux, EIO is thrown when command has finished.
      end
    end

debug "Got output: #{output.inspect}"

# Most often should be noop, but sets $? for us.
    Process.wait pid

output
  end
end0707010000003C000041ED000001C3000001CA000000025F6847FC00000000000000000000000000000000000000000000003100000000fai-toolkit-1600669692.02e37ec/lib/system-update0707010000003D000081A4000001C3000001CA000000015F6847FC000003F8000000000000000000000000000000000000003C00000000fai-toolkit-1600669692.02e37ec/lib/system-update/perform.rb
require "fileutils"
require "optimist"

require_relative "../common"
require_relative "../output"
require_relative "../media/mount"
require_relative "system-update"

# Main class.
class SystemUpdate::Perform
  include Common

def perform
    mount = Media::Mount.new

# Only mount once; this script can be run multiple times to help with debugging.
    unless Dir.exist? "/mnt/filesystem/var/lib/fai/config"
      mount.mount_cd
    end

sourceslist = "/etc/apt/sources.list"
    FileUtils.mv sourceslist, "#{sourceslist}.BAK"
    FileUtils.cp "/mnt/filesystem/#{sourceslist}", sourceslist
    at_exit do
      FileUtils.rm sourceslist
      FileUtils.mv "#{sourceslist}.BAK", sourceslist
    end

execute %{fai -v -N -C /mnt/filesystem/etc/fai -s file:///mnt/filesystem/var/lib/fai/config softupdate}
  end

def options
    options = Optimist::options do
      opt :debug,             "Show debug output"
      opt :verbose,           "Show verbose output"
    end

set_opts options
  end

end
0707010000003E000081A4000001C3000001CA000000015F6847FC00000019000000000000000000000000000000000000004200000000fai-toolkit-1600669692.02e37ec/lib/system-update/system-update.rb
module SystemUpdate
end
07070100000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000B00000000TRAILER!!!279 blocks

Places

File _service:obs_scm:fai-toolkit-1600669692.02e37ec.obscpio of Package fai-toolkit (Revision ff38dfb86bdda42d85c6b4ee77674711)

Places