HowTo: Wait for any process to finish on Bash 1

Well, here’s a quick one.

Let’s say you’re creating a KVM/Qemu guest; using oz-install; and you’re tired, ’cause it’s 06:31 and you need some sleep.

You don’t want to wait the 2k seconds it takes to do all the work so you want to setup a bash script that waits for the process and, then, shuts down your computer.

Here’s how:

Procedure

    # become root
    su -

    # wait for oz-install to end and poweroff
    while pgrep oz-install &> /dev/null; do echo "it's running..."; sleep 5; done; poweroff

HowTo: Build and Use Fedora 20 for/on Google Cloud 1

So, in my last post, I promised to post these instructions. As if I ever kept secrets from you! ¬_¬

Update: Please, visit my repo where you can find the updated code: https://github.com/renich/gce-images

So, here’s the magic:

Initialize (last post)

This takes care of setup. Read and follow instructions carefully. Obviously, you need to take care of billing settings and stuff when you create the account. In fact, probably, you need to create the account prior to login. Let’s see how it goes if you haven’t. Let me know on the comments.

# setup google cloud sdk
curl https://sdk.cloud.google.com | bash

# activate google cloud in current shell
source ~/.bash_profile

# login
gcloud auth login --no-launch-browser

# activate it
source ~/.bash_profile

setup script (unfinished)

This one is needed by virt-builder in order to edit the distro according to Google’s recommendations. It is not finished yet. It just works. I will update this part as soon as I’m finished.

#!/usr/bin/env bash

# WARNING!
# Please, do not run this script by it's own. It can destroy some of your configuration.
# This script is a setup script for the guest we're creating.

# hosts
cat < < 'EOF' > /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
169.254.169.254 metadata.google.internal metadata
EOF

# networking
rm -fr /etc/sysconfig/networking
rm -f /etc/udev/rules.d/70-persistent-net.rules
# rm -f /etc/hostname

cat < < 'EOF' > /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
DEVICE="eth0"
NAME="eth0"
ONBOOT="yes"
BOOTPROTO="dhcp"
IPV4_FAILURE_FATAL="yes"
DEFROUTE="yes"
MTU="1460"
DNS1="208.67.222.222"
DNS2="8.8.8.8"
EOF

# ntp
sed -ri '/^server [1-3]\.fedora.*$/d' /etc/ntp.conf
sed -ri 's@^server 0\.fedora.*$@server metadata.google.internal iburst@' /etc/ntp.conf

# disable firewall
systemctl disable firewalld.service
systemctl disable iptables.service

# yum updates
sed -ri 's@apply_updates = no@apply_updates = yes@' /etc/yum/yum-cron.conf

# ssh config
## delete the keys
rm /etc/ssh/ssh_host_key
rm /etc/ssh/ssh_host_rsa_key*
rm /etc/ssh/ssh_host_dsa_key*
rm /etc/ssh/ssh_host_ecdsa_key*

## patch sshd_config
patch sshd_config.test < <( cat << 'EOF'
18,19c18,19
< #AddressFamily any
< #ListenAddress 0.0.0.0
---
> AddressFamily inet
> ListenAddress 0.0.0.0
48c48
< #PermitRootLogin yes
---
> PermitRootLogin without-password
78c78
< PasswordAuthentication yes
---
> PasswordAuthentication yes
114c114
< #AllowTcpForwarding yes
---
> AllowTcpForwarding no
127c127
< #ClientAliveInterval 0
---
> ClientAliveInterval 420
133c133
< #PermitTunnel no
---
> PermitTunnel no
EOF
)

## lock root
# usermod -L root

# kernel
## remove symbol table
rm -f /boot/System.map*

## configuration
### strongly recommended
cat < < 'EOF' > /etc/sysctl.d/11-google-strongly_recommended.conf
# Google-recommended kernel parameters

# Reboot the machine soon after a kernel panic.
kernel.panic=10

# Addresses of mmap base, heap, stack and VDSO page are randomized.
kernel.randomize_va_space=2

# Ignore ICMP redirects from non-GW hosts
net.ipv4.conf.all.accept_redirects=0
net.ipv4.conf.all.secure_redirects=1
net.ipv4.conf.default.accept_redirects=0
net.ipv4.conf.default.secure_redirects=1

# Ignore source-routed packets
net.ipv4.conf.all.accept_source_route=0
net.ipv4.conf.default.accept_source_route=0

# Log spoofed, source-routed, and redirect packets
net.ipv4.conf.all.log_martians=1
net.ipv4.conf.default.log_martians=1

# Turn on Source Address Verification in all interfaces to
# prevent some spoofing attacks.
net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.default.rp_filter=1

# Don't pass traffic between networks or act as a router
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.default.send_redirects=0
net.ipv4.ip_forward=0

# Ignore ICMP broadcasts to avoid participating in Smurf attacks
net.ipv4.icmp_echo_ignore_broadcasts=1

# Ignore bad ICMP errors
net.ipv4.icmp_ignore_bogus_error_responses=1

# RFC 1337 fix
net.ipv4.tcp_rfc1337=1

# Turn on SYN-flood protections.  Starting with 2.6.26, there is no loss
# of TCP functionality/features under normal conditions.  When flood
# protections kick in under high unanswered-SYN load, the system
# should remain more stable, with a trade off of some loss of TCP
# functionality/features (e.g. TCP Window scaling).
net.ipv4.tcp_syncookies=1
EOF

cat < < 'EOF' > /etc/sysctl.d/12-google-recommended.conf
# provides protection from ToCToU races
fs.protected_hardlinks=1

# provides protection from ToCToU races
fs.protected_symlinks=1

# makes locating kernel addresses more difficult
kernel.kptr_restrict=1

# set ptrace protections
kernel.yama.ptrace_scope=1

# set perf only available to root
kernel.perf_event_paranoid=2
EOF

# google things
# todo:
#   * google-daemon depends on syslog.service; which does not exist in fedora

exit 0

Name this one “setup”. The build commands should be able to access the setup script.

build

Ok, this one takes care of building stuff. We will use the mighty virt-builder; from libguestfs-tools-c; by Mr. Richard WM Jones!

By the way, analyze and change accordingly. Do NOT copy/paste it. Understand it first, change it and use it.

#!/usr/bin/env bash

date=$( date +%Y%m%d%H%M%S )
project='evalinux-test'

# f1-micro
# g1-small
# n1-highcpu-2
# n1-highcpu-4
# n1-highcpu-8
# n1-highcpu-16
# n1-highmem-2
# n1-highmem-4
# n1-highmem-8
# n1-highmem-16
# n1-standard-1
# n1-standard-2
# n1-standard-4
# n1-standard-8
# n1-standard-16
machine_type='n1-highcpu-8'

# asia-east1-a
# asia-east1-b
# europe-west1-a
# europe-west1-b
# us-central1-a
# us-central1-b
zone='us-central1-b'

# virt-builder -l
## centos-6
## cirros-0.3.1
## debian-6
## debian-7
## fedora-18
## fedora-19
## fedora-20
## rhel-7rc
## scientificlinux-6
## ubuntu-10.04
## ubuntu-12.04
## ubuntu-14.04
os='fedora-20'

# Your Google Storage
gs='fedora-images'


function ask_continue
{
    # skip for now
    return
    shopt -s extglob

    echo -n 'Do you want to continue?: '
    read answer

    if [[ "${answer,,}" != @(yes|y|yep|sure|aha|yeah|yea) ]]; then
        exit 0;
    fi
}


# build image
echo 'building...'
virt-builder \
    --format raw \
    --size 10G \
    --timezone Etc/UTC \
    --password-crypto sha512 \
    --root-password password:fedoraadminpass \
    --install "irqbalance,openssh-server,openssh-clients,rsync,git,ntp,python,yum-plugin-fastestmirror,yum-plugin-merge-conf,yum-plugin-remove-with-leaves" \
    --install "yum-cron-daily,yum-utils" \
    --firstboot setup \
    --update \
    --selinux-relabel \
    -o disk.raw \
    ${os}

echo
echo 'done building. Next: compress'
ask_continue

# compress
echo 'compressing...'
tar -Szcf ${date}-image.${os}.tar.gz disk.raw

echo
echo 'done compressing. Next: upload'
ask_continue

# upload
echo 'uploading...'
gsutil cp ./${date}-image.${os}.tar.gz gs://${gs}

echo
echo 'done uploading. Next: add image'
ask_continue

# add test image
echo 'adding image...'
gcutil --project=${project} addimage ${os}-v${date} gs://${gs}/${date}-image.${os}.tar.gz

echo
echo 'done adding image. Next: create instance'
ask_continue

# create test instance
gcutil --project=${project} addinstance \
    --image=${os}-v${date} \
    --machine_type=${machine_type} \
    --zone=${zone} \
    ${os}-v${date}-test

echo 'done creating instance.'

exit 0

This sample script will generate an 8 CPU, ~8 GB RAM, 10 GB HDD instance @ USA. Tweak it if you need something else.

Alternatives


HowTo: Install Google Cloud SDK from the CLI on Fedora 20

Basically, this is a HowTo that prevents the usage of a browser in Google’s Cloud SDK installation.

Why?

Easy. I wanted to build my own image; from scratch. I wanted Fedora 20 on that cloud. The problem is that I have the poorest Internet connection (WiMax @ 2 Mbps/128 Kbps) so building stuff would take ages.

So, I used my CloudSigma Fedora 20 server for the build. That server is headless and I needed to authenticate with a browser if I followed Google‘s instructions.

Solution

# setup google cloud sdk
curl https://sdk.cloud.google.com | bash

# activate google cloud in current shell
source ~/.bash_profile

# login
gcloud auth login --no-launch-browser

This lets me authenticate the SDK by following a link and getting back a key to paste on the CLI. That easy.

You wanna know how to build Fedora on Google’s Cloud? I’ll tell you in another post.

Reference


HowTo: Fedora 20 and the FOSS xorg-x11-drv-ati driver

I’ve been having this problem lately:

My Fedora 20 would freeze; after a while, while using the xorg-x11-drv-ati driver.

Another thing I noticed, was the fan of the card going up and up like crazy.

First thing I did, was install lm_sensors:

yum -y install lm_sensors

And ran the sensors command:

# sensors
radeon-pci-0100
Adapter: PCI adapter
temp1:        +85.5°C  (crit = +120.0°C, hyst = +90.0°C)

k10temp-pci-00c3
Adapter: PCI adapter
temp1:        +38.6°C  (high = +70.0°C)
                       (crit = +90.0°C, hyst = +85.0°C)

Wow! 75.5 ºC on my AMD/Ati card! I know why the freezes happen now! It overheats and crashes!

So, I’ll do what you need to do when you have problems: RTFM!

http://xorg.freedesktop.org/wiki/RadeonFeature/#index3h2

Now, there is a hint right there. You can try and control the power profile; but that is not exactly it. You want the system to do it for you.

To immediately lower the temperature on my video card, I recurred to the following commands:

echo profile >  /sys/class/drm/card0/device/power_method
echo low >  /sys/class/drm/card0/device/power_profile

My answer was enabling dpm; by adding radeon.dpm=1 to the kernel line at /etc/grub2-efi.conf (in my case).

So, it looks something like this:

    linuxefi /vmlinuz-3.12.9-301.fc20.x86_64 root=UUID=6c9b3ffd-c911-4dcb-9425-e44841ab379d ro rootflags=subvol=root vconsole.font=latarcyrheb-sun16  rhgb quiet LANG=en_US.UTF-8 radeon.dpm=1
    initrdefi /initramfs-3.12.9-301.fc20.x86_64.img

Update 2014-07-11: It seems that the radeon.dpm solution is preventing 3d acceleration on my vanilla Fedora 20 installation; preventing gdm (all but LightDm) from working. I have to confirm this later by changing this statement.

Ok, now, the next thing is to rebuild my initrd image with dracut; which proved to be really easy:

su - 
dracut --force

Basically, if you don’t do that, you will be dropped to a shell when booting; just exit that and you’re good.


HowTo: cgminer as systemd service

Ok, I have been setting up my CubieTruck up with Fedora to be able to control my Butterflylabs hardware.

Obviously, I had to compile cgminer, which is straightforward; as mentioned here.

So, I want the CubieTruck to start cgminer, automatically, as a systemd process.

I need a service unit:

# /etc/systemd/system/cgminer.service 
[Unit]
Description=Start mining bitcoin
After=syslog.target network.target nss-lookup.target

[Service]
Type=forking
ExecStart=/usr/local/bin/cgminer --syslog --real-quiet -o stratum+tcp://stratum.mining.eligius.st:3334 -u 1K7FEkbvBtMvtyT7S3RrPjuqWuJWfgUZBa -p somepassword -l 9

[Install]
WantedBy=multi-user.target

And there it is! Next, just activate it to start on boot and start it:

su -c 'systemctl enable cgminer.service'
sy -c 'systemctl start cgminer.service'

If you wanna check out the output of cgminer; which is a lot, btw, you need journalctl:

su -c 'journalctl -u cgminer -f'

I recommend setting some sane limits on your journalctl.conf file:

/etc/systemd/journald.conf 
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.
#
# See journald.conf(5) for details

[Journal]
#Storage=auto
Compress=yes
#Seal=yes
#SplitMode=login
#SyncIntervalSec=5m
#RateLimitInterval=30s
#RateLimitBurst=1000
SystemMaxUse=100M
#SystemKeepFree=
SystemMaxFileSize=20M
RuntimeMaxUse=100M
#RuntimeKeepFree=
RuntimeMaxFileSize=20M
#MaxRetentionSec=
#MaxFileSec=1month
#ForwardToSyslog=yes
#ForwardToKMsg=no
#ForwardToConsole=no
#TTYPath=/dev/console
#MaxLevelStore=debug
#MaxLevelSyslog=debug
#MaxLevelKMsg=notice
#MaxLevelConsole=info

Now, I am no guru or anything like that. I might’ve missed some stuff. Keep the feedback coming, dudesses and dudes!