Tag Archives: Linux

SALT – CLONE YOUR GIT REPO IN A STATE — PYTHON DEV SETUP

GOAL

  • Create a state to be utilised in your working environment
  • Clone git repo!

PERQUISITES:

SaltStack installed on the machine.
https://juhosyrjanen.com/2015/11/05/salty-assignment-i-saltstack/

ENVIRONMENT:

VM1: Running Mint 17.2, Salt master ||  specs

DEV SETUP

I wanted to create a highstate that first installs Git, clones my repository and installs some Python libraries.

GIT

I created this ‘not-so-well-executed’ state to clone my repository for my current project.
State located in /srv/salt/git

git

Running commands in Salt state is not recommended, but this does do the job. My repository is open, therefore there’s no need for passwords in this execution.

I tested this state, it ran fine the first time, but ran into problems using cmd.run.

repo

Instead of using cmd.run, I used cmd.wait, which will only modify directories/scripts if changes are made.

gitfix

Running the state a few times, and all is good.

state

I also added a few line to the git-state to automate the login, so that there’s no need to always enter password on git push.

confgit

 

PYTHON

Another module for the Python installation, done in a similar way to the git-module.
Capture2

Before proceeding, both modules were tested to be working correctly.

Now in the top.sls -file, I added the modules to be set to one of the slaves I have running.
Capture4

Capture3

And then I ran the highstate.
sudo salt slaveVM01 state.highstate

Capture5

It worked.

I created this highstate in order to more easily manage my laptop and other computers I use for development purposes. It will be a lot more easier to run a few states than always manually install python libraries and clone a git repo.

 

  • Copying, modifying and redistributing this and all the other documents in this blog is allowed according to the GNU General Public License (versio 2 or newer).http://www.gnu.org/licenses/gpl.html
  • Based on the Linux course by Tero Karvinen 2015: terokarvinen.com

 

Advertisements

SALTSTACK: MASTER & SLAVES

GOAL:

Setup Salt master and minion -architecture.
Create a module for the minions.

PERQUISITES:

SaltStack installed on the machines.
https://juhosyrjanen.com/2015/11/05/salty-assignment-i-saltstack/

ENVIRONMENT:

Three virtual machines.
VM1: Running Mint 17.2, Salt master || specs
VM2: Running Mint 17.2, Salt minion
VM3: Running Mint 17.2, Salt minion

SETTING UP SALT MASTER AND MINIONS

I started setting up my master and minions following the walk-through in doc.saltstack. 

First I started the salt master -service.

sudo service salt-master start

Capture6

For the minion – master needs to be defined. So I added my master VMs IP to the minions /etc/salt/minion -file.
Capture7

I started the minion with

sudo salt-minion -d

This will generate a minion id, which will be stored in /etc/salt/ as shown in the pictures bellow

Capture8 Capture9

Now when I ran sudo salt-key -L on the master VM:
Capture10

I accepted the pending key with

sudo salt-key -A

And ran the sudo salt-key -L again:
Capture11

Capture12

I verified that the keys do match by running salt-key -F master on the master VM and salt-call key.finger --local on the minion VM.
Capture13Capture14

I verified that the master can ping the minion by running the test.ping command:
Capture15

The master can now start sending command to the minion! Installation successful!

I repeated the steps for the other VM and ran the test command again:
Capture1

MODULE

I created a top.sls file, in which I set states to be installed to my slaves. The file looks a little something like this:

top2

 

In the top.sls -file, I set Apache -state to be installed on all the slaves and specified the other packages to certain individual slaves.

I tested that all my states work on my master VM:

MySQL:
mysql

Apache:
apa

SSH:
ssh

Running the top file

All the individual states worked fine, so I ran the top file!
sudo salt '*' state.highstate

Salt took a while to run the file, but it seemed to have worked without any errors.
result1

result2

result3

Success

I ran the highstate again to confirm the results, and everything seemed to have worked fine.

  • Copying, modifying and redistributing this and all the other documents in this blog is allowed according to the GNU General Public License (version 2 or newer).http://www.gnu.org/licenses/gpl.html
  • Based on the Linux course by Tero Karvinen 2015: terokarvinen.com

SALT: PILLARS & GRAINS

GOAL:

Use Salt pillars & grains.
Setup Salt master – minion

Time elapsed: approx 3h.

SETUP:

Three virtual machines.
VM1: Running Mint 17.2, Salt master || specs
VM2: Running Mint 17.2, Salt minion
VM3: Running Mint 17.2, Salt minion

All VMs are set to bridged connections.

INSTALLING SALT

VM1

This is the same machine I used in the previous article. Salt is already installed. Here’s how the machine is configured: last article

I only uninstalled the Salt minion from this machine using the command:

sudo apt-get remove salt-minion

 

VM1 & VM2

For the slaves, I installed Salt using the same steps as used in the last article:

First I imported the SaltStack repository key

wget https://repo.saltstack.com/apt/ubuntu/ubuntu14/latest/SALTSTACK-GPG-KEY.pub | sudoapt-key add SALTSTACK-GPG-KEY.pub

Opened the sources list

sudoedit /etc/apt/sources.list

I Added the following line to the list:

deb http://repo.saltstack.com/apt/ubuntu/ubuntu14/latest trusty main

Next I ran apt-get update

sudo apt-get update

Installed only minion and SSH -components for the minions.

sudo apt-get install salt-minion salt-ssh

And there, installation completed!

Before proceeding I checked the installation status by running

$ sudo salt-call --version

Capture5

Capture4

SETTING UP SALT MASTER AND MINIONS

I started setting up my master and minions following the walk-through in doc.saltstack. 

First I started the salt master -service.

sudo service salt-master start

Capture6

For the minion – master needs to be defined. So I added my master VMs IP to the minions /etc/salt/minion -file.
Capture7

I started the minion with

sudo salt-minion -d

This will generate a minion id, which will be stored in /etc/salt/ as shown in the pictures bellow

Capture8 Capture9

Now when I ran sudo salt-key -L on the master VM:
Capture10

I accepted the pending key with

sudo salt-key -A

And ran the sudo salt-key -L again:
Capture11

Capture12

I verified that the keys do match by running salt-key -F master on the master VM and salt-call key.finger --local on the minion VM.
Capture13Capture14

I verified that the master can ping the minion by running the test.ping command:
Capture15

Yay! The master can now start sending command to the minion! Installation successful!

Grains

I want to make my slaves run a website that states their basic info, such as IP, hostname, role.. etc. To test this out I will do this on the VM1. I will create a new state, that will install Apache and define a template engine for the website. In the template, I will create a simple webpage, that will fetch the grains.

First I begin with checking what kind on grains I can actually utilize on my minion machines.

$ sudo salt-call --local grains.items

This command returns a huge list of grains that can be utilized. I’m interested in the “host” and “ip4_interfaces” -grains.
Capture1

To make the state, I first created a new folder for it and called it “status”

$ sudo mkdir /srv/salt/status

Inside the folder I created a new sls. -file and called it “init.sls”.

$ sudoedit init.sls

First I used #!pyobjects to install Apache and define the template engine:

#!pyobjects

with Pkg.installed("apache2")
File.managed("/var/www/html/index.html",
source="salt://status/index.html.jinja", template="jinja")

I tested if my state works:

$ sudo salt-call --local state.sls status

Capture2

Next I wrote the template file.

 

Hostname: {{ grains['host']}}
IP: {{ grains['ip4_interfaces']}
MAC: {{ grains['hwaddr_interfaces']}}

Then, I ran the state again and opened http://localhost in Firefox.

Capture3

Worked as expected, with minor formatting flaws.

 

PILLARS

I created a basic pillar following the doc.saltstack walk-through.

So first I created a directory for the pillars in /srv/

sudo mkdir /srv/pillar

And in the pillar-directory I created the top.sls -file.

sudoedit /srv/pillar/top.sls

And in the top.sls file, I just created a very simple file to test the pillar.

base:
'*':
- info

After this I created the info.sls -file and added some data into it.

info: testing pillar

Now that the test pillar files are created, I tested them out using these commands:

sudo salt '*' saltutils.refresh_pillar

sudo salt '*' pillar.items

Getting these results:
Capture16

Basic pillar test completed, worked without problems.

  • Copying, modifying and redistributing this and all the other documents in this blog is allowed according to the GNU General Public License (versio 2 or newer).http://www.gnu.org/licenses/gpl.html
  • Based on the Linux course by Tero Karvinen 2015: terokarvinen.com

SALTY ASSIGNMENT I: SALTSTACK

ASSIGNMENT:

a) Install SaltStack
b) Create a Package-File-Service state for Salt Stack

Assignment based on the course “Linuxin keskitetty hallinta” by Tero Karvinen

PLATFORM

  • Linux Mint 17.2 running on VirtualBox

GETTING SALTY – INSTALLATION

Firstly after booting the virtual machine, I started by running the basic

sudo apt-get update
sudo apt-get upgrade

To install SaltStack, I followed the instructions on docs.saltstack, found here

SaltStack requires repo.saltstack to be added to the sources.list. I proceeded with the instructions by running these commands:

First I imported the SaltStack repository key

wget -O - https://repo.saltstack.com/apt/ubuntu/ubuntu14/latest/SALTSTACK-GPG-KEY.pub | sudo apt-key add

Capture1

Next I added the repo to sources.list:

sudo nano /etc/apt/sources.list

I Added the following line to the list:

deb http://repo.saltstack.com/apt/ubuntu/ubuntu14/latest trusty main

Next I ran

sudo apt-get update

And then install the SaltStack components wanted! I added the master, minion and ssh components.

sudo apt-get install salt-master salt-minion salt-ssh

And there, installation completed!

Before proceeding I checked the installation status by running

 
$ sudo salt-call --version 

Capture2


PACKAGE-FILE-SERVICE

I created a DHCP-service using Salt and PyObjects:

First I created the .sls -file, that is used to run the state.  I created a directory for all the future Salt stuff in which I also created the sls. -file.

 
sudo mkdir -p /srv/salt
sudoedit /srv/salt/dhcp.sls

Documentation for the use of PyObjects in SaltStack states can be found here – docs.saltstack.
I used the article by Tero Karvinen as refenrence.

To use PyObjects in the sls. -file I first defined the shebang as

 #!pyobjects 

After the  So I added a line to the state-file:

 
with Pkg.installed("bind9"): 
        Service.running("bind9", enable=True)

Since dhcp-service will need bind9 to work properly, it will be installed first. I tested the line above and it worked like a charm.

Capture4

Added dhcp install command:

 
with Pkg.installed("bind9"): 
        Service.running("bind9", enable=True)

with Pkg.installed("isc-dhcp-server"):
        Service.running("isc-dhcp-server", enable=True)

After this, I ran the sls. file again with the command:

 
$ sudo salt-call --state-output=mixed --local state.sls dhcp

Capture5

It worked. This is a very basic installation of dhcp-service. Configuration has not been specified and has to be done by hand when using this implementation.

Later I will post a more detailed guide on how to modify the configuration files in Salt inside the state- file.

  • Copying, modifying and redistributing this and all the other documents in this blog is allowed according to the GNU General Public License (versio 2 or newer).http://www.gnu.org/licenses/gpl.html
  • Based on the Linux course by Tero Karvinen 2015: terokarvinen.com

HOW TO CHANGE YOUR IP TO STATIC ON UBUNTU SERVER

To change your IP form dynamic to static, we will need to look into the /etc/networks/interfaces -file.

sudo nano /etc/networks/interfaces

This will open up the configuration file in which you can tweak your IP setting. It will look a bit like this:
Selection_001
You will need to change a few lines in order to make your IP static. Add these changes into the file:

auto eth0
iface eth0 inet static
address 10.5.5.1
netmask 255.255.255.0
#network 10.5.5.0
#broadcast 10.5.5.31

Depending on your needs, you can also configure netmask and other necessary configurations here.
# – means that the setting is commented out, and will not apply.

Your finished file will look like this:
Selection_002

Save the file and run these commands:

sudo ifdown eth0
sudo ifup eth0

This will apply the changes made and give you a new IP. Now if you run ifconfig your IP should display as 10.5.5.1 – you now have a static IP.

DHCP & DNS // CREATING A LOCAL NETWORK BETWEEN VIRTUAL MACHINES // ASSIGNMENT III

So, my external hard drive is a brick. To summarise the problem; It just won’t boot. So I am will run this assignment on a Windows desktop, since my laptop is not strong enough to run so many virtual machines at once.

Specs:
Windows 7 Enterprise 64-bit
Intel Core i5-2400 @ 3.10GHz
Some integrated Intel HD graphics GPU
8GB of RAM

Assignment:

Create four virtual machines:

## Machine A: Linux desktop, I’ll be using Mint 17. This machine will receive its network address from the DHCP -server.  Eth0 in the local network (intnet). ## Machine A not used in this post

Machine B: Master, running Ubuntu server. This machine will work as a DHCP – server. Static IP, Eth0 in local network.

Machine C: Bridge, running Ubuntu server. This machine works as a NAT -distributor (If it works..) between the local and public network. Static IP, Eth0 in public network (bridged), Eth1 in local network (intnet).

Machine D: Running Ubuntu server. Receives IP from the DHCP -server. Address reserved before hand using MAC -address. Eth0 in local network. Calling this machine ‘Aku’ on this post.

Step 1.

Creating and installing the virtual machines will be the first step. I begun with installing all the machines using Ubuntu server. I gave them all 512 MB of RAM and 8 GB of space on the hard drive, the desktop client gets a bit more RAM etc, but servers will manage with less.

During the installation, I also installed the OpenSSH -client. Machine B also got the DNS-server package during installation.

Bridged network setting is necessary here, since we still need to access public network, in order to make a few installations here.

Machine C – the bridge, will need two network adapters here. It will work as a gateway to the public network for the other machines. Set Eth0 to public network setting – Bridged, and Eth0 to internal network. Primary network adapter = Eth0.

After setting up Machine C – the bridge, run this command:
ls /sys/class/net/
This should list you the following adapters: eth0, eth1 and lo.

Step 1.2 DHCP-server installation

After the machines are setup, installing the DHCP -server on the master machine is necessary. It can be done with the following command:

sudo apt-get install isc-dhcp-server

Be sure to have bind9 installed before doing this. (sudo apt-get install bind9)

This will install and start the DHCP -server. However, it still needs go through the process of configuration.

After the installation is completed, everything that needs to be installed on this machine has been installed. Now we can change the IP to a static IP.

sudo nano /etc/networks/interfaces

This will open up the configuration file in which you can tweak your IP setting. It will look a bit like this:
Selection_001
You will need to change a few lines in order to make your IP static. Add these changes into the file:

auto eth0
iface eth0 inet static
address 10.5.5.1
netmask 255.255.255.0
network 10.5.5.0
broadcast 10.5.5.255
#dns-nameservers 10.5.5.1

For now, leave the dns-nameservers commented out.
masterinterfaceconf

Save the file and run these commands:

sudo ifdown eth0
sudo ifup eth0

This will apply the changes made and give you a new IP. Now if you run ifconfig your IP should display as 10.5.5.1 – you now have a static IP.

Configuring the /etc/hosts -file might not be necesary, but I’d say do it just to be sure. Capture
We will want to change the IP here to our own static IP and fill in the host’s name – master.

Step 1.2 Gateway//Bridge

Before testing, make sure all your other machines have also got the appropriate IP settings in /etc/network/interfaces:
Capture2
On the bridge machine, we will want to leave the eth0 -setting untouched and make a similar configuration for the eth1 -adapter.
Eth1 – adapter is in the internal network, so we will want to give it static IP and other configurations as seen above.

Remember, for the settings to apply, you will need to run ifdown and ifup -commands.

Step 2. Testing the work done so far

At this point we can test if that the machines can find each other using SSH.
We can start by trying to connect from MASTER —> BRIDGE

ssh juho@10.5.5.2

You should get similar results:
sshtest

Step 3. Configuring the DHCP -Server

Configuration here is done in a similar way to the prior part of the assignment. You will need to access the .conf file and tweak it a bit.

sudo nano /etc/dhcp/dhcpd.conf

This is how it will look by default:
Selection_003

You will need to add these lines into the .conf -file:

subnet 10.5.5.0 netmask 255.255.255.0 {
range 10.5.5.10 10.5.5.30;
option domain-name-servers 10.5.5.1, 8.8.8.8; 
## 8.8.8.8 is google's name server, which work as a backup here
option domain-name "yournetwork'snamehere.example";
option broadcast-address 10.5.5.255;
#option routers 10.5.5.1 #if you need this 
}

After adding this to the file, it should look a bit like this:
dhcp3

Depending on how you want to build your network, you might want to set fixed addresses to your machines. This is done using MAC-addresses and the configuration is done to the same file we’re in now. Here’s how you’ll do it.

 host hostname {
      hardware ethernet 08:00:27:ff:03:57; 
      fixed-address 10.5.5.40;
      } #remember to close the script

“hardware ethernet” is the mac-address of the machine whose IP you want to set as fixed. It can be found in ifconfig.
“fixed-address” is the IP you want to set. Preferably set it outside the range set above, to avoid overlaps.

My conf-file looks like this:
dhcp4
I gave all my machines fixed-addresses.

Now we need to restart the DHCP -server. It is done with this command:

sudo /etc/init.d/isc-dhcp-server restart

You should get similar results:
Selection_005
Now the DHCP -server has been configured.

Now the other machines should receive a fixed address, if their network is set to internal. Let’s test this. Machine D, aka Aku, let’s refresh the connection with ifdown and up, and see what kind of IP will we get. It’s supposed to be 10.5.5.40.
akutest
Looks like it worked!  We can also see from the messages on the terminal, that the DHCP-server is working properly.

Next step is to configure the server so, that the machines in internal network can also access public network. That will be done in the next post.

DXDIAG EQUIVALENT ON LINUX // HOW TO EASILY CREATE A DOCUMENT OF YOUR WORKSTATION’S SPECS

DirectX Diagnostic Tool (dxdiag) is quite handy on Windows for checking your specs.
We have similar tools on the command prompt on Linux.
The command is as follows:

         sudo lshw -html > specs.html

This command will create a HTML-file with your workstation’s specs listed in it.
The file will be created in to the folder you’re currently located in. So if you want this in your doc file, remember to first relocate there.

The command will generate A LOT of info. However, you can browse through it quickly and find the info needed (such as GPU, CPU, drivers etc).

Alternately, you can just dump the file into your desktop with this command:

         sudo lshw -html > /home/*your_username*/Desktop/specs.html

You can open the file in any browser. The HTML-file looks a little something like this:cpu

Above information about my workstation’s CPU.
Bellow basic header for the file.

linuxboys

The full file of my workstation can be found here here: specs

This is a very handy tool when you need to quickly check your gear or when you need to post the workstation’s stats into an assignment/research w/e.

If you don’t fancy messing around with the command promt, you can also use a program called “Hardinfo”. You can find the program from the software installer on Ubuntu based distributions or use a simple command:

sudo apt-get install hardinfo

If you found this article helpful, please leave a comment or like and let me know!