My First CHEF Cookbook !

[The cookbook works only for debian based OS]

Login To Chef Workstation:

ssh -i key.pem ubuntu@chef-workstation

Chef Workstation, @chef-repo directory
Creating My Apache Cookbook

#knife cookbook create myapache
#vim cookbooks/myapache/recipes/default.rb

package "apache2"  do
        action :install
end

#To delete default VirtualHost
cookbook_file "/etc/apache2/sites-enabled/000-default" do
  action :delete
end

template "/etc/apache2/sites-enabled/default" do
        source "default.erb"
        mode "0644"
end

#To delete default index.html
cookbook_file "/var/www/index.html" do
  action :delete
end

cookbook_file "/var/www/index.html" do
  action :create
end

service "apache2" do
        action [:enable,:start,:reload]
end

Back To Terminal

#echo "This is myapache cookbook" > cookbooks/myapache/files/default/index.html
#vim cookbooks/myapache/templates/default/default.erb
<VirtualHost *:80>
DocumentRoot /var/www/
</VirtualHost>

Back to Terminal, good to practice to update the cookbook version

#vim cookbooks/myapache/metadata.rb
version '0.1.1'

Upload the Cookbook to Chef-Server:

#knife cookbook upload myapache

Go To Chef Manage Console to 

  • Create a New Role called testrole2
  • Place the myapache cookbook in RunList of testrole2

Starting a new node to try out!

#knife ec2 server create -r "role[testrole2]" -I ami-8e987ef9 --flavor t1.micro -G unni-test-sg -x ubuntu -N MyChefPOC --region eu-west-1 -i /pathtosshkeyofNode.pem

Chef MyDoc [ugly version]

Chef – Webinar

Organizations : are like departments like Sales, Business Unit, Product etc
Every Organization starts with a single environment

Environment :  Staging, Development, Production = modeling life stages of the product.
-Environment include data attributes necessary for configuring your infra eg: If its a ecommerce app and its using paypal then the development environment should not use production paypal url for that we need to set sandbox url which is mentioned using data attributes.
-The location of package repo
-The version of chef config files that should be used.

Roles:
To setup the Role of each server.

Nodes :
They represent Servers
Each node will belong to
One Organization
One Environment
have zero or more Roles

Chef-Client:
a process that runs on each node
-gather current system config
-downloads configs from chef server
-and configure the node accordingly.

Policy – means cookbooks mentioned in Run List and Chef-Client ensure that a node follows its policy

Resource:
Represents a piece of the system and its desired state
Eg:
A package[piece of system] that should be installed[desired state]
A service that should be running
A file that should be generated
A cron job that should be configured.
A user that should be managed.

Recipes:
Collection of Resources
Recipes ensures the system is in desired state.
It is the configuration file that describe Resources and its desired state.
They are the real workhorses of chef
Recipes can :
Install and configure software components.
Manage files
Deploy Applications
Execute other recipes
and other

+++++++++++++++++++++++++++++++
Example Recipe:     NOTE: THE ORDER IS IMPORTANT  -> PACKAGE -> TEMPLATE -> SERVICE

package “apache2”

template “/etc/apache2/apache2.conf” do
source “apache2.conf.erb”
owner “root”
group “root”
mode  “0644”
variables (:allow_override => “All”)
notifies :reload, “service[apache2]”
end

service “apache2” do
action [:enable, :start]
support :reload => true
end

++++++++++++++++++++++++++++++++

The Chef Client is app that runs on each node.
It takes the Policy from chef server and inspect current system configuration and it will then walk thru the recipes
Make sure Node complies with Recipe.
Here first resource is apache2 , it will install if not already installed.
We dont have to tell how to install as chef-client is intelligent to figure that out.
And then move on to the next resource in the recipe.

2nd Resource is to tell chef-client to manage a file.

3rd Resource : named apache2 which is a service resource.
:enable – to start apache on reboot.
:start – to start the service if not running . [here we dont say how to start service, becos chef knows how to start its resource by its own.]
support :reload = means this service accepts reload command.

notifies :reload, “service[apache2]”
Chef-Client Lifecycle –
When chef-client runs it will check if the apache.conf file already exist or not.
If already exist then chef-client will create a temporary file based on the erb template and do a diff/comparison with the existing one.
If there is no change then the temporary file is discarded.
If there is change then the existing file is discarded and the newly generated file is put in place.
Only then chef-client will do a apache reload.

Cookbooks = recipes + attributes
Policy =  environments + roles + data bags
Runlist = Cookbooks + Policy

Chef-Client
Chef-Client pulls down the RunList from the chef server
Chef-Client ensures the Node complies with the policy in the RunList.

Set UP a Node & Write a Cookbook

Knife – is the tool that is used to interface between Chef-Server  and Workstation and AWS
Once the Knife establishes SSH to the Node it will pass – ChefServer-URL, validation_client_name, validation_key (its present in knife.erb file in .chef dir)
In the Node, it will install chef-client and chef-client will register itself with chef-server
Then Last process is chef-client process starts running to bring the node to the desired state.

Following Things are installed on a Node:
The Ruby Language
knife – command line tool admins
chef-client – Client Application.
ohai – System Profiler.

Ohai picks all info about the node to chef  server to index it for search .

Cookbook?
It is like a package for chef recipes.
It contains all the recipes, files, templates,libraries etc required to configure a portion of your infrastructure.

knife create cookbook apache
Go To chef-repo/cookbooks/apache/recipes/
vim default.rb
##
package “apache2” do
action :install                               :install is a value for the variable action.
end

service “apache2” do
action [:enable,:start]
end

cookbook_file “/var/www/index.html” do
source “index.html”
mode “0644”
end

##

put index.html in chef-repo/cookbooks/apache/file/defaut/
knife cookbook upload apache
Go to Console = in runlist add apache
@Node
#sudo chef-client

—————————————–

Template and Cross-Platform

Difference between Template and Cookbookfile ?
Cookbook file is for static files and Template is to dynamically insert values like in configuration file to use IP of host.

Modified apache cookbook for Cross-Platform from Above:

Go To chef-repo/cookbooks/apache/recipes/
vim default.rb
####
package_name = “apache2”
service_name = “apache2”
document_root = “/var/www”

if node[“platform”] == “centos”
package_name = “httpd”
service_name = “httpd”
document_root = “/var/www/html”
end

package package_name do
action :install                               :install is a value for the variable action.
end

service service_name do
action [:enable,:start]
end

#cookbook_file “#{document_root}/index.html” do
# source “index.html”
# mode “0644”
#end

template “#{document_root}/index.html” do
source “index.html.erb”
mode “0644”
end

####

Place a file called index.html.erb in chef-repo/cookbooks/apache/template/default/index.html.erb

vim index.html.erb

Hello from <%= node[“fqdn”]  %>!
This server has memory <%= node[“memory”][“total”].to_i %> MB memory

###########

More into Template

**Taking the above cookbook into consideration**

—->moving enclosed block to another file called – chef-repo/cookbooks/apache/attributes/default.rb <—–
package_name = “apache2”
service_name = “apache2”
document_root = “/var/www”

if node[“platform”] == “centos”
package_name = “httpd”
service_name = “httpd”
document_root = “/var/www/html”
end
—————————————————-<>———————————————————

package package_name do
action :install                               :install is a value for the variable action.
end

service service_name do
action [:enable,:start]
end

#cookbook_file “#{document_root}/index.html” do
# source “index.html”
# mode “0644”
#end

template “#{document_root}/index.html” do
source “index.html.erb”
mode “0644”
end

—————<Attribute File Contents > ————————–

#When its the attribute file , we can an elaborate code and change the above code as following

case node[“platform”]
when “ubuntu”
default[“package_name”] = “apache2”
default[“service_name”] = “apache2”
default[“document_root”] = “/var/www/html”
when “centos”
default[“package_name”] = “httpd”
default[“service_name”] = “httpd”
default[“document_root”] = “/var/www/html”
end
———————————————————————
—————-<New Recipe File would Look Like >——————-
#we will have to use the node construct here to call the variables defined in the attribute file.

package node[“package_name”] do
action :install                               :install is a value for the variable action.
end

service node[“service_name”] do
action [:enable,:start]
end

#cookbook_file “#{document_root}/index.html” do
# source “index.html”
# mode “0644”
#end

template “#{node[“document_root”]}/index.html” do
source “index.html.erb”
mode “0644”
end

———————————————————————
############

—————————————–

Template More Features :
Ohai can give all system specific info to input into the file but what if we needed to add some other info as attributes
Example:

vim chef-repo/cookbooks/apache/template/default/index.html.erb
<html>

<h1> Welcome to  <%= node[“company”] %>! </h1>       #This node attribute is not provided by OHAI , therefore we have to make other arrangements to replace it during chef-client execution.

<h2> Hello from  <%= node[“fqdn”] %>!  </h2>

</html>

SAVE.

now here the node[“fqdn”] will be replaced by chef-client using Ohai output.
To test if Ohai provides the values but in case of node[“company”] is not defined.

But

Create a cookbook called my-company

#Knife create cookbook my-company

Create an attribute file @ Location =  cookbooks/my-company/attributes/default.rb

default[“company”] = “MY_COMPANY”

The above attribute is of global scope, but if you need to bring under apache scope put like this

default[“apache”][“company”] = “MY_COMPANY”
Here “apache” is the node object and “company” is its value and “MY_COMPANY” is its nested value.
This method is followed to have more organised and categorises node objects in a sensible manner.

NOTE: Dependency between cookbooks is mentioned in cookbooks/apache/metadata.rb file
Contents :
depends   ‘another-cookbook-name’
Now the chef-client while executing this cookbook will be aware that this cookbook has a dependency on another cookbook which must be executed first.
We will have to use this Metadata mentioning of “another-cookbook” if its attributes is required and this “another-cookbook” is not in RunList.

——————–

Authenitication of API calls
The Node should have the following Files
1./etc/chef/client.pem (if not found then chef will create a client.pem using validation.pem)
2./etc/chef/validation.pem

Client.pem is necessary to sign API requests being made.

During each chef-client run it undergoes through 2 Phase

Phase I : Compile
Load all cookbooks
read each recipe

resource_collection = [
package[“apache2”],
service[“apaceh2”],
cookbook_file[“/var/www/index.html”]
]

Phase II : Execute
During the execution phase the above resource collection will be brought down to this phase.

It will take first resource
package[“apache2”] – is the resource in the desired state

Yes – do nothing
No – Take an action (say action :install) to bring the resource in the desired state.

———–

———–

# Node Object #

command – knife node list
to list out nodes

command – knife client list
to list out client – like validator.pem to create client.pem

Every Node have a uniq name within an organisation.

command – knife node show nodename
to describe a node.

Nodes are made up of attributes
Nodes are stored and indexed in chef server.

command – sudo ohai | less
It will list out the info of the Node when executed this command in Node.

command – knife search node “*:*” -a fqdn
to find all node within my organisation.

————–

What does chef-client do on each run?
1.gather ohai data (ie node objects)
2.authenticate to chef server
3.sync cookbooks
4.load cookbooks

Then it has 2 phase
1.Compile Phase
collects all attribute files like
{
“fqdn” : “ckdhged.dom”,
“package_name” : “apahce2”
or
“apache” : “package_name” : “apache2”    #if using nested values as discussed above.
}
2.Execute Phase

[package, service, template]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ROLE and ENVIRONMENTS

Creating a new Role

# cd chef-repo

Create a file for a Role
vim chef-repo/roles/webserver.rb
—————–
name “webserver”
description “Web servers”
run_list “recipe[my-company]”,”recipe[apache], “recipe[chef-client::cron]”
default_attributes ({
“company” => “Opscode”
})

#In recipe[apache] will include the default.rb file, if other we have to mentioned like in chef-client.
#This attribute is used to override the attribute mentioned in the attribut file of my-company cookbook above.

command – knife role from file webserver.rb
To upload a role to chef-server

command – knife role list
to list out all Roles within the organisation

command – knife role show webserver
to list out all details of Role webserver.

command – knife role delete webserver
to delete a role.

The version of the cookbook has to be mentioned in the metadata.rb file. (Its not getting update by its own.)

Environment comes under Organisation.

-Each environment may include attributes necessary for configuring the infrastructure in that environment.
– Production needs certain Yum Repos
– QA needs different Yum Repos
– The version of the chefcookbooks to be used.

To make use of Environment – Create a new directory called environment.

1. Dev environment :
#vim chef-repo/environment/dev.rb
——
name “dev”
description “For Development”
cookbook “apache” , “= 0.2.0”  #cookbook version values are set by us always.
——

#vim chef-repo/environment/production.rb
——
name “production”
description “For Production”
cookbook “apache” , “= 0.1.1”
——

command – knife environment from file dev.rb
command – knife environment from file production.rb
To upload environment.

command – knife environment list
to list out environment.

Always use the chef-client cookbook to make sure it runs chef-client on a regular basis on Node by setting it by cron.

—————————————————-

USING COMMUNITY COOKBOOKS INSTEAD OF USING OUR OWN COOKBOOK

command – knife cookbook site download chef-client 3.1.0 # site means to communicate with the community site and mentioning version is optional by default it will get us the latest cookbook.
This will download zip dir of the cookbook.

Check the metadata.rb file of all downloaded cookbooks because it shows the dependency.

Chef setup from Scratch

Launch a new instance for Chef Workstation

Go To : http://www.getchef.com

Click “Get Chef”

homechef

Fill up the form :

formchefLogin Credentials :  unni  / unni123456

Login Chef Console

consolechefNow the Chef Workstation Setup :

URL: https://learnchef.opscode.com/quickstart/workstation-setup/

Current OS – Ubuntu 12 and will use chef omnibus installer for quick setup.

curl -L https://www.opscode.com/chef/install.sh | sudo bash

echo 'export PATH="/opt/chef/embedded/bin:$PATH"' >> ~/.bash_profile && source ~/.bash_profile

Login to the Management Console -> https://preview.opscode.com/organizations
Select the Organization - minjarpoc
On Left pane select "Starter Kit" (The link only becomes active after selecting your organization.)
On right window click "Set up your Workstation"

 Click "Download Starter Kit"
scp -i sshkey.pem Downloads/chef-starter.zip  ubuntu@<ip>:/home/ubuntu/
ssh <ip>
sudo su -
mv /home/ubuntu/chef-starter.zip .
apt-get update
apt-get install unzip
unzip chef-starter.zip
cd chef-repo
ALL REMAINING COMMANDS SHOULD BE RUN WITHIN THIS DIR
#knife client list
minjarpoc-validator
[knife is a command-line tool that provides an interface between a local Chef repository and the Chef Server.]
Setup UP a CHEF REPO
apt-get install git
$ git init .
$ git add .
$ git commit -m "Initial commit"

Download the apt and apache2 community cookbooks using knife. In a terminal:
$ knife cookbook site install apt
$ knife cookbook site install apache2

Steps in creating a Simple Cookbook from Chef Docs
knife cookbook site install magic_shell
Creating a New Cookbook
knife cookbook create aliases
vim cookbooks/aliases/metadata.rb
Add the following line:
depends 'magic_shell'

If you're not familiar with Chef, this allows us to leverage LWRP (light weight resource provider) functionality of the magic_shellcookbook inside our aliases cookbook (allowing us to create aliases on our node).
vim chef-repo/cookbooks/aliases/recipes/default.rb
--------------
# Alias `h` to go home
magic_shell_alias 'h' do
  command 'cd ~'
end

# Alias `sites` to cd into apache
magic_shell_alias 'sites' do
  command "cd #{node['apache']['dir']}/sites-enabled"
end

# Set Nano as the default editor
magic_shell_environment 'EDITOR' do
  value 'nano'
end
---------------
Upload the Cookbooks

In order for our nodes to download these cookbooks, we need to upload them to the Enterprise Chef Server using knife. In a terminal:

$ knife cookbook upload --all
Recap

Our workstation is now set up. We now have:

  • Credentials setup and verified
  • Our chef-repo repository setup
  • The following recipes to use on a new node:
  1. recipe[apt] – via community
  2. recipe[apache2] – via community
  3. recipe[magic_shell] – via community
  4. recipe[aliases] – created ourselves

Converge to EC2 Node

Pre-requisite steps:

  1. Register for an Amazon AWS Account
  2. Setup and download your EC2 Key Pair
  3. Create an Access Key and Associated Secret Token on AWS
  4. Allow port 22 access on your EC2 security group

Existing Contents of chef-repo/.chef is

  • minjarpoc-validator.pem
  • unni.pem
  • knife.rb

Add the following lines to knife.rb file:

knife[:aws_access_key_id] = 'ACCESS_KEY_ID'
knife[:aws_ssh_key_id] = 'KEY_ID'
knife[:aws_secret_access_key] = 'SECRET_ACCESS_KEY'
#IMPORTANT NOTE : The aws_ssh_key_id is the name of your EC2 Key Pair.

apt-get install libxml2 libxml2-dev gcc build-essentials

To install the knife ec2 plugin using RubyGems, run the following command:
$ /opt/chef/embedded/bin/gem install knife-ec2

where /opt/chef/embedded/bin/ is the path to the location where the chef-client expects Knife plugins to be located. If the chef-client was installed using RubyGems, omit the path in the previous example.

Documentation with Knife EC2 Plugin : http://docs.opscode.com/plugin_knife_ec2.html

Create a Role and add apache to Runlist from the Chef Management Console

Role = testrole AMI = ami-8e987ef9 #knife ec2 server create -r “role[testrole]” -I ami-8e987ef9 –flavor m1.small -G unni-test-sg -x ubuntu -N MinjarMcAfee2 –region eu-west-1 -i /root/unni-minjarteam-ireland.pem

For testing purpose we can use apache cookbook. If using apache cookbook it will have an empty site-enabled directory hence we have to modify cookbook or after executing chef-client on node configure apache to have site-enabled file and restart apache.

Chef DSL Basics

A recipe is nothing but a list of resources !

Resource Syntax:

type "name" do
   attribute "value"
   action :type_of_action
end
Resource Example I (Directory Resource):

directory "/tmp/folder" do
  owner "root"
  group "root"
  mode 0755
  action :create
end
Resource Example II (Package Resource):

package "tar" do
  version "1.16.1-1"
  action :install
end

Some attributes are available to all resources, for example some attributes are used to send notifications to other resources, whereas others are used to set up conditional execution rules.

All actions have a default value. Only non-default behaviors of actions and attributes need to be specified. For example, the package resource’s default action is to install and the name of the package defaults to the “name” of the resource, therefore it is possible to write a resource block like this:

package "tar"

and the chef-client will use the default action (:install) to install the tar package.

Create New Cookbook and New Role

 

 

Logging Into the Chef-Workstation:
#ssh -i /sshkey root@22.22.22.22

Always work inside the chef repo directory.
#cd /root/chef-repo

Command to create new cookbook
#knife cookbook create mynewcookbook

Putup the ruby codes into the recipe of the new cookbook
#vim /root/chef-repo/cookbooks/mynewcookbook/recipes/default.rb

Upload the cookbook to the Platform
#knife cookbook upload mynewcookbook

Now start the instance using chef on EC2!!!
#knife ec2 server create -G default -I ami-33333 -f m1.small -S AWS_SSH_key_id -i /root/key -x root -r ‘role[rolename]’

[NOTE : If Chef-ec2 api is configured]