Once you’ve created your recipes and other objects for your chef cookbook, it should be uploaded to the Chef server.
There are two ways you can upload your cookbooks:
- Using knife cookbook sub-command – This helps you to manipulate your cookbooks
- Using knife upload sub-command – This helps you to upload all chef objects (including cookbooks, environments, nodes, etc.)
This tutorial explains how to use both of these sub-commands (knife cookbook and knife upload) with examples.
The following are covered in this tutorial:
- Upload Single Cookbook
- Upload ALL Cookbooks
- Freeze Cookbook Version (and Force Upload)
- Upload Cookbook with All Dependencies
- Increase Number of Concurrent Connections
- Specify Cookbook Directory Location
- Upload cookbooks using upload sub-command
- Upload ALL Cookbooks (using upload sub-command)
- Upload Roles, Nodes and Environments
- Upload the Whole Chef Repo (or) Upload Selective Items
- Force Upload Even When It’s not Changed Locally
- Perform Dry Run to Verify What’ll get Uploaded
- Recursive and Concurrency Options
- Chef Repo Mode Option
- Delete Cookbook from Chef Server
I. Knife Cookbook Sub-command
1. Upload Single Cookbook
Using the knife cookbook sub-command, you can perform all kinds of operations on your cookbooks including create, delete, upload, lock, view, etc.
To create a new Chef cookbook, refer to this: 12 Chef Knife Cookbook Command Examples
To upload a single cookbook, use the following syntax:
knife cookbook upload [cookbook-name]
Using knife cookbook sub-command, the following will upload a cookbook to the Chef server. The following will upload the cookbook called thegeekstuff.
Before you upload your cookbook, make sure you working directory is your local chef repository directory. In this example, it is ~/chef-repo. The following uploads prod-db cookbook.
# cd ~/chef-repo # knife cookbook upload prod-db Uploading prod-db [1.0.0] Uploaded 1 cookbook.
Note: When you do an cookbook upload, only the files that doesn’t already exist in the chef server will be upload. So, this is kind of an incremental upload.
2. Upload ALL Cookbooks
If you want to upload multiple cookbooks, use the -a option as shown below.
The following will upload all the available cookbooks on your local chef workstation from the current directory.
The following command will upload all your local cookbooks to chef server.
# knife cookbook upload -a Uploading prod-db [1.0.0] Uploading thegeekstuff [2.1.0] Uploaded all cookbooks.
Note: If you don’t want to upload all the files from the cookbooks, you can use the chefignore file to exclude certain file types. For example, you can exclude .git files (or other temporary files) by specifying them in the chefignore file.
You can also use –all instead of -a option.
knife cookbook upload --all
If you are execute the upload command outside your chef-repo directory, you’ll get the following ” Your private key could not be loaded from /etc/chef/client.pem” ERROR message.
# knife cookbook upload -a WARNING: No knife configuration file found WARN: Failed to read the private key /etc/chef/client.pem: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /etc/chef/client.pem> ERROR: Your private key could not be loaded from /etc/chef/client.pem Check your configuration file and ensure that your private key is readable
3. Freeze Cookbook Version (and Force Upload)
Once you’ve done all your work on a particular version of the cookbook, you can execute the following command, which will freeze that particular version.
# knife cookbook upload prod-db --freeze Uploading prod-db [1.0.0] Uploaded 1 cookbook.
In this example, the 1.0.0 version of prod-db cookbook is frozen. This means that if you want to modify prod-db, you should create a new version of the cookbook, and upload the new version.
If you are trying to update a frozen cookbook, you’ll get the following error message:
# knife cookbook upload prod-db Uploading prod-db [1.0.0] ERROR: Version 1.0.0 of cookbook prod-db is frozen. Use --force to override. WARNING: Not updating version constraints for prod-db in the environment as the cookbook is frozen. ERROR: Failed to upload 1 cookbook.
But, for some reason, if you want to update a fronzen cookbook, then use the –force option as shown below.
# knife cookbook upload prod-db --force Uploading prod-db [1.0.0] Uploaded 1 cookbook.
Note: During a cookbook upload, if you have empty directory on your local machine, they’ll not be uploaded to the Chef server. If you want empty directories to be upload, add some hidden files like .empty inside the empty directory. In that case, technically that directory is not empty, and it will be uploaded by the knife cookbook upload command.
4. Upload Cookbook with All Dependencies
When you are uploading a single cookbook, you can also use -d option (or) –include-dependencies to upload all the dependent cookbooks automatically as shown below.
knife cookbook upload thegeekstuff -d knife cookbook upload thegeekstuff --include-dependencies
5. Increase Number of Concurrent Connections
By default, the number of concurrent connections the knife command uses for uploading the cookbooks to the chef server is 10. This default value is good enough for most situation. But, if you are having 100s of large cookbooks, you can speed-up the upload process by increasing the concurrency value as shown below.
# knife cookbook upload -a --concurrency 20
6. Specify Cookbook Directory Location
You can also specify the location of your cookbook using the -o option. Here, you can specify more than one directory location as shown below. This will look for the cookbook to be uploaded in the specified directory.
# knife cookbook upload prod-db -o /root/chef-repo/cookbooks Uploading prod-db [1.1.0] Uploaded 1 cookbook.
When specifying multiple cookbook directories, separate them with colon as shown below.
# knife cookbook upload prod-db -o /root/chef-repo/cookbooks:/home/chef/cookbooks
You can also use –cookbook-path, which is same as the -o option
# knife cookbook upload prod-db --cookbook-path /root/chef-repo/cookbooks # knife cookbook upload prod-db --cookbook-path /root/chef-repo/cookbooks:/home/chef/cookbooks
II. Knife Upload Sub-command
Knife upload sub-command is used to upload all chef objects including cookbooks. For example, you can use this to upload cookbooks, roles, environment variables, etc.
7. Upload Cookbooks using Upload sub-command
To upload a specific cookbook, go to the chef-repo directory, specify the cookbook name along with the cookbook directory as shown below. This will upload prod-db cookbook from local machine to the Chef Server.
# knife upload cookbooks/prod-db Updated cookbooks/prod-db
Please note that this will do the upload only if anything is changed in the cookbook locally.
8. Upload ALL Cookbooks
First, cd to the chef repository directory, and execute the following, which will upload all the cookbooks from your local machine.
# cd chef-repo # knife upload cookbooks Updated cookbooks/prod-db Updated cookbooks/thegeekstuff
You can also upload cookbooks from anywhere inside the chef repository. For this, specify / in front of cookbooks directory name as shown below.
# knife upload /cookbooks Updated cookbooks/prod-db Updated cookbooks/thegeekstuff
In the above example, we are in the ~/chef-repo/nodes directory, and uploading all the cookbooks.
9. Upload Roles, Nodes and Environments
To upload the roles JSON files, execute the following:
knife upload roles
To upload the nodes JSON files, execute the following:
# knife upload nodes Created nodes/webserver.json
To upload environments, do the following:
knife upload environments
For the roles/nodes/environments, just like cookbooks, you can specify / in front of the directory, and execute the following command from anywhere inside your local chef repository directory.
knife upload /roles knife upload /nodes knife upload /environments
Also, you can upload a specific JSON file for either roles or nodes or environments by specifying the name of the JSON file as shown below.
knife upload roles/dba.json knife upload nodes/webserver.json knife upload environment/oracledb.json
10. Upload the Whole Chef Repo (or) Upload Selective Items
To upload your entire local chef repository to your server, cd to the chef-repo directory and execute the following.
# knife upload . Updated nodes/webserver.json Updated cookbooks/prod-db Updated cookbooks/thegeekstuff Updated roles/dba.json ..
The following is same as the above command, except this can be executed from anywhere inside your local chef repository directory.
knife upload /
Or, you selectively upload few object types. For example, the following will upload the cookbooks and nodes that match the given pattern.
# knife upload cookbooks/prod* nodes/web* Updated nodes/webserver.json Updated cookbooks/prod-db
11. Force Upload even when it’s not Changed Locally
Just like the cookbook sub-command, the upload sub-command also supports freeze and force option as shown below.
knife upload cookbooks/prod-db --freeze
Once a cookbook is frozen, you can upload the same version anymore as shown below.
# knife upload cookbooks/prod-db ERROR: cookbooks failed to write: Cookbook prod-db is frozen
Once a particular version of a cookbook is frozen, for some reason, if you want to update that without creating a new version, use the –force as shown below.
# knife upload cookbooks/prod-db --force Updated cookbooks/prod-db
Note: Please note that, –force command can be used not only to update a frozen version, but also to update all the files in the chef server, even when the file is not changed locally.
For example, the following updates only prod-db, as only that particular cookbook was changed locally.
# knife upload cookbooks Updated cookbooks/prod-db
But, when you give –force, all the local cookbooks will be uploaded even if they are not changed locally.
# knife upload cookbooks --force Updated cookbooks/prod-db Updated cookbooks/thegeekstuff ..
12. Perform Dry Run to Verify what’ll get Uploaded
Dry run is very helpful if you really want to see what local cookbooks are changed, which will be uploaded to the chef server. This way, you can make sure the upload command will upload what you are really expecting it to upload.
The following shows that this would upload only the prod-db and webserver cookbook. Nothing is really uploaded. This is just a dry run.
# knife upload cookbooks --force --dry-run Would update cookbooks/prod-db Would update cookbooks/webserver
13. Recursive and Concurrency Options
By default, the knife upload command, does all the upload recursively. If you don’t want recursive upload, you can specify no-recurse as shown below.
knife upload cookbooks --no-recurse
By default the number of concurrent connections the knife upload command will use is 10. If you have 100s of objects in your local machine that should be uploaded to the chef server, you can increase the concurrent connections to upload as shown below.
knife upload cookbooks --concurrency 20
14. Chef Repo-Mode Option
The default repo mode used by the knife upload command is either everything or hosted_everything depending on how your chef server setup.
The following are the three possible value for chef repo mode:
You can also use static if you are using only cookbooks, roles, environments and data bags as shown below.
# knife upload cookbooks --repo-mode static Updated cookbooks/thegeekstuff Updated cookbooks/webserver Updated cookbooks/prod-db
For your local chef server (Which is installed on your own server), if you try to use everything repo-mode, it will give the following error message:
# knife upload cookbooks --dry-run --repo-mode everything WARNING: You have repo_mode set to 'everything', but your chef_server_url looks like it might be a hosted setup. If this is the case please use hosted_everything or allow repo_mode to default
In this case, use the hosted_everything mode as shown below.
# knife upload cookbooks --repo-mode hosted_everything Updated cookbooks/thegeekstuff Updated cookbooks/webserver Updated cookbooks/prod-db
15. Delete Cookbook from Chef Server
If you’ve removed a cookbook from your local chef workstation, it will still be there on your chef server.
First, remove a cookbook from your local server.
cd ~/chef-repo rm -rf prod-db
Force upload all your local cookbooks
# knife upload cookbooks --force Updated cookbooks/thegeekstuff Updated cookbooks/webserver
But, if you view the cookbooks in the Chef server, you’ll still see the erased prod-db cookbook.
# knife cookbook list thegeekstuff 0.2.0 webserver 0.1.0 prod-db 0.1.0
So, to permanently delete it from the Chef server, use the –purge option as shown below.
# knife upload cookbooks --purge Deleted extra entry cookbooks/prod-db (purge is on)
After the purge, we will not see the prod-db cookbook on the remote server anymore as shown below.
# knife cookbook list thegeekstuff 0.2.0 webserver 0.1.0