Announcement - A library and tool for creating clusters on Linode

I installed the lct following the installation for python 2. when I then try to run the command to save the secret key i get an error message. Do you know why?

linodetool secret set personal-token |personal access token|

Traceback (most recent call last):
File "/bin/linodetool", line 9, in <module>
loadentrypoint('linode-cluster-toolkit==0.1.0', 'consolescripts', 'linodetool')() File "/usr/lib/python2.7/site-packages/pkgresources.py", line 378, in loadentrypoint
return getdistribution(dist).loadentrypoint(group, name) File "/usr/lib/python2.7/site-packages/pkgresources.py", line 2566, in loadentrypoint
return ep.load()
File "/usr/lib/python2.7/site-packages/pkgresources.py", line 2260, in load entry = import(self.modulename, globals(),globals(), ['name'])
File "/usr/lib/python2.7/site-packages/lct/init.py", line 1, in <module>
from .toolkit import Toolkit, ToolkitContext
File "/usr/lib/python2.7/site-packages/lct/toolkit.py", line 5, in <module>
from .linodeapi.apiservice import APIService File "/usr/lib/python2.7/site-packages/lct/linodeapi/apiservice.py", line 126
https://api.linode.com/v4/linode/instances/$linode_id/ips
^
SyntaxError: invalid syntax</module></module></module>

@sdennis69 I looked at the offending line…

https://github.com/pathbreak/linode-cluster-toolkit/blob/280257436105703c9a122e7ed111a72efa79adfc/lct/linodeapi/api_service.py#L126

It appears this code has a syntax error.

More importantly this code's last commit was July 2017. During this time the API was under "early release" status. We've made a whole slew of breaking changes since then. This code probably won't work entirely in its current state. It will need some commits to come back up to speed.

Two other things…

1. https://github.com/linode/linode-cli you might to check out our Python CLI. Its a great way to spin up lots of infrastructure. I personally use this tool in conjunction with stackscripts and a CM tool(Salt in my case) to spin up clusters.
2. I am going to edit your post to remove that personal access token. This can be used to grant access to your account.

Ah, thanks for the awesome feedback. I was hoping someone would point me in the right direction.

would you happen to have an example of how to use your CLI to spin up a cluster with stackscripts? I checked out the github but didnt see anything about how to do it. Can you provision clusters with images too? if you know where i can find more info, im all over it.

@sdennis69 Sure do.

Step 1: Install the thing

You need to install and configure the Linode CLI. The instructions can be found here https://github.com/linode/linode-cli#installation

As part of the installation you'll be asked for a PAT(personal access token). Instructions for generating this can be found here https://developers.linode.com/api/v4#section/Personal-Access-Token.

Step 2: Use the thing

Here is a really stripped down stackscript. All this does is lockdown ssh, setup a firewall, set the hostname,create a sudoer, and install my public key so I can ssh in. You can extend this to configure and install whatever you want.

The Stackscript

#!/bin/bash
# This block defines the variables the user of the script needs to input
# when deploying using this script.
#
#
#<UDF name="hostname" label="The hostname for the new Linode.">
# HOSTNAME=
#
#<UDF name="admin_username" label="the name of the remote admin user">
# ADMIN_USERNAME=
#
#<UDF name="admin_password" label="the password for remote admin user">
# ADMIN_PASSWORD=
#
#<UDF name="admin_public_key" label="the remote admin public key">
# ADMIN_PUBLIC_KEY=

set -x

# sets hostname
hostnamectl set-hostname $HOSTNAME

# This updates the packages on the system from the distribution repositories.
apt-get update
apt-get upgrade -y

#install a user and add keys
useradd -m -p $(openssl passwd -1 $ADMIN_PASSWORD) -s /bin/bash -G sudo $ADMIN_USERNAME

SSHDIR=/home/$ADMIN_USERNAME/.ssh
mkdir $SSHDIR

chmod 700 $SSHDIR
chown $ADMIN_USERNAME:$ADMIN_USERNAME $SSHDIR
touch $SSHDIR/authorized_keys
chown $ADMIN_USERNAME:$ADMIN_USERNAME $SSHDIR/authorized_keys
chmod 644 $SSHDIR/authorized_keys
echo $ADMIN_PUBLIC_KEY >> $SSHDIR/authorized_keys

#lock down ssh
sed -i 's/PermitRootLogin yes/PermitRootLogin no/g' /etc/ssh/sshd_config
sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/g' /etc/ssh/sshd_config
echo "AddressFamily inet" >> /etc/ssh/sshd_config
sudo systemctl restart sshd

#install and configure ufw
apt-get -y install ufw
ufw deny incoming
ufw allow OpenSSH
echo 'y' | ufw enable

The CLI Command in a bash envelope for iteration

for i in $(seq 5 $END) 
do 
    linode-cli linodes create --type g6-nanode-1 --region us-east --stackscript_id <your stackscript id> --stackscript_data "{\"hostname\":\"myhost$i\",\"admin_username\":\"myadminuser\",\"admin_password\":\"thepassword\",\"admin_public_key\":\"<a legit public key>"}" --image linode/ubuntu18.04 --label hostlabel$i --root_pass <root-password>
done

Here is a CLI command that boots 5 linodes using my stackscript. You'll notice two things.

1. A blob of json associated with the stackscript_data switch. These correspond the UDF fields in the stackscript
2. a stackscript_id. Once you've installed your stackscript through the manager this can be obtained with the following CLI command linode-cli stackscripts list --is_public false

Can you provision clusters with images too

You sure can. This would effectively mean image-izing a linode after you baked it with a stackscript or configured it by hand. All the info for creating and managing images can be found here https://www.linode.com/docs/platform/disk-images/linode-images/. In the CLI its as simple as specifying the image you want to boot. The only advantage over stackscripts is boot time. All the install commands and package manager runs are pre-baked into the image. One caveat, images are limited to 2GB after compression so sometimes it can be tricky to fit a fully configured OS. A workaround for this is booting from block storage volumes(https://www.linode.com/docs/platform/block-storage/boot-from-block-storage-volume/) The size limit on these is 10TB.

Debugging hot tip

you'll see I set the set -x flag in that stackscript. If you boot a linode with this and drop into the lish console you can watch the console output of the stackscript. It will be executed at the end of the boot sequence. This is very useful if steps are hanging or failing for some reason

Super hot tip

We are also collaborating on terraform support to increase our infrastructure as code capabilities. This isn't official yet, but you can get on the ground floor in these repos.

Linode Terraform Provider

https://github.com/displague/terraform-provider-linode\

Linode Go wrapper

https://github.com/chiefy/linodego