I have a touch of a love and hate relationship with bash. I spend tons of your time within the terminal, and bash is my default “programming language“. Sometimes I tell folks that find, grep and xargs run their infrastructure, and that they laugh and laugh until they realize I’m serious.
Picking up some Python may be a perfect choice for system administrators. it is also great for anyone who has got to affect anything during a terminal but doesn’t want to use bash, or has needs that are too complex for bash. Once a task goes beyond
[pastacode lang=”bash” manual=”find%20%24(pwd)%20-name%20%22*.txt%22%20%7C%20xargs%20-I%20%7B%7D%20echo%20%22do%20stuff%20with%20%7B%7D%22″ message=”” highlight=”” provider=”manual”/]it’s time to interrupt out the Python!
There are tons of advantages to using Python as your attend instruction language.
- Python has many nice libraries to assist out with just about anything. that has handling system operations, reading files, listing directories, writing for loops, checking for exit codes, and so on.
- Autocomplete with IDEs. Seriously. Who wants to possess to memorize anything?
- Robust testing suite if that’s your thing (and if it isn’t , you ought to consider making it your thing).
- The iPython console. It’s wonderful. It’s amazing. I LOVE IT.
- Python is out there on most systems, and if it isn’t you’ll catch on with Miniconda.
- Robust error checking with attempt to catch blocks.
- If you’re employed on different operating systems you’ll use Python libraries which will affect all that under the hood.
- Even if you’ve got no programming ability Python is a simple language to urge started with.
Let’s get Started
To get started, first you will need to either have Python installed or install it with Miniconda.
Check if you have iPython installed
[pastacode lang=”bash” manual=”which%20python%0Awhich%20ipython” message=”” highlight=”” provider=”manual”/]If both of those are successful, you’re in business! If you’ve got Python, but not iPython, you’ll need to install it. you’ll install it as a system package, but i actually recommend that you simply just install it with Miniconda.
Install Miniconda
Grab the installer for your OS here. I suggest getting the Python3 installation.
Then it’s just an easy installation.
[pastacode lang=”bash” manual=”bash%20Miniconda3-latest-Linux-x86_64.sh” message=”” highlight=”” provider=”manual”/]Follow the prompts and you will have Miniconda3 installed. Once you’ve got it installed you’ll be wanting to run an update, because this is often tech and in fact you would like to run an update. 😉
[pastacode lang=”bash” manual=”conda%20update%20conda%0Aconda%20config%20–add%20channels%20conda-forge%0Aconda%20update%20-y%20–all%0Aconda%20install%20-y%20ipython” message=”” highlight=”” provider=”manual”/]Troubleshooting
If you’ve got trouble installing any packages here are some tips.
- Run conda clean –all and check out again.
- Make sure you’re using the right channel.
- Run conda update -y –all
- Try to install as little as possible to your global conda space. Instead create environments for various tasks and projects, which we’ll get into next.
Create Environments with Conda
If you have ever used virtualenv, pipenv (is that a thing?), Rbenv, plenv, anyenv or any of the opposite various envs that have popped up over the years, this may sound very familiar to you. the thought is that different projects should have their own isolated software environments.
[pastacode lang=”bash” manual=”conda%20create%20-n%20my-project%20ipython%20package1%20package2%20package2″ message=”” highlight=”” provider=”manual”/]If you’re like me and like to have iPython readily availabe make sure you install it to any new environments!
Python Libraries for System Administration
Before we get into the examples let’s just list some handy packages along side their docs.
My attend package is that the os package. you’ll use it to list directories, check if files exist, check if symlinks exist, make directories, run system commands, get and set environmental variables, and more. It’s great!
My second package for running system commands that do not exist as handy python libraries is that the subprocess module.
The shutil has file operations that are not within the os library.
The pprint library prints out complex data structures with nice indentation.
The pytest library let’s you test your Python code, because let’s face it, nothing ever works correctly the primary (few) times.
How Do I Execute my Code?
Finally! Code!
When you’re using Python for system administration you’ll dive straight into the iPython console, or write scripts then execute them with python name-of-script.py.
If you favor to write down your scripts you’ve got numerous choices, and it’s truly a matter of private preference. i exploit PyCharm, which is paid, but Visual Studio Code and Atom are equally excellent free choices.
I find that it depends on what I’m performing on . Sometimes I just open up the iPython console and begin typing, and other times i want something more robust with tests and whatnot.
If you’re using either the iPython console or any of the editors I listed above, you’ll have autocomplete. Autocomplete is awesome! With iPython simply start typing your function and press tab to urge an inventory of potential functions you’ll want.
I cannot express how much I love autocomplete.
Get Help
You can go to any of the doc pages for any library, but if you know the name of either the library or the function you can bring it up in iPython.
You can bring up the help menu in most IDEs and text editors too, but that will be specific to your editor.
Examples
First you will need to import your packages
[pastacode lang=”bash” manual=”import%20os%0Aimport%20subprocess%0Aimport%20shutil%0Afrom%20pprint%20import%20pprint” message=”” highlight=”” provider=”manual”/]Here are some examples of common file and directory operations.
[pastacode lang=”bash” manual=”%23%20Check%20if%20a%20path%20exists%0Aos.path.exists(‘%2Fpath%2Fon%2Ffilesystem’)” message=”” highlight=”” provider=”manual”/] [pastacode lang=”bash” manual=”%23%20List%20the%20contents%20of%20a%20directory%0A%23%20This%20returns%20a%20list%0Adir_list%20%3D%20os.listdir()%0Afor%20item%20in%20dir_list%3A%0A%20%20%20%20print(item)” message=”” highlight=”” provider=”manual”/] [pastacode lang=”bash” manual=”%23%20Get%20the%20Absolute%20Path%20name%20of%20a%20file%20(file%20%2B%20current%20working%20dir)%0Aos.path.abspath(‘some-file’)” message=”” highlight=”” provider=”manual”/] [pastacode lang=”bash” manual=”%23Get%20the%20basename%20-%20returns%20file%0Aos.path.basename(‘%2Fpath%2Fto%2Ffile’)” message=”” highlight=”” provider=”manual”/] [pastacode lang=”bash” manual=”%23%20Split%20a%20directory%20path%20-%20platform%20independent%0Aos.path.split(os.getcwd())%0A%23%20Out%5B17%5D%3A%20(‘%2FUsers’%2C%20’jillian’)” message=”” highlight=”” provider=”manual”/] [pastacode lang=”bash” manual=”%23%20Split%20a%20directory%20path%20-%20platform%20independent%0Aos.path.split(os.getcwd())%0A%23%20Out%5B17%5D%3A%20(‘%2FUsers’%2C%20’jillian’)” message=”” highlight=”” provider=”manual”/] [pastacode lang=”bash” manual=”%23%20Check%20if%20a%20path%20is%20a%20symlink%0Aos.path.islink()” message=”” highlight=”” provider=”manual”/]Move files and directories around
[pastacode lang=”bash” manual=”%23%20Copy%20a%20directory%0A%23%20cp%20-rf%0Ashutil.copytree(‘src’%2C%20’dest’)” message=”” highlight=”” provider=”manual”/] [pastacode lang=”bash” manual=”%23%20Copy%20a%20file%0A%23%20cp%20-rf%0Ashutil.copyfile(‘file1’%2C%20’file2′” message=”” highlight=”” provider=”manual”/] [pastacode lang=”bash” manual=”%23%20Move%20a%20directory%0A%23%20mv%0Ashutil.move(‘src’%2C%20’dest’)” message=”” highlight=”” provider=”manual”/]Not everything is going to be available through python libraries, such as installing system libraries, so run a few system commands!
[pastacode lang=”bash” manual=”%23%20Run%20an%20arbitrary%20system%20command%0Acommand%20%3D%20%22echo%20’hello’%22%0Aresult%20%3D%20subprocess.run(command.split(‘%20’)%2C%20stdout%3Dsubprocess.PIPE%2C%20stderr%3Dsubprocess.PIPE)%0A%23Print%20the%20stdout%20and%20stderr%0Aprint(result.stdout)%0Aprint(result.stderr)” message=”” highlight=”” provider=”manual”/]Write to files!
[pastacode lang=”bash” manual=”%23%20Write%20to%20a%20file%20(and%20create%20it%20if%20it%20doesn’t%20exist)%0A%23%20echo%20%22hello%22%20%3E%20hello.txt%0Af%3D%20open(%22hello.txt%22%2C%22w%2B%22)%0Af.write(%22hello!%22)%0Af.close()” message=”” highlight=”” provider=”manual”/] [pastacode lang=”bash” manual=”%23%20Append%20to%20a%20file%0A%23%20echo%20%22hello%22%20%3E%3E%20hello.txt%0Af%20%3D%20open(%22hello.txt%22%2C%20%22a%2B%22)%0Af.write(%22hello%20again!%22)%0Af.close()” message=”” highlight=”” provider=”manual”/]
Write some tests!
Tests mostly work by using a function called assert, which is essentially saying make sure this is true and if not die loudly.
[pastacode lang=”bash” manual=”def%20test_system_command()%3A%0A%20%20%20%20%22%22%22Test%20the%20exit%20code%20of%20a%20system%20command%22%22%22%0A%20%20%20%20command%20%3D%20%22echo%20’hello’%22%0A%20%20%20%20result%20%3D%20subprocess.run(command.split(‘%20’)%2C%20stdout%3Dsubprocess.PIPE)%0A%20%20%20%20assert%20result.returncode%20%3D%3D%200″ message=”” highlight=”” provider=”manual”/]Put this function in a file called test_my_code.py and run as pytest test_my_code.py.
Wrap Up:
That’s it for my main tips and tricks for using Python as your go-to bash replacement. subsequent time you would like to write down a loop in bash, consider breaking out the iPython console and seeing what you’ll come up with instead!